protean-ordaprompt

Fail-closed Jev-style comparative classifier for OrdaPilot request routing. It is designed for Jev/OpenJEV-like NLI backends and compares all topic labels and session choices together, including new topic, ambiguous, and new session options. It uses calibrated score and margin gates, weighs session continuity against context cost and contamination, and abstains when uncertain.

This release provides the routing layer and backend adapter boundary. It does not bundle a Jev model or start a model server. The default adapter is deterministic and offline; a Jev/OpenJEV-like backend can be connected explicitly through the adapter contract. OpenRouter is optional and reserved for proposing provisional new labels after a high-confidence novelty signal. It is disabled by default.

Hermes Agent plugin. Installable via the curated plugin catalog entry plugin-catalog/protean-ordaprompt.yaml (in the hermes-agent repository) or directly:

hermes plugins install aska-digital/protean-ordaprompt --ref <40-char commit sha>
hermes plugins enable protean-ordaprompt

What it does

Given a classification request plus a candidate set (topic candidates and session candidates), the router makes ONE backend comparison call per surface, scores all candidates, and emits a closed-enum RoutingDecision (automatic, confirm, abstain, abstain_or_new_session) plus a hashes-only RoutingReceipt:

• automatic — high score AND high margin: route without asking. LOCKED OFF in this release (see Calibration below); the band is structurally present but unreachable while calibration_model_id is "none".
• confirm — in between: the router returns the ranked candidates and the caller asks.
• abstain — low confidence, novelty, ambiguity, or a new-session signal: the router declines to route and says why. Abstention is a first-class outcome, not a failure.

Session-utility reuse is deliberately stricter than topic utility: a session that wins on topic but is contaminated (different project hash) or near-tied never routes automatic.

Privacy disclosure

• No telemetry, no network, no background work. The package is Python stdlib only. The CLI runs once per invocation and exits.
• The optional OpenRouter backend ships DISABLED by policy (DisabledByPolicy is raised if anything tries to use it while disabled; even when enabled, batch scoring is gated off — the adapter permits only taxonomy proposals). Even when an operator explicitly enables it, the transport carries hashes and labels only — no free text, no prompts, no file contents, no paths. A privacy validator (assert_no_free_text) re-checks every outbound payload and receipt; violations raise PrivacyViolationError and nothing is written.
• Receipts are hashes + structured scores only (closed fields, closed domains, append-only). The receipt writer refuses free text. No prompt text, user content, or session state is ever serialized into a receipt, log, or the catalog metadata.
• This repository contains only synthetic fixtures (generated by eval/fixturegen.py) and demo request/candidate JSON with placeholder IDs. No hosted prompt data, no real session transcripts, no user state.

Capability boundary

• Declares no tools, no hooks, no middleware, no settings (config_schema: {}). It does not hook the session loop, does not read the Hermes profile config, and does not write anything outside the receipts directory an operator explicitly passes with --receipts-dir.
• No Hermes core changes are required or made. Integration is one catalog YAML entry plus this self-contained source tree.
• The routing surface is the CLI only: python3 -m ordaprompt_router.cli classify and route-session. Callers that want automatic routing must wire it themselves — and it will refuse (abstain/locked) until the calibration gate below passes.

Required configuration

None. The plugin runs with defaults (RouterConfig()) with the OpenRouter backend disabled and the automatic band locked. Optional, explicit, per-invocation JSON:

• --config router-config.json — thresholds, legacy fallback, backend declaration.
• --calibration calibration.json — a CalibrationModel document; shipping eval/calibration.json records the locked state (model_id: "none", active false).
• --enable-openrouter — explicitly enables the (hashes-only) OpenRouter backend for that invocation. Default OFF everywhere.

Supported platforms

Any platform running Python 3.11+ (macOS, Linux). Stdlib only — no Python dependencies to install. The catalog entry declares platforms: [] (all platforms).

Failure and abstention behavior (fail-closed)

Condition	Behavior
OpenRouter used while disabled	DisabledByPolicy raised; nothing sent, nothing written
Malformed request / candidate set	exit code 2, schema error, nothing written
Sub-threshold score or margin	band abstain (never automatic)
Ambiguity: topic and session winners disagree	new_session sentinel wins; no route
Contaminated session (project hash mismatch)	session never promoted
Receipt fails privacy/schema validation	exit code 3, receipt rejected, nothing written
Any unexpected backend error	BackendError; the decision is abstain, not a guess

The rule is uniform: when the router cannot prove the safe choice, it abstains and reports why; it never guesses.

Calibration lock (release state)

The automatic band is LOCKED until calibration unlock criteria pass on real feedback data: expected calibration error (ECE) ≤ 0.05 per surface. The synthetic harness honestly reports ECE 0.1330 (topic) / 0.0635 (session) — above the gate — so production ships calibration_model_id="none", active=false, and the harness prints calibration unlocked=False production_model=none. Until real-data calibration passes, routing outcomes are confirm, abstain, or abstain_or_new_session — never automatic.

Evaluation

python3 eval/harness.py --self-test   # 60 internal checks
python3 eval/harness.py               # 400 synthetic fixtures (267 tuning / 133 holdout)

Current holdout: topic accuracy 1.0000, automatic-band precision 1.0000, contamination 0.0000, abstain rate 0.2707 — on synthetic data only, which is exactly why the automatic band stays locked.

Tests

python3 -m unittest discover -s test -p 'test_*.py' -v
python3 eval/negative_probes.py     # 5/5 fail-closed negatives

Layout

ordaprompt_router/   the package (schemas, adapter, router, receipts, cli)
eval/                offline harness, fixture generator, negative probes, calibration
test/                unittest smoke tests
plugin.yaml          Hermes plugin manifest (manifest_version 1, api_version 1)