protean-ordaprompt

Fail-closed Jev-style comparative classifier for OrdaPilot request routing. It is designed
for Jev/OpenJEV-like NLI backends and compares all topic labels and session choices together,
including new topic, ambiguous, and new session options. It uses calibrated score and
margin gates, weighs session continuity against context cost and contamination, and abstains
when uncertain.

This release provides the routing layer and backend adapter boundary. It does not bundle a
Jev model or start a model server. The default adapter is deterministic and offline; a
Jev/OpenJEV-like backend can be connected explicitly through the adapter contract. OpenRouter
is optional and reserved for proposing provisional new labels after a high-confidence novelty
signal. It is disabled by default.

Hermes Agent plugin. Installable via the curated plugin catalog entry
plugin-catalog/protean-ordaprompt.yaml (in the hermes-agent repository) or directly:

hermes plugins install aska-digital/protean-ordaprompt --ref <40-char commit sha>
hermes plugins enable protean-ordaprompt

What it does

Given a classification request plus a candidate set (topic candidates and session
candidates), the router makes ONE backend comparison call per surface, scores all
candidates, and emits a closed-enum RoutingDecision (automatic, confirm, abstain,
abstain_or_new_session) plus a hashes-only RoutingReceipt:

automatic — high score AND high margin: route without asking. LOCKED OFF in this
release (see Calibration below); the band is structurally present but unreachable
while calibration_model_id is "none".
confirm — in between: the router returns the ranked candidates and the caller asks.
abstain — low confidence, novelty, ambiguity, or a new-session signal: the router
declines to route and says why. Abstention is a first-class outcome, not a failure.

Session-utility reuse is deliberately stricter than topic utility: a session that wins on
topic but is contaminated (different project hash) or near-tied never routes automatic.

Privacy disclosure

No telemetry, no network, no background work. The package is Python stdlib only.
The CLI runs once per invocation and exits.
The optional OpenRouter backend ships DISABLED by policy (DisabledByPolicy is
raised if anything tries to use it while disabled; even when enabled, batch scoring is
gated off — the adapter permits only taxonomy proposals). Even when an operator
explicitly enables it, the transport carries hashes and labels only —
no free text, no prompts, no file contents, no paths. A privacy validator
(assert_no_free_text) re-checks every outbound payload and receipt; violations raise
PrivacyViolationError and nothing is written.
Receipts are hashes + structured scores only (closed fields, closed domains,
append-only). The receipt writer refuses free text. No prompt text, user content, or
session state is ever serialized into a receipt, log, or the catalog metadata.
This repository contains only synthetic fixtures (generated by eval/fixturegen.py)
and demo request/candidate JSON with placeholder IDs. No hosted prompt data, no real
session transcripts, no user state.

Capability boundary

Declares no tools, no hooks, no middleware, no settings (config_schema: {}).
It does not hook the session loop, does not read the Hermes profile config, and does
not write anything outside the receipts directory an operator explicitly passes with
--receipts-dir.
No Hermes core changes are required or made. Integration is one catalog YAML entry
plus this self-contained source tree.
The routing surface is the CLI only: python3 -m ordaprompt_router.cli classify and
route-session. Callers that want automatic routing must wire it themselves — and it
will refuse (abstain/locked) until the calibration gate below passes.

Required configuration

None. The plugin runs with defaults (RouterConfig()) with the OpenRouter backend
disabled and the automatic band locked. Optional, explicit, per-invocation JSON:

--config router-config.json — thresholds, legacy fallback, backend declaration.
--calibration calibration.json — a CalibrationModel document; shipping
eval/calibration.json records the locked state (model_id: "none", active false).
--enable-openrouter — explicitly enables the (hashes-only) OpenRouter backend for
that invocation. Default OFF everywhere.

Supported platforms

Any platform running Python 3.11+ (macOS, Linux). Stdlib only — no Python dependencies
to install. The catalog entry declares platforms: [] (all platforms).

Failure and abstention behavior (fail-closed)

| Condition | Behavior |
| --- | --- |
| OpenRouter used while disabled | DisabledByPolicy raised; nothing sent, nothing written |
| Malformed request / candidate set | exit code 2, schema error, nothing written |
| Sub-threshold score or margin | band abstain (never automatic) |
| Ambiguity: topic and session winners disagree | new_session sentinel wins; no route |
| Contaminated session (project hash mismatch) | session never promoted |
| Receipt fails privacy/schema validation | exit code 3, receipt rejected, nothing written |
| Any unexpected backend error | BackendError; the decision is abstain, not a guess |

The rule is uniform: when the router cannot prove the safe choice, it abstains and
reports why; it never guesses.

Calibration lock (release state)

The automatic band is LOCKED until calibration unlock criteria pass on real feedback
data: expected calibration error (ECE) ≤ 0.05 per surface. The synthetic harness
honestly reports ECE 0.1330 (topic) / 0.0635 (session) — above the gate — so production
ships calibration_model_id="none", active=false, and the harness prints
calibration unlocked=False production_model=none. Until real-data calibration passes,
routing outcomes are confirm, abstain, or abstain_or_new_session — never
automatic.

Evaluation

python3 eval/harness.py --self-test   # 60 internal checks
python3 eval/harness.py               # 400 synthetic fixtures (267 tuning / 133 holdout)

Current holdout: topic accuracy 1.0000, automatic-band precision 1.0000, contamination
0.0000, abstain rate 0.2707 — on synthetic data only, which is exactly why the
automatic band stays locked.

Tests

python3 -m unittest discover -s test -p 'test_*.py' -v
python3 eval/negative_probes.py     # 5/5 fail-closed negatives

Layout

ordaprompt_router/   the package (schemas, adapter, router, receipts, cli)
eval/                offline harness, fixture generator, negative probes, calibration
test/                unittest smoke tests
plugin.yaml          Hermes plugin manifest (manifest_version 1, api_version 1)

README from the reviewed commit 60e89af; it updates when the author re-pins.

protean-ordaprompt

About