claim-check

claim-check — receipts for agent claims

Agents say "all tests pass ✅" when pytest exited 1. claim-check catches that.

It records every tool result in a local evidence ledger, then checks the success claims in the agent's
final reply against what actually happened, and appends a receipt:

---
**claim-check receipts**
✅ file: “I created `calc.py` and all tests pass.” — write_file /repo/calc.py (tool #1)
❌ tests: “I created `calc.py` and all tests pass.” — `pytest -q` exit 1 (tool #2)

A prompt or skill can't do this: it asks the same model that made the claim to check itself.
claim-check reads the real tool results through Hermes hooks and gives the same verdict every time.

Claim types

| Kind | Example claim | Evidence it checks |
|---|---|---|
| file | "I created src/app.py" | write_file/patch to that path (or a shell write) succeeded |
| tests | "all tests pass" | latest test command (pytest, npm test, go test, cargo test, …) exited 0 and no edits since |
| command | "ran make build successfully" | that command (or latest build) exited 0 |
| git | "committed and pushed" | git commit / git push exited 0 |
| install | "installed requests" | a pip/npm/uv/apt… install naming it exited 0 |
| server | "server is running on port 8000" | a start/probe command mentioning it exited 0 |

Verdicts: ✅ verified · ⚠️ unsupported (no evidence, unknown exit, or stale) · ❌ contradicted (the tool result says otherwise).

Install

hermes plugins install claim-check

Use

• Automatic: verdicts get appended to replies that make claims.
• /receipts: shows this session's ledger and the last verdicts.
• Tool claimcheck_report: the agent can check its own ledger before saying it's done.

Settings (plugins.entries.claim-check.settings)

| key | default | |
|---|---|---|
| mode | annotate | annotate · warn_only (footer only when something is ⚠️/❌) · off |
| nudge | false | When a claim is contradicted, inject a correction note into the next turn so the agent fixes it |

Disclosure

• Hooks: post_tool_call (sees every tool's name, args and result), transform_llm_output (appends the footer;
  streaming is not held), pre_llm_call (only when nudge is on).
• Data: writes a per-session JSONL ledger under <HERMES_HOME>/plugin-data/claim-check/ledgers/. Each entry
  keeps the tool name, the file path or command (first 500 chars), the exit status, and the last 300 chars of output.
  Capped at 500 entries per session. Nothing is read outside this directory.
• No network calls, LLM calls, subprocesses, credentials, telemetry, background processes or self-updater.
  Pure Python stdlib.

Develop

python3 -m pytest -q
hermes plugins validate . --install-deps

MIT © 2026 Crypto Bandit (cryptobandit)

README from the reviewed commit 4a0e655; it updates when the author re-pins.