Lossless context memory for Hermes Agent.

Bounded context, unbounded memory. Nothing is ever lost.

LCM-X replaces one-shot active-context compression with a SQLite-backed, DAG-based context engine. It keeps the live prompt bounded, preserves raw messages, and gives the agent tools to recover exact detail after compaction.

The project name is LCM-X. From v0.24.0 the plugin is named hermes-lcm-x and the runtime context engine lcm-x (#471); this is a breaking change for existing installs — see Migrating from hermes-lcm. Unchanged: lcm.db, the lcm: config block, LCM_* environment variables, the lcm_* tools, /lcm, and the bundled skill name hermes-lcm.

Based on the LCM paper by Ehrlich & Blackman (Voltropy PBC, Feb 2026). Inspired by lossless-claw for OpenClaw. For an interactive visualization of the LCM idea, see losslesscontext.ai.

Table of contents

• What it does
• Project status
• LCM vs built-in compression
• Quick start
• Commands and tools
• Recall skill and policy
• Configuration
• Retrieval contract
• OpenClaw/lossless-claw import
• Troubleshooting
• Architecture
• How it works
• Documentation
• Development
• Contributing
• License

What it does

Hermes Agent's built-in compressor is a practical continuity layer: when the prompt crosses its configured threshold, it prunes older tool results, asks an auxiliary model to summarize the middle/older conversation, and rebuilds the active prompt from that summary plus a protected recent tail. The original session rows can still live in Hermes state.db and remain searchable through host tools such as session_search, but the model's active context no longer contains the compacted turns verbatim or a structured drill-down path back to them.

LCM-X instead:

1. Persists messages in a plugin-local SQLite store with FTS metadata.
2. Compacts older context into depth-aware summary nodes.
3. Condenses summaries into a hierarchical DAG as they accumulate.
4. Assembles active context from system prompt, highest-value summaries, and the protected fresh tail.
5. Provides recall tools so agents can search, inspect, and expand compacted material without flooding the main prompt.

Nothing is lost in normal operation. Raw messages stay recoverable in bounded pages, summaries retain source lineage, and oversized externalized payloads keep stable refs for later expansion.

Core capabilities:

• SQLite message store - preserves raw messages before compaction
• Summary DAG - builds depth-aware summary nodes over compacted history
• Bounded recovery - pages raw messages, child summaries, and externalized payloads instead of dumping everything into the prompt
• Agent tools - lcm_grep, lcm_recall, lcm_query_state, lcm_compute, lcm_compile_evidence, lcm_evidence_pack, lcm_retrieve, lcm_recent, lcm_load_session, lcm_describe, lcm_expand, lcm_expand_query, lcm_status, lcm_inspect, and lcm_doctor
• Source-aware retrieval - filters raw rows and summaries by descendant source lineage
• Session controls - ignore noisy sessions or keep sessions read-only with glob patterns
• Large payload controls - externalize oversized tool/media/raw payloads and protect SQLite from inline media-ish base64 blobs
• Sensitive-pattern controls - two independent layers: opt-in durable redaction of API keys, bearer tokens, passwords, and private keys before LCM stores or summarizes them (off by default — the durable store is lossless), and automatic protection of the copies sent to known cloud embedding providers
• Diagnostics - runtime health, database checks, optional /lcm slash commands, backup-first repair/rotate paths

Beyond the core loop, three opt-in (default-off) feature families extend LCM from a compression layer into a memory system: large-output externalization and context-budget controls (giant tool results move to recoverable refs instead of crowding the prompt), temporal memory (day/week/month rollups plus natural-time recall through lcm_recent), and semantic retrieval (embedding-backed lcm_grep semantic/hybrid modes with Voyage, OpenAI-compatible endpoints, Ollama, or in-process FastEmbed). See the Feature overview for what each family does and why, and Agent configuration profiles for copy-paste setups per agent type.

Project status

The latest stable release is v0.24.1@98ac62fee5316cdf461e77f28951720f523489f1 (plugin hermes-lcm-x, engine lcm-x; the rename shipped in v0.24.0, #471). It carries the v0.23.2 lossless default (durable sensitive-pattern redaction is opt-in; cloud-embedding privacy is an independent flag that transforms only the provider-bound copy — see "Sensitive-pattern redaction" below), the PEM private-key redaction hardening, loud privacy-policy failures on the recall path, and the rc-first release gauntlet, while preserving the 15-tool surface, schema-5 core store, and default-off experimental features.

Stable release identity and the continuing main development line are separate proof planes; do not describe an arbitrary main checkout as the installed stable product.

main and the v0.24.1 tag identify themselves as hermes-lcm-x v0.24.1 (15 tools) — the #488 replay-reconciliation patch (#510, #515; #517 rollback-readable proof) on top of the v0.24.0 release tag, which identifies itself as hermes-lcm-x v0.24.0 (15 tools). The next identity bump is the forward bump for the next release, never a restamp of any past commit's own recorded identity (#385 fixed the earlier drift).

Eva has accepted exact stable v0.23.1 with hosted voyage-4-large, 1024-dimensional float32 summary vectors under one privacy-bound identity. That evidence establishes runtime safety for Eva only, not fleet, customer, Teams, answer-accuracy, or universal benchmark readiness.

The active finite evaluation is #341: an answer-blind, exact-stable LongMemEval retrieval-provenance audit. It keeps product and benchmark-instrument identities separate and cannot change product retrieval behavior. See Current project state, the roadmap, and Benchmark methodology for current identities, gaps, and proof boundaries.

LCM vs built-in compression

Hermes core may persist original conversation history in state.db before built-in compression rewrites the active prompt. Built-in compression can still be lossy in the active context, but previous content may be recoverable later through host-level history tools such as session_search.

LCM-X is different because recall is part of the active context engine:

• plugin-local store and DAG built specifically for drill-down
• current-session retrieval through LCM tools, not an auxiliary cross-session search step
• explicit source-lineage and session-boundary rules

Position LCM around retrieval quality, autonomy, and drill-down behavior. Do not claim that Hermes core has no persisted record of pre-compression history.

Quick start

Prerequisites

• Hermes Agent
• Python 3.11+
• No required third-party runtime dependencies

tiktoken is used if available; otherwise LCM falls back to character-based token estimates. regex is used if available to apply timeouts to message ignore patterns; without it, message-level regex filtering is disabled with a warning rather than running unbounded stdlib re matches.

Install the plugin

Canonical install path: clone LCM-X as a general user plugin into plugins/hermes-lcm-x.

git clone https://github.com/electricsheephq/lcm-x \
  ~/.hermes/plugins/hermes-lcm-x

For a profile-specific install:

git clone https://github.com/electricsheephq/lcm-x \
  ~/.hermes/profiles/myprofile/plugins/hermes-lcm-x

From an existing checkout, install a symlink:

./scripts/install.sh

# Optional profile-aware install:
HERMES_PROFILE=myprofile ./scripts/install.sh

Run scripts/install.sh even when the checkout already lives at the canonical plugin path. It leaves that checkout in place and exposes the bundled hermes-lcm skill in the matching global/profile skills/ directory. The installer preflights both paths and refuses conflicts before creating links. When it finds a pre-0.24 install (plugins/hermes-lcm, or a config.yaml that enables hermes-lcm / selects context.engine: lcm), it prints the migration steps below. It never edits config.yaml and never deletes the old directory.

Activate it

The plugin has two names:

• plugin manifest name: hermes-lcm-x
• runtime context engine name: lcm-x

Both must be configured:

plugins:
  enabled:
    - hermes-lcm-x

context:
  engine: lcm-x

Restart Hermes after changing plugin or context-engine config.

Verify it loaded

Run:

hermes plugins list

Expected signals:

• plugin list includes hermes-lcm-x
• selected context engine is lcm-x
• tool list includes lcm_grep, lcm_recall, lcm_recent, lcm_load_session, lcm_describe, lcm_expand, lcm_expand_query, lcm_status, lcm_inspect, and lcm_doctor
• the normal available-skills index includes hermes-lcm; current hosts can also resolve the explicit plugin-qualified skill hermes-lcm-x:hermes-lcm

On the main line, typical output is:

Plugins (1):
  ✓ hermes-lcm-x v0.24.1 (15 tools)

Provider Plugins:
  Context Engine: lcm-x

Older v0.23.x stable tags instead report hermes-lcm v0.23.x (15 tools) and engine lcm; verify the loaded commit before treating either string as release proof.

For source checkouts, lcm_status, /lcm status, lcm_inspect, lcm_doctor, and /lcm doctor also report the loaded plugin path and best-effort git identity: plugin_git_commit, plugin_git_branch, and plugin_git_dirty.

If startup logs say LCM tools are available through context-engine schemas or mention the Path B fallback, that is expected on older Hermes hosts such as Hermes Agent v0.16. All 15 lcm_* tools remain available through the context-engine path; standalone plugin-registry registration is not required there.

Update it

If you cloned directly into the plugin directory:

cd ~/.hermes/plugins/hermes-lcm-x && git pull --ff-only

For a profile-specific install:

cd ~/.hermes/profiles/myprofile/plugins/hermes-lcm-x && git pull --ff-only

A clone from v0.23.x or earlier still lives at plugins/hermes-lcm. Stop Hermes, update that clone in place (Hermes matches the manifest name, not the directory), then make the config change in Migrating from hermes-lcm before you start Hermes again:

cd ~/.hermes/plugins/hermes-lcm && git pull --ff-only
# profile-specific:
cd ~/.hermes/profiles/myprofile/plugins/hermes-lcm && git pull --ff-only

If you installed a symlink from a separate checkout, update that checkout and rerun the installer (it is idempotent):

git -C /path/to/lcm-x pull --ff-only && /path/to/lcm-x/scripts/install.sh

A catalog install (hermes plugins install hermes-lcm-x) updates only through the reviewed catalog pin: hermes plugins update hermes-lcm-x. LCM-X ships no self-updater.

Restart Hermes after updating.

Before upgrading to v0.23.1, resolve the effective database path from LCM_DATABASE_PATH or lcm_status. For an online backup, set LCM_ENABLE_SLASH_COMMAND=true in the old runtime's launch environment, restart that runtime so /lcm is registered, then run /lcm backup. Otherwise stop every SQLite writer and copy that configured database with its WAL/SHM companions as one cold set. Check out the exact stable tag/SHA, restart Hermes, and verify the loaded path, version, database path, schema 5, and all 15 tools. No manual core migration is required for the upgrade itself. Existing cloud vectors from a pre-v0.23.1 identity require privacy-policy warmup, an identity check from the warmup/profile output, and the finite monotonic dry-run/apply loop in the operator guide before semantic coverage is complete.

Cloud embeddings are a separate opt-in. Provider-bound copies use the configured sensitive-pattern catalog by default, while durable messages and FTS remain raw. LCM_EMBEDDING_PRIVACY_ENABLED=false explicitly opts out and binds vectors to the distinguished privacy:off revision. When provider-copy privacy is on, the catalog must be nonempty and contain only supported pattern names (any unknown name fails warmup and dispatch loudly). Dry-run the backfill first; raw-chunk cloud backfill also requires explicit raw-text consent. See the operator upgrade and privacy notes before enabling embeddings.

Migrating from hermes-lcm (v0.23.x and earlier)

BREAKING in v0.24.0. The plugin manifest is now hermes-lcm-x and the context engine lcm-x (#471). Hermes matches plugins.enabled against the manifest name, not the install directory, so a config that enables only hermes-lcm stops loading LCM-X after an update. Run exactly one LCM copy, and change the config while Hermes is stopped:

1. Stop Hermes (every process that uses this profile).

2. Update the code: git pull --ff-only in the existing clone (it may stay at plugins/hermes-lcm; see Update it), or install a new checkout with scripts/install.sh.

3. Config: in plugins.enabled, replace hermes-lcm with hermes-lcm-x, and set context.engine: lcm-x:

   plugins:
     enabled:
       - hermes-lcm-x
   context:
     engine: lcm-x
   

   • If a separate older copy is still installed (for example v0.23.x left at plugins/hermes-lcm next to a new plugins/hermes-lcm-x), do not keep both names enabled: both copies would load. LCM-X detects that state, logs Another LCM generation is already loaded, and stays inert so only one copy writes lcm.db, but LCM-X itself is then not running.
   • Keeping hermes-lcm listed next to hermes-lcm-x is harmless only when plugins/hermes-lcm is this same checkout (an in-place clone, or the link scripts/install.sh reuses): there is one physical copy.
   • The legacy context.engine: lcm still selects LCM-X through an alias, but logs a once-per-process DEPRECATED LCM-X config warning and reports an identity_migration field in lcm_status and a warn check in lcm_doctor naming the exact change.

4. Start Hermes and verify: hermes plugins list shows hermes-lcm-x as enabled and hermes-lcm as not enabled (an old directory kept for rollback may still be listed), and the log shows LCM plugin loaded — lossless context management active.

5. Later, by hand: after verifying, remove a separate old directory (and its skills/hermes-lcm link). Keep it until then for rollback. The installer never edits config.yaml or deletes anything.

If the config is not updated: Hermes logs Context engine 'lcm' not found — falling back to built-in compressor and runs without LCM-X. The existing lcm.db is untouched, but turns handled while Hermes runs without LCM-X (built-in compressor) are not in lcm.db and their compacted content may not be recoverable. Therefore update the config before restarting Hermes after the update.

Managed fleets (PCS / managed-plugin payloads) stay pinned to v0.23.x until their config stages hermes-lcm-x in plugins.enabled. That staging is a separate fleet-migration change owned outside this repository.

Hermes plugin catalog and install-scanner notes

The catalog build follows the plugin-catalog admission rules: no self-updater (scripts/update.sh was removed; catalog installs update only through hermes plugins update hermes-lcm-x), provides_hooks matches what register() registers (pre_llm_call, post_llm_call, subagent_start, subagent_stop, all through ctx.register_hook), and hermes plugins validate reports the install scanner verdict safe: zero dangerous and zero caution (high-severity) findings. Planted-secret fixtures build their PEM markers by string concatenation and the bench env captures avoid piping the env listing, so test data is not read as a real key or an environment dump.

The scanner still lists informational (medium/low) findings; none affects the verdict:

• embedded_private_key, hardcoded_secret, openai_key_leaked — planted, non-functional fixtures in tests/ that prove redaction and privacy controls.
• python_subprocess, hex_encoded_string, uv_run, path_traversal — benchmark and release-gauntlet harnesses under bench/ and benchmarks/ that drive local Hermes runs; not loaded by the plugin.
• git_clone, unpinned_pip_install, dump_all_env — install/contributor docs and CI (pip install pytest numpy), plus one doc describing an env dump; documentation only.
• eval_string, proc_access, system_passwd_access — test assertions that such strings are rejected or handled.
• oversized_file, oversized_bundle, too_many_files — the repository ships docs images, large test modules, and benchmark tooling with the plugin.
• agent_config_ref, hermes_config_ref, hardcoded_ip_port (low) — maintainer skills and config examples that reference AGENTS.md, config.yaml, or loopback endpoints.

hermes plugins validate also warns that the 15 provides_tools are not registered by register(): LCM-X serves its tools through the context-engine tool schemas (ContextEngine.get_tool_schemas), which is the intended path.

Commands and tools

Agent tools

Use these tools for current-session recall after compaction. Use Hermes session_search for earlier separate sessions or broad cross-session history outside the LCM database.

Tool	Use
lcm_grep	Search current-session raw messages and summaries. Opt into `content_scope='externalized'
lcm_recall	Search the entire memory across ALL conversations and all time by meaning. Fuses full-text, summary-vector, and chunk-vector arms with RRF, then applies a soft current-conversation (scope_bias) and recency prior. Returns bounded summary and verbatim-excerpt hits with lcm_expand handles; works FTS-only when embeddings are disabled.
lcm_query_state	Query the opt-in V4 assertion sidecar for bounded current or historical facts, preferences, recommendations, commitments, actions, and status. Every result carries an exact message ref/span/quote; conflicts stay visible.
lcm_compute	Execute supported dates, distinct counts, compatible-unit sums, directed/absolute differences, ordering, and latest-state operations over exact cited evidence. Planning, arithmetic, and trace verification are dependency-free and provider-neutral; ambiguous or incomplete inputs fail closed.
lcm_compile_evidence	Validate one bounded provider-neutral semantic proposal against exact stored refs and return a compact evidence brief with explicit sufficiency state and an optional canonical computation. It never returns final prose or treats model claims as finite-coverage proof.
lcm_evidence_pack	Hydrate and validate bounded baseline exact refs in the same lcm.db, repair only unique in-window quote spans, preserve occurrence/observation time separation, and optionally emit an immutable canonical computation trace without prose.
lcm_retrieve	Opt-in bounded controller for one continuous answerer tool turn. It tracks typed evidence slots, permits at most three targeted calls to existing retrieval tools, accepts only exact observed refs, caps the evidence context, and can finish through lcm_compute. It persists evidence views and traces, never final prose.
lcm_recent	Retrieve recent summaries by natural UTC period, preferring ready rollups and transparently falling back to time-bounded leaf summaries.
lcm_load_session	Load one ordered raw-message transcript page for an explicit session_id. Continues with after_store_id from next_cursor; opt into exact slice refs with include_exact_ref=true.
lcm_describe	Inspect the current-session DAG or preview an externalized_ref without loading full content.
lcm_expand	Recover source messages, child summaries, or externalized payloads with pagination. Use store_id to fetch a single raw message from a cross-session lcm_grep result and include_exact_ref=true when its returned slice must be cited or computed.
lcm_expand_query	Answer a question using expanded current-session LCM context while returning a bounded answer.
lcm_status	Show runtime health, context pressure, config, source lineage, and lifecycle stats.
lcm_inspect	Read-only operator inventory for current-session lineage, frontier/fresh-tail metadata, externalized refs/readability, compaction skip/no-op reasons, and matched ignore/stateless patterns. Returns metadata only; use retrieval tools for content.
lcm_doctor	Run database, FTS, lifecycle, config, and context-pressure diagnostics.

Recall skill and policy

LCM-X ships skills/hermes-lcm/SKILL.md plus progressive-disclosure references for configuration, architecture, diagnostics, recall routing, and session lifecycle. The installer links that directory into the active Hermes profile so it appears in ordinary skill discovery. On hosts with plugin skill registration, it is also available explicitly as hermes-lcm-x:hermes-lcm.

The canonical policy is distributed through that bundled skill. It is not injected through pre_llm_call: current Hermes persists hook context in the current user's api_content and replays it as user-authored history. Keeping policy out of that seam prevents attribution errors and one-copy-per-user-turn replay growth while preserving clean transcript and LCM ingest. The policy:

• treats summaries as recall cues rather than exact proof;
• prefers newer source-backed evidence and verifies contradictions;
• teaches narrow FTS query construction and bounded scope selection;
• routes current compacted, cross-conversation, and recent/time-bounded recall through the appropriate existing tools;
• does not force a tool call when the current context is already sufficient.

The canonical bytes live in skills/hermes-lcm/references/recall-policy.md. The active-LCM pre_llm_call hook remains available for bounded pre-answer evidence when that feature is explicitly enabled, but it never prepends the recall policy. Older hosts without skill or hook registration keep their existing schema-driven behavior.

Slash commands

Slash commands are disabled by default. Enable them only in trusted operator contexts:

export LCM_ENABLE_SLASH_COMMAND=1

Available commands:

• /lcm or /lcm status - current runtime/session status
• /lcm doctor - read-only health checks
• /lcm doctor clean - read-only scan for obvious junk/noise session candidates
• /lcm doctor clean apply - backup-first cleanup for safe pattern-matched candidates; requires LCM_DOCTOR_CLEAN_APPLY_ENABLED=true
• /lcm doctor repair - read-only SQLite/FTS repair diagnostics
• /lcm doctor repair apply - backup-first SQLite/FTS repair
• /lcm doctor source - read-only scan for legacy blank-source rows
• /lcm doctor source apply - backup-first normalization of legacy blank-source rows to unknown
• /lcm doctor retention - read-only retention analysis
• /lcm backup - timestamped SQLite backup
• /lcm rotate - read-only preview of an in-place tail-preserving compact of the active session
• /lcm rotate apply - backup-first rotate that advances the lifecycle frontier past pre-tail raw messages
• /lcm help - command help

Apply paths are intentionally narrow and backup-first. Start with diagnostics before cleanup or repair.

Rotate: compact in place without changing session identity

/lcm rotate compacts a long-running session in place without changing session_id or conversation_id. It is the in-session counterpart to Hermes-level /new and to /lcm doctor clean.

What rotate does:

• preserves the live tail (LCM_FRESH_TAIL_COUNT most-recent messages)
• advances the lifecycle frontier marker past every raw message before the tail, so subsequent bootstrap stops replaying them into the active prompt
• writes a rolling *-rotate-latest.sqlite3 backup under the same backup directory as /lcm backup, overwriting the previous rotate slot atomically so disk usage stays bounded across repeated rotates

What rotate does not do:

• it does not delete raw messages; pre-tail rows remain recoverable through lcm_load_session and lcm_expand
• it does not invoke the summarization model; trigger normal compaction first if you want pre-tail content covered by summary nodes before rotating
• it does not change session or conversation identity, and it refuses ignored or stateless sessions with a clear reason

lcm_status and /lcm status surface last_rotate_at and the rolling backup path. Re-running /lcm rotate apply after the frontier is already at or ahead of the target boundary reports status: noop and does not overwrite the previous known-good rolling backup.

Configuration

Most installs only need plugins.enabled and context.engine: lcm-x.

Common settings

Variable	Default	Use
LCM_CONTEXT_THRESHOLD	0.35	Fraction of the context window that triggers LCM compaction
LCM_ABSOLUTE_THRESHOLD_TOKENS	0	If > 0, force compaction at this absolute prompt-token count instead of context_length × LCM_CONTEXT_THRESHOLD. Cross-model context-health setpoint (common coding default: 130000) so large windows do not delay compaction and degrade recall
LCM_MODEL_THRESHOLDS	empty	Per-model threshold overrides. Format: "glm-5.2:0.70,glm-5.2-1M:0.25". Keys matched as substrings (longest wins). Also settable as lcm.model_thresholds in config.yaml.
LCM_FRESH_TAIL_COUNT	32	Recent messages protected from compaction
LCM_FRESH_TAIL_MAX_TOKENS	0	Optional token cap for the protected fresh tail (0 disables it); always retains the newest message and complete assistant/tool-result groups
LCM_FRESH_TAIL_PRESSURE_YIELD_ENABLED	true	Default-on: when compaction is deadlocked because the count-protected tail covers the whole over-threshold session (#441), the tail yields to a derived token bound so compaction can progress; false restores the strict count tail (rollback switch)
LCM_FRESH_TAIL_PRESSURE_YIELD_MIN_OBSERVATIONS	3	Consecutive tail-blocked compaction attempts under host-observed pressure before the yield engages; any attempt not blocked by the tail resets the count; 1 yields on first observation
LCM_INCREMENTAL_MAX_DEPTH	3	Max DAG condensation depth (-1 = unlimited, 0 = leaf only); enables hierarchical summarization
LCM_LEAF_CHUNK_TOKENS	20000	Raw-backlog floor before leaf compaction; with dynamic chunking enabled, the base chunk target
LCM_DYNAMIC_LEAF_CHUNK_ENABLED	false	Enable chunk-sized leaf compaction passes instead of compacting the whole non-tail raw backlog per pass
LCM_DYNAMIC_LEAF_CHUNK_MAX	40000	Upper bound for dynamic leaf chunk targets
LCM_THRESHOLD_FULL_SWEEP_ENABLED	false	At threshold, opt into one synchronous bounded sweep that drains chunked raw history before publishing one new active context
LCM_SUMMARY_PREFIX_TARGET_TOKENS	0	Sweep-only summary-frontier target; 0 derives one LCM_LEAF_CHUNK_TOKENS budget
LCM_NEW_SESSION_RETAIN_DEPTH	2	DAG depth retained after manual /new (-1 all, 0 none)
LCM_DATABASE_PATH	auto	SQLite database path. Empty config resolves to HERMES_HOME/lcm.db; plugin installs or operators may set this env var to another profile-scoped path such as ~/.hermes/hermes-lcm.db.
LCM_FTS_INTEGRITY_CHECK_INTERVAL_HOURS	24	Minimum hours between startup FTS5 deep integrity-checks (O(index size)). 0 checks every startup; a negative value never checks on startup. Structural checks always run regardless.
LCM_ENABLE_SLASH_COMMAND	false	Enable the optional /lcm operator command surface

When LCM_FRESH_TAIL_MAX_TOKENS is enabled, the protected suffix must satisfy both the message-count and token bounds. The newest message is never dropped, and a boundary that would begin inside an assistant tool-call/result group is moved back to that assistant even when doing so exceeds a configured bound.

Filtering and storage settings

Variable	Default	Use
LCM_IGNORE_SESSION_PATTERNS	empty	Comma-separated session globs excluded from LCM storage
LCM_STATELESS_SESSION_PATTERNS	empty	Comma-separated session globs kept read-only
LCM_IGNORE_MESSAGE_PATTERNS	empty	Comma-separated regex patterns; matching message content is excluded from LCM storage
LCM_SENSITIVE_PATTERNS_ENABLED	false	Opt in to durable deterministic redaction before LCM storage, FTS indexing, summarization, active replay, and externalized ingest payloads; it does not control cloud embedding privacy
LCM_EMBEDDING_PRIVACY_ENABLED	unset (auto)	Protect provider-bound copies for known cloud embedding providers without rewriting durable data. Set false for an explicit raw-cloud opt-out (privacy:off vector revision); local providers remain unchanged
LCM_SENSITIVE_PATTERNS	api_key,bearer_token,password_assignment,private_key	Comma-separated named sensitive pattern catalog entries to apply when redaction is enabled
LCM_LARGE_OUTPUT_EXTERNALIZATION_ENABLED	false	Store oversized ingest payloads, including tool results, media blocks, and generic raw content, in plugin-managed JSON files
LCM_LARGE_OUTPUT_EXTERNALIZATION_THRESHOLD_CHARS	12000	Externalization threshold for normalized payload text
LCM_LARGE_OUTPUT_ACTIVE_REPLAY_STUBBING_ENABLED	false	Replace token-heavy textual tool results with recoverable externalized refs in active replay; current-turn ingest is immediate and historical assembly respects the protected fresh tail; requires large-output externalization
LCM_LARGE_OUTPUT_ACTIVE_REPLAY_STUB_THRESHOLD_TOKENS	25000	Token-aware threshold for active-replay tool-result stubbing
LCM_LARGE_OUTPUT_TRANSCRIPT_GC_ENABLED	false	Rewrite already-externalized summarized tool rows to compact placeholders
LCM_DOCTOR_CLEAN_APPLY_ENABLED	false	Permit destructive /lcm doctor clean apply in trusted operator contexts
LCM_EMPTY_LIFECYCLE_GC_ENABLED	true	Master toggle for automatic pruning of lifecycle rows for sessions that never ingested any messages or summary nodes
LCM_EMPTY_LIFECYCLE_GC_THRESHOLD	200	Number of lifecycle rows at which the GC pass fires
LCM_EMPTY_LIFECYCLE_GC_MAX_AGE_HOURS	24	Automatic GC only deletes empty lifecycle rows at least this old; set 0 only in trusted/test environments that intentionally want immediate empty-row pruning
LCM_DISABLED_TOOLS	empty	Comma-separated lcm_* tool names excluded from injected tool schemas, refused in handle_tool_call before message ingest, and filtered from plugin-registry registration — disabled tools cost zero tokens per turn. Unset to restore all tools.

Model and timeout settings

Variable	Default	Use
LCM_SUMMARY_MODEL	auxiliary	Override summarization model
LCM_SUMMARY_REASONING_EFFORT	task/provider default	Summary reasoning override: none, minimal, low, medium, high, or xhigh; YAML key: lcm.summary_reasoning_effort. An unsupported value is ignored (falling back to this default) and reported by /lcm status and /lcm doctor
LCM_SUMMARY_FALLBACK_MODELS	empty	Comma-separated summarization models tried after LCM_SUMMARY_MODEL or the auxiliary task default fails
LCM_SUMMARY_CIRCUIT_BREAKER_FAILURE_THRESHOLD	2	Consecutive failed summarization calls before a route is skipped temporarily
LCM_SUMMARY_CIRCUIT_BREAKER_COOLDOWN_SECONDS	300	Seconds to skip an open summary route before retrying it
LCM_EXPANSION_MODEL	summary model / auxiliary	Override lcm_expand_query synthesis model
LCM_EXPANSION_REASONING_EFFORT	task/provider default	Expansion synthesis reasoning override with the same supported values and the same ignore-and-report handling; YAML key: lcm.expansion_reasoning_effort
LCM_EXPANSION_CONTEXT_TOKENS	32000	Context budget used by the auxiliary LLM for lcm_expand_query
LCM_SUMMARY_TIMEOUT_MS	60000	Timeout for one summarization call
LCM_NATIVE_RECOVERY	false	Opt-in recovery mode: ingest sources normally, then generate a native active-context summary for the Hermes host's archive transaction, without attempting LCM publication. Retains LCM sources/recall and does not advance its frontier. Keeps sanitized user text in active replay even when its durable copy is externalized, so native summarization can read it; already-published references are not automatically expanded. Requires the host cancellation fence; failure retains context without trimming.
LCM_EXPANSION_TIMEOUT_MS	120000	Timeout for one lcm_expand_query synthesis call
LCM_CRITICAL_BUDGET_PRESSURE_RATIO	0.0	Disabled at 0.0; when set, permits critical-pressure bypasses for bounded deferred catch-up and cache-friendly follow-on condensation only

Advanced compaction, assembly, and extraction knobs are defined in config.py.

Sensitive-pattern redaction

LCM is lossless by default: durable messages, FTS rows, summaries, and payloads remain untouched unless LCM_SENSITIVE_PATTERNS_ENABLED=true is explicitly set. Cloud embedding privacy is separate and transforms provider-bound copies only; it defaults on for known cloud providers and can be explicitly opted out with LCM_EMBEDDING_PRIVACY_ENABLED=false, which uses the privacy:off vector revision and sends raw provider input. The nonempty known-pattern policy requirements for warmup, backfill, and semantic-query dispatch apply only when provider-copy privacy is ON — the explicit privacy:off opt-out bypasses catalog validation by design. Local providers receive byte-identical input and do not enter the cloud privacy path.

When LCM_SENSITIVE_PATTERNS_ENABLED=true, matched secret values are replaced with metadata-only placeholders before SQLite, FTS, summaries, active replay, and externalized payload JSON receive the content. This is intentionally not lossless for matching values: the raw matched secret is unrecoverable after redaction.

Supported named catalog entries are:

• api_key: api_key, api_token, access_token, secret_key, and client_secret assignments or JSON keys.
• bearer_token: Bearer ... strings and token-like JSON keys.
• password_assignment: password, passwd, pwd, and passphrase assignments or JSON keys, including quoted values with spaces.
• private_key: PEM private-key blocks.

Redaction is forward-only. Enabling it does not rewrite existing SQLite rows, FTS shadow tables, DAG summaries, or externalized payload JSON that were written before the setting was enabled. Non-password placeholders include a short truncated SHA-256 digest for correlation. password_assignment placeholders omit the digest to avoid making password-like values easier to dictionary-check. lcm_status, lcm_inspect, and lcm_doctor expose the enabled state, configured pattern names, unknown names, source, and placeholder format without exposing raw secret values.

Cloud embeddings add a separate provider-input-only privacy transform. It does not rewrite historical SQLite, FTS, summary, or payload data. It resolves ON automatically for known cloud providers and does not require LCM_SENSITIVE_PATTERNS_ENABLED. While it is on, cloud warmup, summary/chunk backfill, and semantic-query dispatch require a nonempty, known LCM_SENSITIVE_PATTERNS catalog; matching provider input becomes pattern-only placeholders, residual matches are rejected, and the transform version plus active-pattern-name hash bind into the vector profile revision. LCM_EMBEDDING_PRIVACY_ENABLED=false opts out explicitly and binds the distinguished privacy:off revision instead of blocking. Either way a posture or policy change refuses dispatch until /lcm embed warmup registers the new identity. FastEmbed is in-process. The shipped Ollama provider does not apply the cloud gate, so use it only with a trusted local/loopback endpoint; #337 tracks endpoint-aware locality for remote Ollama configurations.

Threshold ownership

When context.engine: lcm-x is active, LCM_CONTEXT_THRESHOLD is the compaction threshold LCM uses. Hermes core compression.threshold belongs to the built-in compressor. Hermes core compression.enabled is still the global gate that allows compaction, so leave it enabled when using LCM.

If LCM_ABSOLUTE_THRESHOLD_TOKENS is set to a positive integer, it overrides the ratio-derived trigger after window math runs. Use this when you want a fixed context-health setpoint across model switches (for example 130000 for coding agents) so a larger window does not silently delay compaction, lower recall, or let long sessions accumulate more noise before LCM intervenes. Leave it at 0 to keep ratio-based behavior. When the absolute override is active, Codex GPT-5.5 ratio auto-raise is suppressed so the absolute setpoint stays pinned.

If startup/status output shows a host-side compression percentage that disagrees with LCM, trust live LCM status after a normal message has initialized the session.

For the 2026-08-31 gpt-5.4 Codex-auth retirement, migrate to gpt-5.6-sol (quality) or gpt-5.6-luna (cost); codex_gpt_long_context remains correct for their 272K effective window.

Tuning for large context windows

Long-context models change the tuning problem. A 1M-token model does not mean you always want to spend 750k prompt tokens before LCM starts compacting. Start with the active prompt budget you are willing to pay for, then tune the threshold around that budget. If the budget itself should stay fixed across models, set LCM_ABSOLUTE_THRESHOLD_TOKENS instead of recomputing a ratio per window.

compaction trigger = effective context window * LCM_CONTEXT_THRESHOLD
LCM_CONTEXT_THRESHOLD = desired compaction trigger / effective context window
# or, model-independent:
# LCM_ABSOLUTE_THRESHOLD_TOKENS = desired compaction trigger

Examples, as math rather than universal recommendations:

Effective context window	Desired trigger	Threshold
128000	96000	0.75
200000	140000	0.70
400000	240000	0.60
1000000	250000	0.25
1000000	400000	0.40
1000000	600000	0.60

A reasonable first pass for a true 1M effective window is:

Goal	Desired trigger	Threshold	Notes
Lower spend / earlier DAG building	200000 to 300000	0.20 to 0.30	Good when cost and latency matter more than maximum live context
Balanced large-context use	350000 to 500000	0.35 to 0.50	Good starting point for many long-running agents
Keep more raw context active	600000+	0.60+	Higher token burn, later compaction

Tune against your effective context_length if Hermes caps the provider's advertised window.

Start with LCM_CONTEXT_THRESHOLD, LCM_FRESH_TAIL_COUNT, and large output externalization. Only tune leaf chunking after checking lcm_status and understanding whether your workload is dominated by huge raw backlog passes.

LCM_THRESHOLD_FULL_SWEEP_ENABLED=true is an opt-in cache-shape policy. Once threshold pressure triggers compaction, the invocation keeps summarizing the oldest raw chunks outside the protected fresh tail even after pressure falls below the trigger. It then condenses the provider-visible summary frontier only when that frontier exceeds LCM_SUMMARY_PREFIX_TARGET_TOKENS (or one leaf budget when the target is 0). One invocation is bounded to 12 total leaf plus condensation calls and 120 seconds between calls, persists each completed DAG pass, and publishes one newly assembled active context at the end. It is synchronous and independent of deferred/background maintenance.

Cache policy boundary

LCM is cache-friendly, not fully cache-aware. It may avoid some follow-on condensation churn, but current cache usage counters are retrospective status data only; they do not tell the plugin whether the next prompt mutation will break a hot provider cache.

LCM_CRITICAL_BUDGET_PRESSURE_RATIO is a narrow escape hatch. It is disabled by default (0.0). When set, LCM compares prompt pressure against the context window and only at or above that ratio may bypass existing polite gates for bounded deferred maintenance catch-up and cache-friendly follow-on condensation. Revisit full cache-aware deferred compaction only after Hermes core exposes reliable cache state / cache-break signals.

Session pattern syntax

Pattern matching checks multiple keys: raw session_id, platform, and platform:session_id.

• * matches within one colon-delimited segment
• ** can span across colons

Example: cron:* can match Hermes cron sessions, while exact raw session IDs still work.

Noise suppression

LCM offers two layers of noise filtering:

• Session-level filters (LCM_IGNORE_SESSION_PATTERNS, LCM_STATELESS_SESSION_PATTERNS) catch noisy traffic that arrives as its own session or platform.
• Message-level patterns (LCM_IGNORE_MESSAGE_PATTERNS) catch cron alerts or other noise injected into a normal Telegram or WhatsApp conversation as ordinary visible messages.

Message-level patterns are comma-separated Python regex strings compiled once at engine start. They run against plain text first; structured multimodal payloads use concatenated text parts first, then normalized JSON fallback when there are no text parts. Matching messages are skipped before storage.

Example operator config:

LCM_IGNORE_MESSAGE_PATTERNS=^Cronjob Response:,^>>>Cronjob Response<<<:

Invalid regex entries are logged at warning level and dropped. Pattern matching uses a 50 ms per-pattern timeout when the optional regex package is installed. If regex is not installed, LCM logs a warning and disables message-level regex filtering rather than running unbounded stdlib re matches in the ingest path.

Known limitation: the filter runs at ingest time. When a matching message is part of the chunk summarized in the same turn it arrived, the text may appear inside the resulting summary node. The filtered message is still not written to the message store, so DAG lineage stays clean; only serialized summary text can carry it.

lcm_status surfaces the full filter contract under session_filters, including pattern sources, whether the current session is ignored/stateless, and a process-lifetime ignored_message_count.

Ignored/stateless sessions are a storage ownership boundary, not a context-window opt-out. LCM does not ingest raw messages or create DAG nodes for sessions that match LCM_IGNORE_SESSION_PATTERNS, LCM_STATELESS_SESSION_PATTERNS, or the in-process auxiliary/thread stateless marker. If those sessions cross the normal context threshold, LCM delegates the compaction call to Hermes' native ContextCompressor so the active request is still bounded before model overflow. If the native compressor is unavailable, LCM falls back to a deterministic head/tail trim as a last-resort safety net, still without writing the bypassed session to lcm.db.

Large tool-output handling

Storage-boundary payload guard contract: LCM prevents media-ish inline payloads from being written into plugin-local SQLite rows at the storage boundary.

Externalization for ordinary large tool output is opt-in. When enabled, oversized tool results are written to plugin-managed JSON files and referenced from summaries. They remain inspectable through lcm_describe(externalized_ref=...) and lcm_expand(externalized_ref=...).

Active-replay stubbing is a second, independently opt-in replay policy. When both externalization and active-replay stubbing are enabled, newly ingested textual tool results above the token threshold are durably externalized and replaced immediately in provider-visible replay, including results in the protected fresh tail. This lets a stub-only replay change converge even when no leaf is eligible for compaction. A historical assembly pass applies the same policy to older tool results before budgeting, while respecting the protected fresh tail. Tool-call ids and compatible structured text block types/keys are retained; raw SQLite rows and DAG lineage are not rewritten by the historical pass. Structured image/media results remain inline, preserving the provider replay contract established by upstream Hermes-LCM PR #226. If durable externalization cannot be confirmed, replay keeps the original payload inline. Results from lcm_describe and lcm_expand also remain inline so recovery does not recursively produce another ref.

The storage-boundary payload guard is separate from that opt-in. LCM always scans messages at the store boundary before writing messages.content or messages.tool_calls to SQLite. Inline data:*;base64,... payloads and conservative long base64-looking runs are replaced with compact placeholders and written to the same plugin-managed externalized-payload directory.

This avoids duplicating media-ish payload bytes into lcm.db, FTS shadow tables, WAL files, or ordinary SQLite backups while preserving lossless recovery via the placeholder ref and lcm_expand(externalized_ref=...). If externalization fails, LCM logs a warning and leaves the original text inline rather than dropping data.

lcm_doctor reports SQLite journal_mode, quick_check, database/WAL sizes, largest content/tool-call rows, suspicious inline payload rows, and aggregate externalized-payload stats. Doctor output is metadata-only for these scans.

This guard is scoped to LCM's own lcm.db write boundary. It does not prevent Hermes core, or any other host layer, from writing inline payloads to Hermes state.db, and it does not rewrite historical rows already present in lcm.db. If bytes already landed in Hermes state.db, that is upstream/outside LCM scope; use backup-first cleanup or migration procedures before mutating historical host rows.

Transcript GC is separate and opt-in. It only rewrites already-externalized, already-summarized tool-role rows to compact placeholders. It keeps the same store_id, keeps payload files, skips pinned messages, and preserves lossless recovery through externalized_ref.

Retrieval contract

LCM retrieval tools default to current-session scope. lcm_grep accepts session_scope='all' or session_scope='session' as an explicit opt-in for bounded archive search over rows already present in lcm.db (raw-message hits only). Once a session id is known, lcm_load_session can enumerate that session's raw transcript in chronological store_id pages without a search query. Use Hermes session_search for broad cross-session history outside the LCM database.

Within the current session, source filters raw rows directly and filters summary nodes by descendant raw-message source lineage. unknown is a real source value, not a wildcard. Legacy blank-source rows are treated as unknown. role, time_from, and time_to are raw-message filters applied in the message search query before result limiting. When a raw-message filter is active, lcm_grep returns raw rows only and reports summary_results_omitted.

Tool responses are bounded so one retrieval call cannot flood the main context.

Lossless raw recovery contract

Lossless recovery means raw content is stored with stable source lineage and can be recovered in deterministic pages:

• lcm_expand(node_id=...) pages immediate sources with source_offset and source_limit
• lcm_load_session(session_id=...) pages ordered raw session rows with after_store_id and next_cursor
• oversized raw messages continue with content_offset
• lcm_expand(externalized_ref=...) pages payload content with content_offset
• lcm_expand_query uses context_max_tokens for auxiliary context and reports truncation/pagination hints when needed

Carried-over summary nodes can become current-session content after /new, but their source eligibility still comes from descendant raw messages. Expanding a carried-over current-session node recovers the original raw message sources even when those sources still belong to a previous session.

OpenClaw/lossless-claw import

LCM-X includes an opt-in operator script for backfilling OpenClaw history into the local LCM-X SQLite store. It supports two source shapes:

1. SQLite LCM database (--source-db) for migrations from an existing lossless-claw/OpenClaw lcm.db.
2. JSONL session exports (--source-jsonl / --source-jsonl-dir) for fresh installs, plugin-off catch-up, or migrations where no source SQLite database is available.

SQLite source example:

python scripts/import_lossless_claw.py \
  --source-db ~/.openclaw/path/to/lcm.db \
  --target-db ~/.hermes/lcm.db \
  --agent sammy

JSONL source example:

python scripts/import_lossless_claw.py \
  --source-jsonl ~/.openclaw/agents/sammy/sessions/session-a.jsonl \
  --target-db ~/.hermes/lcm.db \
  --agent sammy \
  --json

For a directory of session exports:

python scripts/import_lossless_claw.py \
  --source-jsonl-dir ~/.openclaw/agents/sammy/sessions \
  --target-db ~/.hermes/lcm.db \
  --agent sammy

The script is intentionally conservative:

• dry-run is the default; pass --apply to write
• run it against an explicit target DB path, preferably while Hermes is stopped for that profile
• writes create a timestamped target DB backup first when the target already exists
• SQLite imports can include OpenClaw summaries with --include-summaries; this migrates compatible summary rows into Hermes summary_nodes
• JSONL imports migrate raw message rows only; JSONL does not carry a summary DAG
• imported rows keep explicit provenance in session_id and source, for example openclaw-lcm:agent:sammy:<source-session> or openclaw-jsonl:agent:sammy:<source-session>
• the SQLite default provenance identity is the source conversations.session_id, preserving source session boundaries even when many conversations share one session_key
• pass --session-identity session_key only for SQLite imports when you intentionally want conversations with the same source session key grouped into one imported LCM session
• reruns are idempotent for the same --import-id; the default import_id is source-path-derived, so pass a stable --import-id if you may import the same copied DB or JSONL export set from different paths
• --json prints a reconciliation report with scanned, eligible, would_import, imported, skipped_existing, skipped_empty, invalid_rows, warnings, and summary counters
• changing --agent, --namespace, or --session-identity under the same --import-id is treated as the same import and will skip already-tracked source messages; use a new --import-id for a different mapping
• no OpenClaw config or separate secret tables are imported, but raw transcripts, summaries, and tool payloads may contain sensitive user data

This is a local archive migration path. It does not make LCM a general memory provider, and it does not change the current-session retrieval contract for agent tools.

For databases that already contain large historical tool rows, scripts/backfill_externalized_tool_outputs.py can pre-create native recovery sidecars without rewriting SQLite. It is dry-run by default, writes a scrubbed digest/count manifest, is idempotent on repeat apply, and supports guarded manifest-based rollback. See the operator guide.

Troubleshooting

hermes plugins shows lcm-x (not found) but LCM tools exist

If plugins.enabled contains hermes-lcm-x, context.engine: lcm-x is set, and the runtime exposes LCM tools, LCM is loaded. The lcm-x (not found) line is a Hermes host discovery/status mismatch, not an LCM storage or compaction failure.

/lcm status looks unbound after restart

Compatible Hermes gateway hosts expose task-local lane metadata while dispatching /lcm. Once that lane has constructed an agent, /lcm status resolves the same active LCM runtime as lcm_status, including its foreground view while an ignored or stateless side channel is bound. Before the first normal message constructs an agent, or in a genuinely sessionless process, session_id: (unbound) and threshold_tokens: (uninitialized) remain expected.

Architecture

The engine sits between Hermes context assembly and the backing conversation store. It records raw messages, compacts old material into a summary DAG, and exposes retrieval tools that can drill back into exact stored sources.

How it works

1. Ingest - persist each message in SQLite with FTS metadata
2. Compact - summarize older messages outside the fresh tail into D0 leaf nodes
3. Condense - merge same-depth nodes into higher-depth summaries
4. Escalate - shrink oversize summaries from detailed to bullets to deterministic truncate
5. Assemble - combine system prompt, highest-depth summaries, and fresh tail
6. Retrieve - use LCM tools to drill into compacted history or synthesize from expanded context

Documentation

• Feature overview — every feature family, what it does, why it exists, and the switch that enables it
• Agent configuration profiles — copy-paste env profiles: coding agent, long-horizon assistant, fully-local, cost-guarded
• Operator guide — install, activation, full configuration reference, diagnostics
• Retrieval tools reference — exact tool contracts
• Current project state — v0.24.1 stable baseline, separate main-development identity, active work, and proof boundaries
• Benchmark methodology and results — retrieval and judged-QA evaluation contracts, reproduction, and landed result index
• Embeddings setup — free-tier and local embedding providers, warmup, backfill
• LCM paper
• Architecture diagram
• Standard compression diagram
• LCM compression diagram
• Contributing guide
• Code of conduct
• Security policy
• Tags

Development

Important files:

plugin.yaml      manifest
__init__.py      plugin registration and optional slash-command registration
engine.py        LCMEngine main orchestrator
store.py         SQLite message store and FTS
dag.py           summary DAG and FTS
config.py        env var defaults and overrides
command.py       /lcm command handlers
tools.py         lcm_grep, lcm_load_session, lcm_describe, lcm_expand, lcm_expand_query
schemas.py       tool schemas shown to the model
tests/           standalone pytest coverage

Run tests:

pip install pytest numpy
python -m pytest tests/ -v

No Hermes Agent checkout is required for the test suite; tests include a lightweight ABC stub.

Contributing

Issues and PRs welcome. Bug fixes and correctness improvements are highest priority. New features should be scoped, backwards-compatible, and tested.

See CONTRIBUTING.md for branch, validation, and PR guidance. See CODE_OF_CONDUCT.md for project conduct expectations and SECURITY.md for vulnerability reporting. See the releases page, tags page, and CHANGELOG for version history. v0.24.1 is the latest stable GitHub Release; verify its exact SHA before installation.

License

MIT

Star history