glasses-platform

Your own Hermes agent on your smart glasses: it sees what you see, answers by voice, and taps you on the lens when something is due. A gateway platform that adds glasses as a channel over one WebSocket, with short spoken-style replies to the device that asked, camera photos (requested when you ask what you are looking at, or taken by the agent with its glasses_capture_photo tool), streamed replies for heads-up displays, approval and clarify prompts on every device, and cron and heartbeat pushes to all of a user's devices, queued in memory while they are offline. Speaks the bridge protocol of the community hermes-glasses iOS app (v0) and a richer v1; a simulator is included. Disclosure — opens an unencrypted WebSocket listener on all interfaces (port 8765, /voice) that refuses to start without a device token (use Tailscale or a TLS proxy off your home network); each device token gets full agent chat as its user, bypassing the gateway allowlist, including answering approval prompts and clarify questions; the agent can take a glasses camera photo at any time during a glasses conversation and photos are also captured automatically on visual keywords (auto_capture: off disables that), saved in Hermes's image cache; the device's new-session button answers Hermes's /new confirmation for that user; no outbound network calls, shell commands or telemetry.

100yenadminPlatformv0.1.2Updated 2 hours ago

About

glasses-platform

Talk to your own Hermes agent from smart glasses. Ask by voice, show it what you see, and hear or read the
answer. Reminders and long tasks can reach your glasses later.

Status: v0.1.2 preview (milestone M0). The Hermes side is proven against the bundled simulator and a real Hermes
gateway with real models (evidence). CI tests it on Hermes v0.22.2, v0.23.0 and
main. No real glasses or phone app has been tested yet. Read the design plan, the
device protocol and the roadmap.

What it is

A Hermes Agent gateway platform plugin that adds smart glasses as a
channel, the same way Hermes already talks to Telegram or Slack.

A thin companion app on your phone (or the glasses' own app platform) connects over one WebSocket.
Hermes keeps the conversation, memory, tools, cron jobs, heartbeat and approvals.

glasses ─▶ companion app ─WebSocket─▶ Hermes gateway (glasses platform) ─▶ your agent

Planned support

| Glasses | Path |
|---|---|
| Meta Ray-Ban (Gen 1/2), Ray-Ban Display, Oakley Meta | A phone app on Meta's Wearables Device Access Toolkit. The community hermes-glasses iOS app speaks protocol v0 in its bridge mode; not yet tested against this plugin on a device. |
| Even Realities G2 | Even Hub app (M4) |
| Mentra Live | Mentra cloud app (M4) |
| Meta AI on glasses | Meta AI Connectors endpoint (M4, gated on Meta early access) |

Install

Requirements:

Hermes Agent 0.22.2 or later. The plugin needs aiohttp, which is declared in pyproject.toml. If your Hermes
was installed without the messaging extra, hermes plugins install asks to install it. Without a terminal,
pass --yes-deps.
In-process plugin loading, which is the default. The opt-in plugins.isolation: host mode cannot load platform plugins.

hermes plugins install https://github.com/100yenadmin/glasses-platform --enable

Configure

Give each device its own token. Tokens map to user:device, and all devices of one user use the same conversation.
Put this in ~/.hermes/.env:

GLASSES_DEVICES=me:rayban=<long-random-token>,me:g2=<another-token>
# or, for a single device:  GLASSES_TOKEN=<long-random-token>   (user "owner", device "default")

Then enable the platform in config.yaml and restart the gateway:

platforms:
  glasses:
    enabled: true
streaming:
  enabled: true      # optional: stream replies to v1 clients as they are written

Photo turns need a model that can see: either a main model that accepts images, or an auxiliary.vision model in
config.yaml.

The plugin refuses to start without at least one token. It listens on 0.0.0.0:8765 at /voice by default.
GLASSES_HOST, GLASSES_PORT and the other settings are listed in docs/PLAN.md.

To use it away from home Wi-Fi, put it behind Tailscale or a TLS reverse proxy. The plugin does not terminate TLS.

Try it

A demo gateway: demo/run_demo_gateway.sh <empty-dir> starts an isolated Hermes gateway with this plugin, a
generated device token and only the vision, glasses, cronjob and memory toolsets (it uses the z.ai GLM coding plan;
see the script's header).

With the simulator:

GLASSES_SIM_TOKEN=<token> python sim/glasses_sim.py --url ws://127.0.0.1:8765/voice --say "What can you do?"

To test a photo turn, add --photo photo.jpg. The plugin asks the "glasses" for a photo when you say something like
"what am I looking at?". The agent can also take one itself with the glasses_capture_photo tool.

With the hermes-glasses iOS app (untested so far): in the app, set Settings → Assistant → Backend to Bridge,
and point it at ws://<your-mac>.local:8765/voice?token=<token>.

Scheduled messages: send them to your glasses with --deliver glasses:<user>, for example
hermes cron create "every day at 9am" "Brief me on today" --deliver glasses:me. They arrive as pushes, or wait in a
short offline queue until a device reconnects.

What it does on your machine

Opens a network listener. A WebSocket server on all interfaces, port 8765, path /voice (set GLASSES_HOST
to 127.0.0.1 to keep it local). It refuses to start until at least one device token is configured.
A device token is full access to your agent. Anyone holding one can chat with your agent as you, and can answer
the approval prompts and clarify questions the agent sends to your devices. Keep tokens long, random and private.
Photos. Photos from the glasses are saved in Hermes's own image cache, like photos sent from any other channel.
New session. The device's new-session button answers Hermes's /new confirmation for that user's own session.
The plugin never answers command approvals; the wearer does.
No third-party calls. The plugin makes no outbound network requests, runs no shell commands, sends no telemetry
and starts no background processes beyond its listener.
Tokens are read through Hermes's per-profile settings reader, from ~/.hermes/.env or config.yaml.
Offline queue. Pushes for a disconnected device wait in memory (50 items, 24 hours) and are lost on a gateway
restart.

Why a plugin rather than a bridge script

Existing bridges start a new Hermes CLI process for every utterance. They cannot reach the glasses after the
turn ends, and they lose streaming and approvals.
A gateway platform keeps one live session per person across all their devices. Cron and heartbeat results
arrive as pushes, and replies stream to a heads-up display.

License

MIT. See LICENSE.

README from the reviewed commit dbc797b; it updates when the author re-pins.

More by 100yenadmin

11 plugins by 100yenadmin in the catalog.

100yenadminhermes-lcm-xLossless memory for long sessions: LCM-X keeps every turn in a local store instead of compacting it away, and the agent can search and expand it. The store is local SQLite (lcm.db) with a recoverable DAG of summaries over it and 16 lcm_* tools for search, recall and expansion (lcm_memory only with LCM_ENTITIES_ENABLED=1). Select it with context.engine: lcm-x. Upgrading from hermes-lcm (breaking in v0.24.0): stop Hermes, then in plugins.enabled replace hermes-lcm with hermes-lcm-x and run exactly one LCM copy (the install script prints the steps; the old engine name lcm still resolves with a deprecation warning); lcm.db and the lcm: config block are unchanged. Summaries use the model Hermes is configured with. Disclosure — local semantic recall is on by default once its dependency companion is installed (hermes plugins install electricsheephq/lcm-x/companions/hermes-lcm-x-local-embeddings --ref <the LCM-X commit>; it carries fastembed): summaries and message chunks are embedded in-process with BAAI/bge-small-en-v1.5 after a one-time model download, in the background, and stay on the machine. Without the companion, recall answers from full text and lcm_doctor prints the install command; on hosts with 2 or fewer effective CPUs the default answers from full text unless LCM_EMBEDDINGS_ENABLED=true; LCM_EMBEDDINGS_ENABLED=false turns semantic recall off. Cloud embedding providers (Voyage via VOYAGE_API_KEY, an OpenAI-compatible endpoint, or a local Ollama) send message text to that provider and run only with LCM_EMBEDDINGS_ENABLED=true and an explicit model. For known cloud providers the provider-bound copy is privacy-transformed by default (LCM_EMBEDDING_PRIVACY_ENABLED=false opts out), while lcm.db keeps the raw text. Every turn (user, assistant and tool output) is stored in plaintext in lcm.db (secret redaction at rest is opt-in); on gateway platforms the sender's platform user ID is recorded with each user message by default (LCM_SENDER_CAPTURE=0 turns it off), and oversized tool outputs are by default written as plaintext files under HERMES_HOME/lcm-large-outputs and replaced in the model's context by recoverable references. The engine also reads Hermes' state.db read-only for session lineage, reads private AIAgent attributes via the call stack on Hermes builds without subagent hooks, and runs git on its own install directory for lcm_status/lcm_doctor; tiktoken may fetch its BPE file on first use.Communitymemory116 100yenadminfish-audioYour agent speaks and listens with Fish Audio: expressive voices in 80+ languages, voices you design from a sentence, and live word-by-word dictation. The official Fish Audio plugin: Fish's Drama 3 model with word-level direction, speech-to-text, Fish Audio's community voice library, voice cloning and voice design, with a Voices page in Hermes Desktop. Fish Audio becomes a Hermes speech and transcription provider for voice replies, read-aloud, voice notes and Desktop dictation, with sentence-by-sentence streaming in CLI voice mode and Desktop, and word-by-word live dictation captions on Hermes 0.21.6+ (stt.streaming: true). Adds the fish_speak, fish_voices and fish_transcribe tools, /fish and hermes fish commands, and four skills. Disclosure — sends your Fish API key and the text, audio (with file names) and voice titles, tags, descriptions, cover images and design instructions you ask it to process to api.fish.audio, or to the base_url you configure, and reads your Fish wallet for balances and model choice; with live dictation on, microphone audio streams to Fish's native realtime speech-to-text at api.fish.audio (transcribe-stream-1, Preview) while you speak, billed $0.36 per hour of open session, per second, including silence; a session opens with the first audio and closes once Fish confirms the last words (the plugin stops waiting after 15 seconds), or 30 seconds after the last audio of a cancelled recording; if a live session fails or returns no text, Hermes transcribes the recording as a file instead, a separate billed request; the live_transcription: compat setting uses Fish's older realtime endpoint instead, billed per committed phrase rounded up to a second; synthesis, transcription, cloning and voice design are paid requests billed to your Fish account, except the free s2.1-pro-free model (free through 30 November 2026), whose requests Fish may use to improve its models; when no model is set and your Fish wallet shows no API credit, free credit or past top-up, the plugin uses that free model (allow_free_model: false turns this off). The key comes from the profile's secret scope (Hermes stores it in the profile .env; a single-profile install may also read FISH_API_KEY from the environment). Generated audio and subtitles are written to Hermes's audio cache; clone samples, cover images and recordings it uploads must be regular audio or image files, and Hermes config, secret and credential paths are refused. Cloning and deleting voices from chat or the model tools go through Hermes's approval gate and follow your approval settings. The Desktop Voices page and a Fish Audio settings page add gateway routes under /api/plugins/fish-audio/, behind the Hermes dashboard's existing authentication. On the Voices page, previews, clones and voice designs are billed; cloning a voice or saving a designed one creates a voice in your Fish account; cloning and deleting are confirmed on the page (a speaker-consent box, a typed voice name) instead of through Hermes's approval gate; and Use sets the profile's Fish voice, and its speech provider when none is set. The settings page and the /fish and hermes fish commands can set or reset the profile's Fish model and voice and save up to 50 voice names in the profile's config; /fish use and hermes fish use also make Fish the speech provider when none is set, and given a voice title they search your own Fish voices for it, and hermes fish login sets the speech and transcription providers when they are unset or you confirm. Preview and design audio files are deleted from the gateway once returned; a design's audio is held in gateway memory so you can save it, until it is saved, a later design or save finds it over an hour old, or the gateway stops; clone uploads are deleted after the clone, or by the next upload once idle for 15 minutes. With a key set, Desktop reads your Fish wallet when an agent is selected and about every five minutes, for the status-bar credit; favourites stay in Desktop's plugin storage on that computer, and fetched previews stay in the window's memory until it closes. When you ask for a voice that fits the agent, the voice-studio skill reads the agent's SOUL.md or persona and sends Fish a one- or two-sentence voice description, never the whole persona. Includes a report-a-problem skill: when you report a problem with Fish Audio, the agent runs hermes fish doctor --no-synth, /fish status, hermes --version and hermes plugins list --json, reads recent Hermes log lines that mention Fish, searches this plugin's GitHub issues for a few key words, drafts a bug report and files it to this plugin's repository as a public GitHub issue or comment with your gh login only after you approve the exact text; reports leave out keys, credit amounts and voice ids not confirmed as public library voices, and security problems go to the repository's private advisory form instead. No telemetry.Communityvoice68 100yenadminhermes-teammatesGive your agent a team: named subagents with their own instructions, tools and model, which it assigns work to, checks on and steers mid-task. The operator defines the roster; the agent follows up with a digest of earlier results and can work a Kanban teammates lane with a review handoff. Launches subagents only through the public ctx.subagent_lifecycle API. Stock Hermes; no patched core. Disclosure — adds six tools (teammates_roster, teammate_assign, teammate_check, teammate_wait, teammate_message, teammate_stop) and a /teammates command; the model picks only a teammate name, and the operator's config decides what it means. Teammates run as in-process Hermes subagents using credentials selected by Hermes from the parent, delegation settings or an operator-configured teammate route where supported. Toolsets narrow the parent's tools at toolset granularity only (the file toolset includes write_file and patch); Hermes can add inherited MCP toolsets back unless delegation.inherit_mcp_toolsets is false. A teammate with invalid settings refuses to launch. A delegated child cannot start teammates. Steering uses the lifecycle steer where Hermes provides it, otherwise delegate_task's steer action with the turn-bound parent. Under plugins.isolation: host every tool except the roster refuses. Optionally the roster is read from an operator-named YAML file (roster_file, relative to the profile home or absolute, so several profiles can share one); the plugin never writes it. It keeps a per-profile SQLite run ledger in its plugin-data folder (goals, statuses, clipped summaries, role hashes and available token receipts; no full transcripts). It does not read credential stores. Where supported in-process, subagent_start and post_api_request hooks collect token receipts for mapped child sessions. For a card assigned to its non-profile lane, it claims the card through the Kanban DB, heartbeats the claim and its worker, then moves the card to review without a reviewer, or blocks it on failure, a stop or an unavailable result; it never completes a card. On Hermes builds with plugin prompt sections it adds one short paragraph (under 600 characters) about the Kanban lane to new top-level session prompts. The paragraph is rendered from its settings, carries no conversation content, and is left out without a usable teammate, for delegated children and under host isolation. It runs one in-process monitor thread per run to retain finished results and, for lane runs, keep the claim alive; no network calls, child processes or self-updates of its own. Completion-notice code exists but never injects on current Hermes because the host cannot bind notices to the receiving session; use teammate_wait instead. Includes a report-a-problem skill: the agent drafts a bug report and files it to this repository with gh only after the user approves the draft.Communityautomation59 100yenadminplan-modeClaude Code-style plan mode for Hermes: the agent explores and writes a plan, and edits nothing else until you approve it. It works on every interactive Hermes surface: /plan or /planmode on blocks mutating Hermes tools dispatched through pre_tool_call (writes outside .hermes/plans, terminal, code execution, delegation, MCP) while the agent explores, asks clarifying questions and writes a plan. The agent shows the complete plan, submits it with plan_mode(submit) and one approval card asks: approve and auto-accept edits, approve and review each file edit (when Hermes approvals are on, every write_file and patch then gets Hermes' approval prompt with its path and a preview), or keep planning; Other sends feedback, and the revised plan says what changed since the last revision. Once approved, it implements in the same turn and tracks todo_list progress. To change the plan yourself, keep planning, run /planmode edit, save, then /planmode show and /planmode approve; the text you were shown is saved as an approved snapshot, so later edits to the file do not change the approval. The classic profile (plan_profile: classic), approval_card: false and surfaces without a card use Hermes' own approval prompt (gateway buttons, the TUI/Desktop card, the CLI panel or /approve). When yolo or approvals.mode: off would auto-approve, it asks through clarify instead, so a person still decides. By default plans are compact and decision-complete, the lean profile asks at most three targeted questions (say "grill me" for more, or set questions: thorough for an interview before every plan, one decision at a time), the reply names any tool call plan mode blocked (Hermes itself shows none), and the agent does not commit unless asked (plan_style, plan_profile, questions, allow_commits and plan_skill settings). Classic CLI enforced on every supported Hermes; gateway, TUI and Desktop from Hermes v2026.9.24 (0.21.5), and refused on older builds; 0.21.6 or newer recommended, tested through 0.23.0. On the interactive classic CLI, Hermes' inline diff shows a new plan file in full, so the agent does not retype it (write_once). A report-a-problem skill drafts a bug report and files it with gh only after you approve the draft. Disclosure — plan-mode does not enforce Codex app-server exec/applyPatch or TUI /background and btw side agents. A plan is approved only on its approval card (a clarify question only the plugin issues and checks), through Hermes' own approval prompt, or with /planmode approve. It reads nine groups of internal Hermes seams for session identity, workspace, profile, skills.inline_shell and the classic CLI's inline diff, each guarded with a documented fallback. It creates .hermes/plans under the session working directory, reads plan files there, saves the approved text of an edited plan as a new file under .hermes/plans/approved, and keeps per-session state in plugin state. It adds a turn note while planning or executing, a ≤200-char system-prompt hint, and a one-line footer to short replies on chat platforms. When a turn ends while a submitted plan still awaits approval and no approval card was answered, it adds one line to that reply on any surface, whatever its length, saying how to approve (unless footer is off). When you pick "Always" on a plan approval or a per-edit review, Hermes core writes a plugin_rule:plan-mode entry to command_allowlist in config.yaml; each entry is keyed to one plan revision, so it cannot approve another plan. With allow_gateway_injection: true it queues one message to start work after a typed /planmode approve. On TUI, Desktop and hermes serve sessions (lean profile, plan_once) it reads the visible reply text from post_api_request while planning and saves its <proposed_plan> block as the plan file; if another plugin holds the stream to withhold commentary, that text can include the commentary it kept off screen (plan_profile: classic turns plan_once off, unless a custom levers list names plan_once or all). On the interactive classic CLI (lean, write_once) it checks that Hermes printed a newly written plan file in full, then tells the agent not to retype it. Its report-a-problem skill has the agent file to the plugin repository with gh only after the user approves the exact saved draft. The plugin itself makes no network calls, launches no subprocesses and has no self-updater.Communitytools48

All 11 by 100yenadmin