All plugins

hermes-body

Hermes Body turns a phone into a first-class Hermes channel: a private camera-and-voice page streams frames and speech transcripts into the gateway, and the full agent answers out loud and can act with any tool. Disclosure — the plugin runs a local HTTP/WebSocket server (default 127.0.0.1:8788, token-gated) that serves the phone page and receives camera frames, speech transcripts and recorded voice clips; it stores the pairing token (0600) and the newest 50 received frames and voice clips in the plugin data directory; frames reach the configured model provider the same way any photo attachment does, and voice clips go through Hermes' configured speech-to-text like any voice note; it opens no outbound connections of its own and sends no telemetry; the page must be served over HTTPS (Tailscale Serve or another proxy) for browser camera and microphone access.

ItsmetiboPlatformv0.2.0Updated 4 hours ago

About

Hermes Body turns a phone into a first-class Hermes channel: a private camera-and-voice page streams frames and speech transcripts into the gateway, and the full agent answers out loud and can act with any tool. Disclosure — the plugin runs a local HTTP/WebSocket server (default 127.0.0.1:8788, token-gated) that serves the phone page and receives camera frames, speech transcripts and recorded voice clips; it stores the pairing token (0600) and the newest 50 received frames and voice clips in the plugin data directory; frames reach the configured model provider the same way any photo attachment does, and voice clips go through Hermes' configured speech-to-text like any voice note; it opens no outbound connections of its own and sends no telemetry; the page must be served over HTTPS (Tailscale Serve or another proxy) for browser camera and microphone access.

Hermes Body

Your phone is the agent's eyes, ears and mouth.

Hermes Body turns a phone into a first-class Hermes channel. Point the camera, hold to
talk, and the agent sees the frame, hears the transcript, answers out loud, and can act
with any tool it already has.

It is not a skill and not a wrapper around a prompt: it registers a real gateway
platform adapter, so the phone gets the same agent loop, session, memory, tools and
approvals as Telegram or the CLI.

phone (HTTPS PWA)  ──WebSocket──►  BodyAdapter  ──MessageEvent──►  gateway agent
      ▲                                                                  │
      └──────────────── reply / speak ◄──────────────────────────────────┘

What it looks like

  • Full-screen camera preview with a hold-to-talk control.
  • Speech-to-text on the phone (Web Speech API) and text-to-speech for the reply.
  • If the browser has no speech service, the page records the clip and sends it instead;
    Hermes transcribes it with the configured STT provider. Local Whisper is free: install
    the stt-whisper extra and Hermes picks it up (see Requirements).
  • Every frame is attached to the message as an image, so the agent's normal vision path
    handles it.
  • The reply is displayed and spoken; interim status lines are not spoken.
  • A body_speak tool lets the agent talk through the phone from any other session.

Install

hermes plugins install Itsmetibo/hermes-body --enable
hermes config set platforms.body.enabled true
hermes config set platforms.body.extra.host 127.0.0.1
hermes config set platforms.body.extra.port 8788

Phone cameras and microphones require a secure context, so front the local port with
HTTPS. On a tailnet:

tailscale serve --bg --https=8444 http://127.0.0.1:8788
hermes config set platforms.body.extra.public_url https://<machine>.<tailnet>.ts.net:8444
hermes gateway restart
hermes body url

Open the printed URL on the phone. Tap Start camera, allow camera and microphone,
and hold the orange button to talk.

Configuration

| Key | Default | Meaning |
| --- | --- | --- |
| host | 127.0.0.1 | Bind address. Keep loopback and proxy it. |
| port | 8788 | Local port. |
| public_url | empty | HTTPS base URL printed by hermes body url. |
| token | generated | Pairing secret. Empty generates one in the plugin data directory. |
| max_width | 1280 | Frames are resized to this width before the model sees them. |
| voice | device default | Preferred speechSynthesis voice name. |

Security and privacy

  • The WebSocket requires the pairing token, compared in constant time. The page shell is
    public; no data moves without the token.
  • The adapter declares authorization_is_upstream: the token is the authorization, so
    the gateway does not ask for a second pairing code.
  • The token is stored 0600 in ~/.hermes/plugin-data/hermes-body/pairing.json.
    Rotate it with hermes body token --rotate.
  • Nothing is recorded. A frame is captured only when you send a message or press the
    shutter. Frames and voice clips are written under the plugin data directory, pruned to
    the newest 50, and sent to the model the same way any photo or voice attachment is.
  • Bind to loopback and expose it through Tailscale Serve or another HTTPS proxy. Do not
    put this on the public internet.
  • Approvals still belong to Hermes. A tool that needs approval shows up as a message on
    the phone; the agent never bypasses the approval system.

Requirements

  • Hermes >=0.22.0.

  • aiohttp, declared as a plugin dependency and installed by Hermes' package manager.
    Pillow is optional: when present, frames are resized server-side; without it the
    phone's JPEG is stored as-is.

  • Chrome on Android (or another browser with SpeechRecognition) for voice input. The
    keyboard button is always available as a fallback.

  • For the audio fallback (browser without SpeechRecognition), Hermes needs an STT
    provider. The free, local option is Whisper:

    hermes pm install --extra stt-whisper

    Hermes then auto-detects local and downloads the configured model
    (stt.local.model, default base) on first use. No API key.

  • HTTPS for camera/microphone access.

Development

python3 tests/test_body.py
hermes plugins doctor . --ci
hermes plugins validate .

The test suite starts a real adapter on 127.0.0.1:8799, exercises token rejection,
inbound image handling and outbound reply/commentary routing, and needs no phone.

Limitations

  • One phone is the intended setup; multiple connections all share the body chat.
  • Voice input depends on the browser's speech recognition; text always works.
  • The page is portrait-first and designed for a phone, not a desktop browser.

README from the reviewed commit a061ee8; it updates when the author re-pins.