All plugins

hermes-talk

Realtime voice for Hermes through terminal, Discord, dashboard and Desktop. Disclosure — Realtime on the ChatGPT subscription uses the profile's Codex sign-in only through Hermes's credentialed_provider_request (requires_auth: openai-codex); the plugin never reads, refreshes or holds a token. GPT-Live on the subscription (TALK_VOICE_MODE=live, TALK_LIVE_AUTH=subscription) starts one local `codex app-server` child per call on the Codex CLI's own login (API-key variables removed from its environment); that call uses the undocumented chatgpt.com Codex realtime endpoint, which is outside the vendor's published API terms and may change or stop working without notice. The app-server's Codex thread is ephemeral and fenced (no environment, read-only sandbox, approvals declined, apps, shell, web search, plugins, sub-agents and each configured MCP server disabled, replies only "skip"); Talk reads only the MCP server names to disable them, and each voice request costs one small Codex turn of subscription quota. Voice audio and conversation context go to the selected provider (OpenAI, xAI, Google, optionally ElevenLabs); on Desktop the voice model can list the profile's sessions, read their titles and recent messages, create sessions and queue or steer prompts into them without a per-action confirmation; spoken requests are handed to the user's Hermes agent, whose tools run under Hermes approvals. No telemetry.

TheSmokeDev180Voicev0.22.1Updated 7 days ago

About

Realtime voice for Hermes through terminal, Discord, dashboard and Desktop. Disclosure — Realtime on the ChatGPT subscription uses the profile's Codex sign-in only through Hermes's credentialed_provider_request (requires_auth: openai-codex); the plugin never reads, refreshes or holds a token. GPT-Live on the subscription (TALK_VOICE_MODE=live, TALK_LIVE_AUTH=subscription) starts one local `codex app-server` child per call on the Codex CLI's own login (API-key variables removed from its environment); that call uses the undocumented chatgpt.com Codex realtime endpoint, which is outside the vendor's published API terms and may change or stop working without notice. The app-server's Codex thread is ephemeral and fenced (no environment, read-only sandbox, approvals declined, apps, shell, web search, plugins, sub-agents and each configured MCP server disabled, replies only "skip"); Talk reads only the MCP server names to disable them, and each voice request costs one small Codex turn of subscription quota. Voice audio and conversation context go to the selected provider (OpenAI, xAI, Google, optionally ElevenLabs); on Desktop the voice model can list the profile's sessions, read their titles and recent messages, create sessions and queue or steer prompts into them without a per-action confirmation; spoken requests are handed to the user's Hermes agent, whose tools run under Hermes approvals. No telemetry.

Hermes Talk

Realtime duplex voice for your own agent — talk to it, it runs real work in the background, it reports back out loud.

Hermes Talk is a realtime voice plugin for Hermes Agent: you speak, it answers out loud, and it calls the agent's own tools without leaving the conversation. It runs in the terminal (hermes talk), in a Discord voice channel (/talk join), in the Hermes dashboard Talk tab, and in Hermes Desktop.

It rides your ChatGPT subscription. No API key required.

The realtime lanes are OpenAI Realtime (gpt-realtime-2.1, on your ChatGPT subscription through your Hermes Codex sign-in, hermes auth add openai-codex), xAI Grok Voice (XAI_API_KEY), and Gemini Live (which does need a GEMINI_API_KEY — free-tier AI Studio keys work); GPT-Live is selected separately by TALK_VOICE_MODE=live and runs gpt-live-1-codex on the ChatGPT subscription through the Codex CLI (codex login) or gpt-live-1 on explicitly chosen API billing. Talk never holds a subscription token: Hermes signs the Realtime requests, and the Codex CLI owns the GPT-Live call.

Talk calls the host's tools, delegates background work while you keep talking, reports results, and handles current approval requests. Provider support differs by surface; see the tables below.

What it is: a plug-in that adds interruptible, duplex speech-to-speech voice to an
existing Hermes Agent install — one bidirectional audio session in which the agent uses
its real tools, hands work to background agents, and speaks the results when they land.

Who it is for: people already running Hermes Agent who would rather talk to it than
type at it, and who want to keep talking while it works. It is not a standalone
assistant, and it does not replace Hermes's built-in turn-based voice mode — that is a
different shape, and a good one.

The Talk tab in the Hermes dashboard — a live voice session, a background agent delegated mid-conversation, its result landing in the runs panel (8× speed)

🔊 Watch the 2:27 cut with sound — delegate, keep talking, hear the result land.

Real Realtime session, 8× speed in the GIF, recorded at v0.3.0. It does not demonstrate the new GPT-Live or Codex-worker integration.

Install

This is the Hermes plugin catalog build of Hermes Talk (branch
catalog). It meets the
catalog's admission rules: the ChatGPT subscription is used only through
Hermes-signed requests or the Codex CLI's own login. Install it by name:

hermes plugins install hermes-talk --enable
pip install "hermes-talk[audio]"   # mic + speaker support (sounddevice); skip if dashboard-only

Needs Python ≥ 3.11 and Hermes Agent ≥ 0.21.7. GPT-Live on the subscription also
needs the Codex CLI (npm install -g @openai/codex, then codex login). The
standalone build on the main branch keeps every subscription lane, including
the Grok subscription, and installs from GitHub as described in its README.

Every version with its receipts: CHANGELOG.md.

Providers and billing

| Lane | How it authenticates | Default model | Surfaces |
|---|---|---|---|
| OpenAI Realtime (TALK_PROVIDER=openai, default) | ChatGPT subscription through the Hermes Codex sign-in (hermes auth add openai-codex; Hermes signs the request), or TALK_OPENAI_API_KEY / OPENAI_API_KEY | gpt-realtime-2.1 | every surface |
| GPT-Live (TALK_VOICE_MODE=live) | TALK_LIVE_AUTH=subscription (the default: the Codex CLI's own ChatGPT login through codex app-server; needs npm install -g @openai/codex and codex login) or explicitly chosen api billing; no automatic paid fallback | gpt-live-1-codex (subscription) / gpt-live-1 (API) | terminal, Discord, dashboard, Desktop — on an explicit task |
| xAI Grok Voice (TALK_PROVIDER=grok) | TALK_XAI_API_KEY / XAI_API_KEY (the SuperGrok / X Premium login returns once Hermes signs xAI requests for plugins) | grok-voice-latest | terminal + Discord |
| Gemini Live (TALK_PROVIDER=gemini) | GEMINI_API_KEY / TALK_GEMINI_API_KEY — free-tier AI Studio keys work | gemini-3.1-flash-live-preview | terminal + hermes realtime; Discord refuses it for now |
| Cascade voice (TALK_VOICE_MODE=cascade) | TALK_ELEVENLABS_API_KEY / ELEVENLABS_API_KEY, on top of the OpenAI Realtime lane | eleven_flash_v2_5 | terminal, Discord, dashboard |

The provider knob is fail-closed and never inferred from which keys exist.
Grok supports all 28 documented built-in voices through TALK_GROK_VOICE
(for example, carina); see the voice list.
Per-lane detail and the credential order: docs/PROVIDERS.md.
Speaking in a voice of your own: docs/CASCADE.md.
Experimental xAI speech effects and phrase wrappers, including how native Talk
differs from verbatim TTS: docs/XAI-SPEECH-TAGS.md.

What Talk touches

What you would want to know before installing (catalog rule 13):

  • Network: your voice and the conversation context go to the provider you
    select — OpenAI (api.openai.com, and chatgpt.com for subscription calls),
    xAI, Google, and optionally ElevenLabs for cascade speech.
  • Credentials: API keys are read through Hermes's profile-scoped secret
    route. The ChatGPT subscription is used only through Hermes-signed requests
    (requires_auth: [openai-codex]) or the Codex CLI's own login; Talk never
    reads, refreshes or stores a subscription token, and never reads
    ~/.codex/auth.json or Hermes's auth.json.
  • Background processes: live voice keeps WebSocket/WebRTC connections open
    for the call. GPT-Live on the subscription starts one codex app-server
    child per call (stopped when the call ends) with the API-key variables
    removed from its environment. Talk reads only the names of your Codex MCP
    servers so it can switch them off for that call's fenced Codex thread; that
    thread runs a small Codex turn for each voice request (subscription quota),
    replies only skip, and cannot run tools.
  • Work on your behalf: spoken requests are handed to your Hermes agent,
    whose tools can run shell commands under Hermes's own approvals.
  • Local data: run history and voice-session state under Hermes's home. No
    telemetry.

Surfaces

| Surface | Entry point |
|---|---|
| Terminal | hermes talk — task attachment with hermes talk --task TARGET |
| Inside a Hermes session | /talk — the one surface with an attached agent loop, so lookups and delegation answer inline |
| Discord voice channel | /voice join, then /talk join [TARGET] |
| Dashboard Talk tab | hermes dashboard, select a task, Start |
| Desktop Talk composer action | open a conversation → Talk → Connect; works on stock Desktop, with a floating window on the Talk-enabled build (host lanes) |

Is it working?

hermes plugins list        # → hermes-talk · enabled · current version
hermes talk doctor         # → read-only: auth lane, provider, model/voice, audio, host lanes
hermes talk check          # → doctor + one live provider turn + one bounded Hermes run
# then, in any session: say "status report" — talk_status answers with
# version, auth lane, agent lane, and audio state.

Doctor is read-only by design: it names which lane came up and what is missing,
and never writes, probes, or refreshes a token. check is the other half and is
deliberately not read-only — one short provider turn and one short agent run,
exit 0 only if every step passed. A mock can never go green.

Filing an issue? hermes talk diagnostics --bundle writes one redacted,
owner-only file — versions, the names of the variables you have set, device and
host facts, and every doctor outcome; no values, logs, prompts, transcripts,
audio, or paths. It is safe to paste into a public issue and it is what the
bug template asks for.

Upgrade with hermes plugins update hermes-talk — not a second install —
then restart the gateway: a running process keeps executing the old code
until you do.

The full diagnostic walk, every receipt, and the upgrade runbook:
docs/OPERATING.md.

Documentation

Everything Hermes Talk does, in depth:

  • OPERATING.md — install, upgrade, use, every knob, the three agent lanes, the Discord lane, the dashboard tab, current boundaries, troubleshooting.
  • PROVIDERS.md — per-lane provider detail and the fail-closed OpenAI credential order.
  • BACKGROUND-WORK.md — delegation, admission control, steering a running agent, and the capability bridge.
  • CASCADE.md — the cascade lane: your own ElevenLabs voice over a realtime provider.
  • GPT-LIVE.md — GPT-Live billing and voice, task attachment, Codex workers, operator acceptance.
  • DESKTOP.md — Talk in the Hermes desktop app: the stock lane, and what the Talk-enabled build adds.
  • VOICE-COMMANDS.md — say this, hear this, and what each spoken receipt commits to.
  • REALTIME-ORCHESTRATOR.md — architecture map of the tool-calling realtime lane.
  • dashboard-task-continuity.md — joining, continuing and reconnecting to a canonical Hermes task.
  • recipient-routing.md — addressing an existing application task from a voice task.
  • codex-workers.md — selecting a Codex background worker from a bound task.
  • task-event-projection.md — the worker-side library that restores task and work state.
  • passive-attachment-client.md — the shared passive-history client used by all three surfaces.
  • PROVIDER-RECEIPT.md — how to report a provider lane that worked, or broke, for you.
  • CAPABILITY-KERNEL-PORT.md — the capability-plugin kernel adaptation guide.

Design rules

The three that shaped everything else:

  • Nothing fails quietly. A degraded backend, a missing tool, a run whose
    watcher died — each is said out loud in the conversation. A voice surface
    that silently does less than you asked is worse than one that refuses.
  • The credential never leaves the process. Key or OAuth token hits exactly
    one OpenAI endpoint (the mint) and the socket only ever sees the ephemeral
    secret it returns.
  • Hermes owns the tools and the session. The Realtime layer is ears, mouth,
    and turn-taking. It never owns the agent loop.

Background

Hermes Talk began as a plugin and became a reference implementation for the
speech-to-speech contract Hermes core now carries:

Status

Under active development; the PyPI badge above is the released version.
Every version with its receipts: CHANGELOG.md. Open epics and
threads:
#19 provider-neutral
Realtime voice platform,
#32 operator-grade
orchestration UX,
#43 Talk over the Bot
Mode roster,
#44 channel-neutral
voice transport.

Contributing

pip install -e ".[dev]" -r requirements-lint.txt, pytest -q, ruff check .
— offline, no keys, seconds. Priorities, the path for each kind of change
(a provider, a surface, a tool, a fix), the merge bar, and the one test trap
on a box that has Hermes installed: CONTRIBUTING.md.
First response within 24 hours;
good first issue
fits in one sitting; a provider that works or broke for you is a
contribution too (docs/PROVIDER-RECEIPT.md).

Contributors adapting The Homie's v1.7.0 capability-plugin lessons to Hermes
should use the capability-kernel port plan.
It maps the reusable safety and lifecycle contracts onto Hermes-owned APIs;
it does not claim that hot lifecycle support already exists here.

Contributors

@danclaw93: room-scoped spoken approvals send
the request_id the Hermes run API reads, so they stop failing with HTTP 400
(0.17.1).

@HearthCore: diagnosed why Desktop Talk could
never connect on a stock Hermes host, and made it connect there; the composer
popover also stops closing when the pointer crosses the chat
(#169,
#170, 0.21.1).

@kvnloo: PulseAudio WebRTC echo cancellation
on Linux, and the fix that stopped quiet words being clipped during
playback (#81);
semantic turn-detection controls across the three lanes
(#107, in review).
@TheAngryPit — a renderer-owned Realtime
transport for the Hermes desktop app that keeps core as the single chat
authority (#80, in
review). @webdevtodayjason —
field-tested feedback from a second live consumer on the upstream
RealtimeVoiceProvider contract these lanes register on
(hermes-agent#81404).

License

MIT. Adapted-source licenses and contributor credits are in
THIRD_PARTY_NOTICES.md.

README from the reviewed commit 790f6d2; it updates when the author re-pins.