hermes-local-voice

hermes-local-voice

Offline voice capture and transcription plugin for Hermes Agent (>=0.21.5) powered by faster-whisper and PipeWire (pw-record), ALSA (arecord), or ffmpeg.

Disclosure

Audio files and microphone recordings processed by hermes-local-voice are transcribed locally on your machine and are not uploaded to external servers. No API keys, cloud tokens, or remote endpoints are used during transcription. The only network activity occurs when you explicitly run hermes voice --download <model> to fetch a model from Hugging Face into HERMES_HOME.

What It Does

hermes-local-voice adds four agent tools, a speech-to-text provider, two optional inbound chat hooks, an in-session /local-voice slash command, and a headless hermes voice CLI subcommand:

• Tools:
  • transcribe_audio(path): validates and transcribes a local audio file (.wav, .mp3, .ogg, .opus, .m4a, .flac, .webm, .aac, .mp4, .wma, .caf). Returns transcript text, duration in seconds, detected language, and the model name.
  • voice_record_start(): starts 16 kHz mono microphone capture via pw-record, arecord, or ffmpeg. Degrades cleanly with a structured status message on headless machines without a microphone.
  • voice_record_stop(auto_transcribe=False, cleanup_on_transcribe_success=False): stops active microphone capture and returns the saved audio path and duration in seconds.
  • transcribe_last(): returns the most recent transcript from memory or HERMES_HOME/plugin-data/hermes-local-voice/last_transcript.json.
• Message Hooks:
  • pre_gateway_dispatch (opt-in, off by default): transcribes inbound voice notes and audio attachments arriving through messaging gateways into text before agent dispatch.
  • pre_llm_call: automatically transcribes local audio file attachments referenced in CLI or desktop chat turns and injects the transcript context.
• Headless CLI:
  • hermes voice <file> prints the transcript to standard output without starting an interactive chat session.

Why Local Processing Matters

Cloud speech-to-text APIs require network access, account credentials, and per-minute billing, and they transmit raw voice audio to third-party servers. Running faster-whisper with CTranslate2 int8 quantization on the local CPU or loopback worker keeps voice notes inside the local environment, works on air-gapped or offline machines, and requires zero API keys.

Dependencies and Supported Platforms

Supported Platforms

• Desktop Hosts (x86_64, arm64): full support for file transcription and live microphone capture via PipeWire (pw-record), ALSA (arecord), or PulseAudio (ffmpeg).
• macOS (arm64, x86_64): full support for file transcription and microphone capture via ffmpeg (avfoundation).
• Headless Hosts: file transcription and inbound chat hooks work without audio hardware; voice_record_start reports status: "unavailable" with an actionable message when no audio capture device is present.

Install

Install from Git or a local checkout using the Hermes plugin manager:

hermes plugins install https://github.com/soilgrowerx/hermes-local-voice --yes-deps --enable

For a local repository checkout:

hermes plugins install file:///path/to/hermes-local-voice --yes-deps --enable

First-Run Model Download and Disk Cost

hermes-local-voice is offline by default and does not download model weights unannounced during transcription. Models are stored under $HERMES_HOME/plugin-data/hermes-local-voice/models (or read from an existing local Hugging Face cache at ~/.cache/huggingface/hub).

Check cached models and disk cost before downloading:

hermes voice --models

Download a model explicitly (announces disk cost before fetching and can be cancelled with Ctrl+C):

hermes voice --download base.en

CLI Usage

Transcribe an audio file headlessly:

hermes voice sample.wav

Emit structured JSON with duration, detected language, and model used:

hermes voice sample.wav --json

Retrieve the most recent transcript:

hermes voice --last

Record from the microphone and transcribe:

hermes voice --record-start
hermes voice --record-stop

Configuration

Configure settings in ~/.hermes/config.yaml under plugins.entries.hermes-local-voice.config:

plugins:
  entries:
    hermes-local-voice:
      enabled: true
      config:
        model_size: "base.en"
        compute_type: "int8"
        language: "en"
        model_dir: ""
        max_file_mb: 100
        server_url: ""            # empty: never contact a loopback server
        prefer_server: false      # only honoured when server_url is set
        auto_transcribe_inbound: false

Inbound Transcription (preferred path)

Register this plugin as Hermes's speech-to-text provider instead of relying on the
gateway hook:

stt:
  provider: hermes_local_voice

Core then transcribes only messages it has already admitted, so a stranger on a
connected platform cannot trigger transcription.

auto_transcribe_inbound still exists for compatibility, but it runs in
pre_gateway_dispatch, which fires before sender authorization. It is off by
default for that reason. Enable it only on a private, single-user deployment.

Child Processes and Secrets

Every child process (ffprobe, ffmpeg, the Whisper worker, the model downloader, and
microphone capture) receives an explicit minimal environment: PATH, HOME, LANG,
plus the capture variables or Hugging Face variables that child actually needs.
Provider API keys are never inherited.

Cleanup Boundary

cleanup_on_success deletes a file only when it resolves inside this plugin's own
plugin-data/hermes-local-voice/recordings directory. Any other path is transcribed
and left alone.

Temporary File Policy

When converting non-WAV audio files or transcribing temporary recordings:

• Temporary converted audio files are removed automatically after transcription succeeds.
• If transcription fails, temporary audio files are preserved on disk and their path is returned in the preserved_audio_path field so no recording is lost.

License

MIT

README from the reviewed commit b84c82c; it updates when the author re-pins.