All plugins

omnivoice

OmniVoice TTS provider for Hermes Agent

ThaungThanHan18VoiceUpdated 1 month ago

Hermes OmniVoice TTS Plugin

Powered by k2-fsa/OmniVoice from Xiaomi. 🎙️

OmniVoice is an impressive local multilingual zero-shot TTS model with broad language support, voice cloning, voice design, fine-grained speech controls, and fast inference.

hermes-omnivoice is a community plugin that integrates Xiaomi's OmniVoice TTS with Hermes Agent, adding an omnivoice TTS provider for multilingual speech synthesis, voice design, and voice cloning.

Once enabled, the plugin registers a Hermes TTS provider named:

omnivoice

Hermes' existing text_to_speech flow can synthesize speech through OmniVoice.

What can it do?

  • 🌍 Multilingual TTS - generate speech in 600+ languages supported by OmniVoice.
  • 🎭 Voice design — describe the voice you want, such as gender, pitch, age, accent, or speaking style.
  • 🧬 Voice cloning — use any reference audio to create a similar voice.
  • 🛠️ Advanced controls — configure speed, diffusion steps, guidance scale, duration, and more.
  • 🧩 Hermes-native — works as a Hermes TTS provider without modifying Hermes core.
  • 💻 Local inference — run speech generation locally with OmniVoice.

Use cases

  • Personalized, custom voices for Hermes agents
  • Voice cloning to tune the voice as you desire
  • Marketing and product-demo voiceovers
  • Video narration for tutorials, reels, and explainers
  • Multilingual content creation
  • Creative voice experiments

Install

You can let your Hermes agent install the hermes-omnivoice plugin by itself. Just give it the installation commands or the repository link.

Install the plugin and OmniVoice runtime in the same Python environment that runs Hermes:

pip install "hermes-omnivoice[omnivoice] @ git+https://github.com/ThaungThanHan/hermes-omnivoice.git"
hermes plugins enable omnivoice
hermes config set tts.provider omnivoice

If you only want to install the lightweight plugin package without OmniVoice:

pip install "hermes-omnivoice @ git+https://github.com/ThaungThanHan/hermes-omnivoice.git"

Basic config

Add this to ~/.hermes/config.yaml:

tts:
  provider: omnivoice
  omnivoice:
    model: k2-fsa/OmniVoice
    device: auto
    dtype: float16
    voice: auto

After changing config, restart Hermes.

Voice modes

OmniVoice supports three main voice modes through this plugin.

1. Auto voice

Let OmniVoice pick the voice automatically:

tts:
  provider: omnivoice
  omnivoice:
    voice: auto

Good for quick testing.

2. Voice design

Describe the voice you want:

tts:
  provider: omnivoice
  omnivoice:
    voice: "female, low pitch, british accent"

You can also pass a voice-design instruction at call time:

instruct:female, low pitch, british accent

Example ideas:

male, young adult, high pitch
female, whisper
older male, deep voice
female, british accent

3. Voice cloning

Use a short reference audio file:

tts:
  provider: omnivoice
  omnivoice:
    ref_audio: /absolute/path/to/reference.wav
    ref_text: "Transcript of the reference audio"

Hermes can also pass reference audio at call time:

ref:/absolute/path/to/reference.wav

or simply pass an existing local audio-file path as the voice value.

For best results, use a clean reference clip with one speaker, low background noise, and an accurate transcript.

Runtime voice values

When Hermes passes a voice value, the provider handles it like this:

| Voice value | Behavior |
| --- | --- |
| auto | Automatic voice selection |
| default | Automatic voice selection |
| random | Automatic voice selection |
| instruct:female, british accent | Voice-design instruction |
| ref:/path/to/reference.wav | Reference audio for voice cloning |
| /path/to/reference.wav | Reference audio if the file exists |
| Any other non-empty string | OmniVoice instruct value |

Configuration

The provider reads tts.omnivoice from ~/.hermes/config.yaml.

Environment variables override config values.

| Config key | Environment variable | Default |
| --- | --- | --- |
| model | OMNIVOICE_MODEL | k2-fsa/OmniVoice |
| device | OMNIVOICE_DEVICE | auto |
| dtype | OMNIVOICE_DTYPE | float16 |
| voice | OMNIVOICE_VOICE | auto |
| ref_audio | OMNIVOICE_REF_AUDIO | unset |
| ref_text | OMNIVOICE_REF_TEXT | unset |
| instruct | OMNIVOICE_INSTRUCT | unset |
| language | OMNIVOICE_LANGUAGE | unset |
| num_step | OMNIVOICE_NUM_STEP | 32 |
| guidance_scale | OMNIVOICE_GUIDANCE_SCALE | 2.0 |
| speed | OMNIVOICE_SPEED | Hermes speed or 1.0 |
| duration | OMNIVOICE_DURATION | unset |
| load_asr | OMNIVOICE_LOAD_ASR | enabled when ref_text is absent |
| asr_model | OMNIVOICE_ASR_MODEL | openai/whisper-large-v3-turbo |

Generation controls also support:

t_shift
denoise
postprocess_output
layer_penalty_factor
position_temperature
class_temperature

Multilingual example

Set a language in your Hermes config:

tts:
  provider: omnivoice
  omnivoice:
    language: th

Then ask Hermes to speak:

Use text_to_speech to say: สวัสดีครับ นี่คือเสียงจาก OmniVoice

You can also test Chinese, English, Burmese, Japanese, Korean, and other languages supported by OmniVoice.

Audio format

OmniVoice generates waveform audio.

This plugin can write these formats directly through soundfile:

wav
flac
ogg

If Hermes requests mp3, the plugin uses ffmpeg when available. If ffmpeg is not available, it writes a .wav file and returns that path.

Install ffmpeg for MP3 output and Telegram voice-bubble conversion:

brew install ffmpeg
# or
sudo apt install ffmpeg

Command-provider fallback

If you prefer Hermes' config-driven command provider instead of the Python plugin, you can use the included wrapper command:

tts:
  provider: omnivoice-cli
  providers:
    omnivoice-cli:
      type: command
      command: "omnivoice-hermes-tts --text-file {input_path} --output {output_path} --model k2-fsa/OmniVoice --voice {voice} --speed {speed}"
      output_format: wav
      timeout: 600
      voice_compatible: true

The omnivoice-hermes-tts wrapper is installed by this package. It reads Hermes' {input_path} text file and calls the same provider code used by the Python plugin.

The Python plugin is recommended for normal use because it can keep the model loaded during the Hermes session.

Quick test

Create a text file:

echo "Hello from Hermes OmniVoice." > test.txt

Run:

omnivoice-hermes-tts \
  --text-file test.txt \
  --output test.wav \
  --voice auto \
  --format wav

Then play the generated file:

open test.wav

On Linux:

ffplay test.wav

Notes

  • First generation may take longer because the model needs to download and load.
  • GPU is recommended for faster inference.
  • CPU may work, but can be slow.
  • Voice cloning quality depends heavily on the reference audio quality.
  • Voice design behavior depends on OmniVoice model capabilities.

Safety

OmniVoice supports voice cloning.

Only clone voices you have the right to use. Do not use this plugin to impersonate people, deceive others, bypass verification, or create misleading content.

You are responsible for following the OmniVoice model license and local laws.

Contributing

Issues, ideas, bug reports, and pull requests are welcome.

Useful contribution ideas:

  • More tested config examples
  • Better Hermes setup docs
  • Voice-design presets
  • Language examples
  • Tests for provider behavior
  • Troubleshooting notes for CUDA, CPU, macOS, and Linux

If you try this plugin, feedback is warmly welcome.

README from the reviewed commit 21d2900; it updates when the author re-pins.