SceneMux for Hermes
Turn any Hermes agent into a music-video director: drop an MP3 into the chat,
review a storyboard, and get back a beat-aligned, lip-synced music video —
the finished MP4, muxed with the master audio.
SceneMux owns audio slicing and the final stitch. Video engines (Comfy Cloud,
local ComfyUI on a GPU host, Intel AI Playground Home Agent, optional FAL
drafts) only produce per-scene picture.
What the plugin provides
- Tools (toolset
scenemux):scenemux_doctor(health check),
scenemux_setup(configure + start the server), and the studio tools
(project / plan / render / status / export). - Skill
scenemux:studio— the operating knowledge (pipeline lanes,
engines, happy path, spend rules). - Zero Python dependencies — the plugin client is stdlib only.
Install
# From the catalog (once listed)
hermes plugins install scenemux
# From the repo (development / before catalog listing)
hermes plugins install ep150de/scenemux/hermes-plugin
The plugin installs as a standalone client (no server code is copied with
it). For scenemux_setup to launch the studio server for you, set the
plugin's repo_path setting to a full clone of
the repo — the directory that
contains docker-compose.yml + server/ — or start the server yourself and
point base_url at it (see Setup).
Setup
- Keys — cloud mode (works on any machine) needs:
COMFY_CLOUD_API_KEY— https://platform.comfy.org/profile/api-keys (Standard+ plan)OLLAMA_API_KEY(https://ollama.com/settings/keys) orOPENCODE_API_KEY(https://opencode.ai/)- optional
FAL_KEY(https://fal.ai/dashboard/keys) — drafts only - Local mode (GPU host) needs no cloud keys, just ComfyUI models + a local LLM.
Keys live in~/.hermes/.env(or the Desktop plugin settings). The plugin
reads them only through Hermes' secret scope — never from files directly.
- Server — Ask your agent: "set up SceneMux in cloud mode." It runs
scenemux_setup, which launches the server via Docker (detached; log at
<HERMES_HOME>/plugin-data/scenemux/server-launch.log) and waits for it
to become healthy. First run builds the image (a few minutes).scenemux_setup
finds the server code via therepo_pathsetting (point it at a full repo
clone) — or, if the server is already running, it just checks health. - Optional — make Hermes itself the director — pass
director_backend=hermestoscenemux_projectand the storyboard is
planned by this Hermes agent (its full toolset, not an external LLM
lane). This needs the Hermes API server:
The plugin reads# add to ~/.hermes/.env, then (re)start the gateway API_SERVER_ENABLED=true API_SERVER_KEY=<some-long-random-key> hermes gatewayAPI_SERVER_KEYthrough the secret scope and sends it to
the server per session (only when the studio server is on loopback) — no
extra configuration per project.scenemux_doctor
reports whether the API server is configured and reachable. - Check —
scenemux_doctortells you exactly what's missing, if anything.
Make a video
Just ask: "make a music video for ~/songs/demo.mp3 — dark neon city,
synthwave." The agent will: doctor → start server → create project + upload
→ plan the storyboard (show it to you) → estimate + confirm → render →
deliver the MP4 (and 9:16 on request).
Security & disclosure
- Network calls to: the SceneMux server (loopback by default; tool
calls to non-loopback hosts are refused), Comfy Cloud, Ollama Cloud,
OpenCode Go, FAL.ai (only the endpoints the server uses with the keys
you provide). - Spawns a long-running process:
docker compose up -dfor the SceneMux
server (detached, logs captured; published on127.0.0.1only, no
wildcard CORS). It keeps running after the session ends. - Spend: rendering consumes Comfy Cloud credits (and optional FAL
credits). Every render goes through an estimate, and aconfirm=true
call is escalated to a Hermes human-approval prompt (the plugin's
pre_tool_callspend gate) — the render cannot start without that
approval. - Secrets are read only through the Hermes secret scope, sent only to
loopback /host.docker.internalhosts, and written to the server's
.env(chmod 600) only for a 127.0.0.1-bound server.docker compose
is spawned with a minimal explicit environment, not Hermes's whole env. - No telemetry, no analytics, no credential storage of its own.
- This plugin does not override any Hermes core behaviour; it only registers
its tools, its spend-gate hook, and one skill.
Troubleshooting
scenemux_doctorsays server unreachable → re-runscenemux_setup, or
check the launch log.HTTP 401/403from a studio call → a key is missing/invalid; doctor lists
which one.director_backend=hermesfails → the Hermes API server is not up: doctor
reportshermes_director.reachable. Fix:API_SERVER_ENABLED=true+
API_SERVER_KEYin~/.hermes/.env, thenhermes gateway.- Home Agent port (e.g.
:8765) ≠ Comfy port (:8188). - The server itself reads its own
.envwhen run manually (docker compose
auto-loads the repo.env); the plugin always supplies credentials via
the environment it spawns, and never reads credential files.
README from the reviewed commit ca6ef43; it updates when the author re-pins.

ep150de