All plugins

scenemux

Turn an MP3 into a beat-aligned, lip-synced music video from any Hermes agent. Eight tools: scenemux_doctor (health/keys), scenemux_setup (start the studio server via Docker, provisions keys from the agent secret scope), scenemux_project (upload audio + references, beat-grid analysis, Whisper lyrics), scenemux_plan (LLM storyboard with beat-locked scenes, shown for review before anything is generated), scenemux_render (spend-gated: cost estimate first, confirm=true is escalated to a Hermes human-approval gate), scenemux_status, scenemux_export (final MP4, optional 9:16 vertical cut), scenemux_cancel. The storyboard director can run on a cloud LLM, a local Ollama model, or the agent itself (director_backend=hermes) via the local Hermes API server. A bundled skill (scenemux:studio) carries the operator workflow. Disclosure - the plugin talks only to the user's own SceneMux studio server (default http://127.0.0.1:8000) and reads the audio/reference paths the agent passes it; scenemux_setup starts one local server process (Docker container, port-published on 127.0.0.1 only) that runs until the session ends; planning and stills call the LLM/image/video backends configured on that server; the default cloud preset spends Comfy Cloud credits on renders, which is why renders are estimate-first and a confirm=true call is escalated to the user's approval gate; the server is reachable on loopback only (compose publishes 127.0.0.1 ports, no wildcard CORS), and tool calls to non-loopback hosts are refused; secrets are read only through the agent secret scope, are sent only to loopback / host.docker.internal hosts, and the server .env is written (chmod 600) only for a 127.0.0.1-bound server; no telemetry. Disclosure — SceneMux talks to your own SceneMux studio server (default http://127.0.0.1:8000), which scenemux_setup can start as a local Docker container published on 127.0.0.1 only; it uploads the audio and reference images the agent passes it, and the studio sends prompts and images to the LLM, image and video backends configured there (the default cloud preset spends Comfy Cloud credits, so renders show a cost estimate first and confirmed renders go through Hermes' approval prompt); API keys are read through the agent secret scope and sent only to a local studio, the Home Agent address comes only from the plugin's home_agent_url setting, and a ComfyUI address the agent passes for local rendering receives your reference images; no telemetry.

ep150de2Toolsv0.5.1Updated 3 days ago

About

Turn an MP3 into a beat-aligned, lip-synced music video from any Hermes agent. Eight tools: scenemux_doctor (health/keys), scenemux_setup (start the studio server via Docker, provisions keys from the agent secret scope), scenemux_project (upload audio + references, beat-grid analysis, Whisper lyrics), scenemux_plan (LLM storyboard with beat-locked scenes, shown for review before anything is generated), scenemux_render (spend-gated: cost estimate first, confirm=true is escalated to a Hermes human-approval gate), scenemux_status, scenemux_export (final MP4, optional 9:16 vertical cut), scenemux_cancel. The storyboard director can run on a cloud LLM, a local Ollama model, or the agent itself (director_backend=hermes) via the local Hermes API server. A bundled skill (scenemux:studio) carries the operator workflow. Disclosure - the plugin talks only to the user's own SceneMux studio server (default http://127.0.0.1:8000) and reads the audio/reference paths the agent passes it; scenemux_setup starts one local server process (Docker container, port-published on 127.0.0.1 only) that runs until the session ends; planning and stills call the LLM/image/video backends configured on that server; the default cloud preset spends Comfy Cloud credits on renders, which is why renders are estimate-first and a confirm=true call is escalated to the user's approval gate; the server is reachable on loopback only (compose publishes 127.0.0.1 ports, no wildcard CORS), and tool calls to non-loopback hosts are refused; secrets are read only through the agent secret scope, are sent only to loopback / host.docker.internal hosts, and the server .env is written (chmod 600) only for a 127.0.0.1-bound server; no telemetry. Disclosure — SceneMux talks to your own SceneMux studio server (default http://127.0.0.1:8000), which scenemux_setup can start as a local Docker container published on 127.0.0.1 only; it uploads the audio and reference images the agent passes it, and the studio sends prompts and images to the LLM, image and video backends configured there (the default cloud preset spends Comfy Cloud credits, so renders show a cost estimate first and confirmed renders go through Hermes' approval prompt); API keys are read through the agent secret scope and sent only to a local studio, the Home Agent address comes only from the plugin's home_agent_url setting, and a ComfyUI address the agent passes for local rendering receives your reference images; no telemetry.

SceneMux for Hermes

Turn any Hermes agent into a music-video director: drop an MP3 into the chat,
review a storyboard, and get back a beat-aligned, lip-synced music video —
the finished MP4, muxed with the master audio.

SceneMux owns audio slicing and the final stitch. Video engines (Comfy Cloud,
local ComfyUI on a GPU host, Intel AI Playground Home Agent, optional FAL
drafts) only produce per-scene picture.

What the plugin provides

  • Tools (toolset scenemux): scenemux_doctor (health check),
    scenemux_setup (configure + start the server), and the studio tools
    (project / plan / render / status / export).
  • Skill scenemux:studio — the operating knowledge (pipeline lanes,
    engines, happy path, spend rules).
  • Zero Python dependencies — the plugin client is stdlib only.

Install

# From the catalog (once listed)
hermes plugins install scenemux

# From the repo (development / before catalog listing)
hermes plugins install ep150de/scenemux/hermes-plugin

The plugin installs as a standalone client (no server code is copied with
it). For scenemux_setup to launch the studio server for you, set the
plugin's repo_path setting to a full clone of
the repo — the directory that
contains docker-compose.yml + server/ — or start the server yourself and
point base_url at it (see Setup).

Setup

  1. Keys — cloud mode (works on any machine) needs:
  2. Server — Ask your agent: "set up SceneMux in cloud mode." It runs
    scenemux_setup, which launches the server via Docker (detached; log at
    <HERMES_HOME>/plugin-data/scenemux/server-launch.log) and waits for it
    to become healthy. First run builds the image (a few minutes). scenemux_setup
    finds the server code via the repo_path setting (point it at a full repo
    clone) — or, if the server is already running, it just checks health.
  3. Optional — make Hermes itself the director — pass
    director_backend=hermes to scenemux_project and the storyboard is
    planned by this Hermes agent (its full toolset, not an external LLM
    lane). This needs the Hermes API server:
    # add to ~/.hermes/.env, then (re)start the gateway
    API_SERVER_ENABLED=true
    API_SERVER_KEY=<some-long-random-key>
    hermes gateway
    
    The plugin reads API_SERVER_KEY through the secret scope and sends it to
    the server per session (only when the studio server is on loopback) — no
    extra configuration per project. scenemux_doctor
    reports whether the API server is configured and reachable.
  4. Check — scenemux_doctor tells you exactly what's missing, if anything.

Make a video

Just ask: "make a music video for ~/songs/demo.mp3 — dark neon city,
synthwave."
The agent will: doctor → start server → create project + upload
→ plan the storyboard (show it to you) → estimate + confirm → render →
deliver the MP4 (and 9:16 on request).

Security & disclosure

  • Network calls to: the SceneMux server (loopback by default; tool
    calls to non-loopback hosts are refused), Comfy Cloud, Ollama Cloud,
    OpenCode Go, FAL.ai (only the endpoints the server uses with the keys
    you provide).
  • Spawns a long-running process: docker compose up -d for the SceneMux
    server (detached, logs captured; published on 127.0.0.1 only, no
    wildcard CORS). It keeps running after the session ends.
  • Spend: rendering consumes Comfy Cloud credits (and optional FAL
    credits). Every render goes through an estimate, and a confirm=true
    call is escalated to a Hermes human-approval prompt (the plugin's
    pre_tool_call spend gate) — the render cannot start without that
    approval.
  • Secrets are read only through the Hermes secret scope, sent only to
    loopback / host.docker.internal hosts, and written to the server's
    .env (chmod 600) only for a 127.0.0.1-bound server. docker compose
    is spawned with a minimal explicit environment, not Hermes's whole env.
  • No telemetry, no analytics, no credential storage of its own.
  • This plugin does not override any Hermes core behaviour; it only registers
    its tools, its spend-gate hook, and one skill.

Troubleshooting

  • scenemux_doctor says server unreachable → re-run scenemux_setup, or
    check the launch log.
  • HTTP 401/403 from a studio call → a key is missing/invalid; doctor lists
    which one.
  • director_backend=hermes fails → the Hermes API server is not up: doctor
    reports hermes_director.reachable. Fix: API_SERVER_ENABLED=true +
    API_SERVER_KEY in ~/.hermes/.env, then hermes gateway.
  • Home Agent port (e.g. :8765) ≠ Comfy port (:8188).
  • The server itself reads its own .env when run manually (docker compose
    auto-loads the repo .env); the plugin always supplies credentials via
    the environment it spawns, and never reads credential files.

README from the reviewed commit ca6ef43; it updates when the author re-pins.

Developer Mode

Explore built-in Hermes toolsets and actions.

HermesAgent

Developer?

Create a Hermes extension for Desktop or agent workflows.

Developer Section