All plugins

pilot

Give Hermes a real interactive terminal. Pilot runs commands inside a private tmux server, so the agent can drive programs that need a TTY: ssh logins, vim/nano, htop/btop, fzf, less, package installers, ncurses menus and REPLs. It types literal text and named keys, reads the pane back as plain text, and keeps sessions alive across tool calls; a human can attach to the same pane and take over. The suite exposes 24 tools for session control, scripted expect flows, scrollback search, snapshots/diffs, logging, signals, resize, health and base64 file transfer. Disclosure — Pilot starts a private, profile-scoped tmux server as your user and runs model-requested commands and keystrokes in it (each one asks for your approval); terminal text, including output from remote hosts, enters the model's context; snapshots, pane logs and downloaded files can hold sensitive output and are kept under plugin-data/pilot; pilot_upload can type a local file into a remote session after approval.

LangerSword4Toolsv0.4.1Updated 20 hours ago

About

Give Hermes a real interactive terminal. Pilot runs commands inside a private tmux server, so the agent can drive programs that need a TTY: ssh logins, vim/nano, htop/btop, fzf, less, package installers, ncurses menus and REPLs. It types literal text and named keys, reads the pane back as plain text, and keeps sessions alive across tool calls; a human can attach to the same pane and take over. The suite exposes 24 tools for session control, scripted expect flows, scrollback search, snapshots/diffs, logging, signals, resize, health and base64 file transfer. Disclosure — Pilot starts a private, profile-scoped tmux server as your user and runs model-requested commands and keystrokes in it (each one asks for your approval); terminal text, including output from remote hosts, enters the model's context; snapshots, pane logs and downloaded files can hold sensitive output and are kept under plugin-data/pilot; pilot_upload can type a local file into a remote session after approval.

Pilot

A real TTY for Hermes.

Hermes's terminal tool runs one command through a pipe. Programs that ask a question, draw a full screen,
or wait for a keystroke die there: ssh logins, vim, htop, fzf, less, installers, gh auth login,
REPLs. The usual workarounds are -y flags, yes |, expect, script -qec, or giving up.

Pilot gives the agent a real terminal: a private tmux server, one session per task. It opens a program,
reads the screen as text, types literal text and named keys, waits for a pattern, and keeps the session
alive across tool calls. You can attach to the same pane and take over at any time.

demo: video ·
site: langersword.github.io/hermes-portal ·
catalog: PR #153787

If this is useful, a star helps the next person find it.

the plugin, recordedthe page, hosted
Pilot typing in a real tmux panePilot site on GitHub Pages
recorded: real keystrokes, 900×506hosted: GitHub Pages, 1600×900

How it works

How Pilot works: a coloured whiteboard sketch

Agent calls pilot, pilot drives a private tmux pane, the screen comes back as text, and you can attach
to the same pane at any time.

Five decisions carry it:

  • Profile-scoped tmux server (-L hermes-pilot-<id>): your own tmux sessions are never listed or touched,
    and another Hermes profile gets a different socket.
  • remain-on-exit before the pane is created: a command that exits instantly still leaves its final
    screen and #{pane_dead_status} to read.
  • Text, not ANSI: capture-pane -p -J returns the screen a human sees, with no escape parser.
  • wait before reading: full-screen apps redraw asynchronously, so poll for a pattern or a quiet pane
    instead of sleeping.
  • argv, not interpolation: commands go through sh -c as argv, named keys are allow-listed, and session
    names match [A-Za-z0-9_-]{1,32}.

Tools

The suite registers 24 tools. pilot keeps the full open/screen/type/wait loop; the focused tools are
cheaper to call and add operations the single tool never had.

toolwhat it does
pilotthe full loop: open, screen, type, wait, list, status, attach, close
pilot_openopen a program in a new persistent pane
pilot_screenread the visible screen plus scrollback
pilot_typetype literal text and named keys
pilot_waitwait for a regex, or for the pane to settle
pilot_listlist sessions with pid, command, size, dead state, exit status
pilot_statusstatus for one session
pilot_attachprint the human attach command
pilot_closeclose one session
pilot_killallclose every session
pilot_runone-shot command in a fresh TTY; output, exit status, duration
pilot_expectscripted wait/type/keys/sleep sequence with a step trace
pilot_searchregex search the screen and scrollback with context
pilot_diffsnapshot a pane, then diff the current screen
pilot_logrecord a pane to a log file, stop, or read the tail
pilot_signalsend Ctrl-C, Ctrl-\ or Ctrl-Z
pilot_resizeresize the pane
pilot_renamerename a session
pilot_healthtmux, socket, session counts and limits
pilot_uploadwrite a local file into a shell session over base64
pilot_downloadread a remote file out through base64
pilot_sendtype text/keys and optionally wait in one call
pilot_snapshotsave the current screen to a text file
pilot_cleanupreap finished sessions past a TTL

State survives between calls: a REPL keeps its variables, ssh keeps its shell, vim keeps its buffer.

Terminal backend

hermes pilot backend on sets terminal.backend: pilot. Every terminal call then runs in a persistent
tmux shell instead of a pipe:

  • cd, exports and aliases survive across calls and turns;
  • TERM=tmux-256color and a real /dev/pts, so pagers and curses work;
  • a timed-out command returns exit 124 and is left running; pilot screen and pilot type can inspect it;
  • the next call refuses to type into a running program (exit 126, pilot_busy: true), so a REPL never
    receives your next echo — and once that program finishes, the following call recovers on its own;
  • output is bounded to the last 20K characters of the pane: a long build never dumps stale scrollback
    into the model, and the completion marker at the tail always survives the trim;
  • if the session is lost before the command is delivered, the shell is recreated once and the
    command retried; loss after delivery is reported as indeterminate (pilot_indeterminate,
    exit -1) and never auto-retried, so non-idempotent commands (rm, mv, deploys) are not duplicated.

Long-running servers belong in pilot_open sessions: they outlive terminal calls, while jobs started
inside the terminal shell end when that session is cleaned up.

Restart once from outside the agent with hermes gateway restart. Turn it off with hermes pilot backend off.

Run it

git clone https://github.com/LangerSword/hermes-pilot ~/.hermes/plugins/pilot
hermes plugins enable pilot
hermes pilot list            # sessions, pids, attach commands
hermes pilot attach live1    # Ctrl-b d to detach
hermes pilot killall         # closes tool sessions; live terminal-backend shells survive
hermes pilot killall --include-terminal   # explicit full reset

Needs tmux 3.x on PATH. No Python dependencies. The pilot tool appears in the next session.
Backend switching needs a restart and plugins.isolation: in_process (the default).

Limits

  • Text, not pixels: it reads the terminal grid, so GUI apps are out.
  • Full-screen programs redraw. wait for a pattern before treating the screen as final.
  • An unclosed session keeps running; hermes pilot killall cleans up tool sessions. Terminal-backend
    shells (term-*) belong to live tasks and survive killall unless --include-terminal is passed.
  • It adds a TTY, not privilege: the agent could already run commands as you.

Safety

  • Mutating tool calls (pilot, pilot_open, pilot_run, pilot_type, pilot_send, pilot_expect,
    pilot_upload, pilot_download) go through Hermes's pre_tool_call approval gate. Read-only
    screen, wait, list, status and attach calls do not. The rule key is pilot:<tool>;
    unattended runs fail closed.
  • Panes inherit a minimal environment: PATH, HOME, USER, LANG/LC_*, TERM, SHELL, TZ,
    TMPDIR and XDG_RUNTIME_DIR. Provider keys and adapter tokens are not passed. Explicit per-pane
    env values are added when a caller asks for them.
  • pilot_snapshot and pilot_download resolve writes under plugin-data/pilot and refuse paths that
    escape it. pilot_upload refuses sources under the Hermes home, ~/.ssh, ~/.aws, ~/.gnupg,
    key stores and .env/key files. Logs live in the same plugin data directory.
  • The plugin has no credential store of its own. Terminal snapshots, pane logs and downloaded files
    can contain sensitive output and persist under plugin-data/pilot when those tools are used.

Evidence

checkcommandresult
unit testspython3 -m unittest discover -s tests84 tests, OK, against real tmux
admissionhermes plugins validate .validation passed
runtimehermes plugins doctor .24 tools, 1 hook registered

The suite covers stdin, REPL, vim keystrokes, fzf, exit status, dead-pane reaping, large-output
completion, timeout recovery, session-loss recovery safety (no re-execution after delivery),
shared busy state, killall scoping and the send Enter contract. Every test module rides a private
tmux socket (HERMES_HOME isolation), so the suite never touches live sessions. scripts/demo.py
runs the same flow on camera.

Stack

componentrole
Python 3 standard libraryplugin, tmux wrapper, CLI
tmux 3.xpanes, screen capture, keys, exit status
Hermes >= 0.22plugin host, tool and terminal backend registration

Layout

plugin.yaml        manifest (24 tools)
__init__.py        register(ctx): tools, terminal backend, /pilot, hermes pilot
pilot_tmux.py      tmux layer: open, screen, type, wait, status, close, attach
pilot_env.py       terminal backend: persistent shell, markers, busy guard, timeout handoff
pilot_tools.py     legacy pilot tool schema and JSON handlers
pilot_toolset.py   the 24-tool suite: schemas, wrappers, dispatch
pilot_features.py  run_once, expect, search, log, transfer, safe paths, health
pilot_hooks.py     pre_tool_call approval policy for mutating tools
pilot_cli.py       hermes pilot and /pilot
tests/             55 stdlib unittest cases (skip cleanly without tmux)
scripts/demo.py    recorded demo: stdin, REPL, vim, fzf, htop
art/media/         README media: demo gif, site shot, whiteboard

MIT, Lakshaya Sharma (@LangerSword) · v0.3.2.

README from the reviewed commit 3724e8f; it updates when the author re-pins.

Developer Mode

Explore built-in Hermes toolsets and actions.

HermesAgent

Developer?

Create a Hermes extension for Desktop or agent workflows.

Developer Section