hermes-genui

Ask "How's the market?" and get a chart, not a paragraph. Hermes GenUI lets your Hermes agent answer with
real, interactive UI inside Hermes Desktop chat replies: charts, tables, day-by-day plans, photo strips, calculators
with live charts, forms, checklists that remember your ticks, and step-by-step explainers.

https://github.com/user-attachments/assets/5b274da9-b756-4bb6-be72-9a5427fadc0d

A one-minute demo with sound: a Yosemite trip plan, a stock comparison, an explainer and a 401(k) what-if.
▶ Also on YouTube

✨ What it looks like

| | |
|:---:|:---:|
| | |
| "Compare NVDA, AAPL and MSFT over the past year" | "Plan a 4-day Yosemite trip for two in May, around $2,000" |
| | |
| "Explain how transformers work" | "I put 6% in my 401(k). What if I raise it to 10%?" |

The model decides what to show. The plugin decides how it looks, so every answer comes out in one
consistent, accessible design that follows your light or dark theme and your language's number format
(4.21B, 4,21 Mrd., ₹18.4L).

🧱 The blocks

| Family | Blocks |
|---|---|
| 📈 Charts | line (with gain/loss colouring), columns, horizontal bars, donut, pie, stacked, area, combo, heatmap, all with hover cards and keyboard support |
| 🔢 Numbers | KPI cards with trend lines, single stats, progress toward a goal, streaks |
| 📋 Tables and lists | sortable tables with mini bars, ranked lists with icons or photos |
| 🗓️ Plans | timelines (day by day, timed stops), checklists whose ticks are saved |
| 🖼️ Media | photo strips with credits, cover tiles, flash cards |
| 🎛️ Interactive | calculators with sliders and live charts, forms that ask before a big answer, buttons that ask the next question |
| 🧠 Explainers | flow diagrams whose boxes show what's inside them as you step through, simulations you play over time (with sliders), and stacks you can turn in 3D |
| 🧩 Layout | columns, grids, tabs, foldable sections, rows, headings, callouts, sources |

Plus recipes: ready-made plans the model follows for common asks. They cover trip planning, comparisons,
what-ifs, stock dashboards, dashboards, reports, learning, explainers and trackers. A recipe builder lets you
save your own.

💬 What you can ask

Pretty much anything. There's no fixed set of screens: the model builds each answer from the blocks above to
fit the question, so something nobody planned for still comes back as a report, a dashboard or a walkthrough. For
bigger asks it may show a short form first to check the details.

Bring your own data. The blocks aren't tied to this plugin's data tools. Whatever your agent can reach (your own
tools, MCP servers, files, web search) can feed a board: ask it to fetch the data and show it, and the model lays it
out with the same components.

[!TIP]
Use a capable model. A good board means planning a layout and writing valid structured data in one go. Tested
so far:

• DeepSeek V4.1 Flash (Ollama Cloud): works great.
• Qwen3.6 35B-A3B (local, MLX 4-bit): works well.
• Gemma 4 26B-A4B (local): fine for small boards, but big ones sometimes come back as text instead of a drawing.

Each model lays out its answers its own way; the plugin keeps the look the same. If boards come out thin or keep
failing their checks, try a stronger model.

| | Try asking |
|---|---|
| 📈 Markets | How's the S&P 500 doing this year? · Compare NVDA, AAPL and MSFT over the past year · Build a dashboard comparing Tesla, Ford and GM, with a tab each · How's the NIFTY doing this year? |
| 💰 Money | Mortgage on a $450,000 home, let me play with the rate · What will $500 a month be worth in 20 years? · I put 6% in my 401(k). What if I raise it to 10%? · EMI on a ₹40 lakh home loan |
| ✈️ Trips and plans | Plan a 4-day Yosemite trip for two in May · Show me Kyoto · A packing checklist for Iceland in June · Plan a 5-day Kerala trip for a family of four |
| 📊 Reports | Write a report on the electric vehicle market · Top 10 companies by market cap · Where does my $4,000 a month go? |
| 🧠 Learn and explain | Explain how transformers work · How does a card payment go through? · Explain how compound interest grows my money · Make me flash cards on photosynthesis |
| 🔌 Your own data | Read sales.csv and build me a dashboard · Use the GitHub tools to chart my open issues by label · Pull today's tickets from the helpdesk MCP and rank them by urgency |
| 🔁 Track | Track my 30-day walking streak · A checklist for moving house (ticks are saved) |
| 🛠️ Your own | Make this a recipe so you always build my weekly review this way (the recipe builder saves it) |

Buttons under each board ask the obvious next question, such as Make it $1,500, Add a day in Sequoia or More
technical, so you can keep going without typing.

🚀 Install

You need Hermes Agent 0.22.5 or later with Hermes Desktop.

hermes plugins install sagarmainkar/hermes-genui --enable   # add -p <profile> for a profile other than default
hermes config set tools.tool_search.enabled off             # keeps the plugin's tools in view (why: below)
hermes gateway restart                                       # loads the plugin's tools and data routes

Prefer clicking? In Hermes Desktop, open Capabilities → Plugins → Install from Git and enter
sagarmainkar/hermes-genui, then run the second and third lines above.

[!IMPORTANT]
Why turn off tool search? Hermes's default (auto) hides plugin tools behind a search bridge, where the model
has to write each board as escaped text inside another call. Big boards often break there: you'll see red
"Unnamed call" rows before the board appears. With tool search off, every tool in that profile is offered on each
message, so a profile with many MCP servers uses more tokens. A separate profile just for hermes-genui avoids that.

Then turn on the drawing half in Hermes Desktop:

1. Open Capabilities → Plugins.
2. Pick the profile you installed into at the top of the page. The list is per profile.
3. Open hermes-genui and switch Desktop on.

Hermes on one machine, Desktop on another? (say a Mac mini at home and a laptop that connects to it) Run the
commands above on the machine that runs Hermes. Then, on each computer whose Desktop connects to it, use Install from
Git with sagarmainkar/hermes-genui and switch Desktop on. The drawing half lives in each Desktop app, not on
the server.

Start a new chat and try:

Compare NVDA, AAPL and MSFT over the past year · Plan a 4-day Yosemite trip for two · Explain how transformers
work · Mortgage on a $450,000 home, let me play with the rate · How's the NIFTY doing this year?

🔑 Optional keys, for more and better photos

Photos work with no keys at all, from Wikimedia Commons, Openverse and the pictures on pages your own web search
finds. Each key below adds one more source. Add them under Settings → Plugins → hermes-genui in Hermes Desktop;
they're stored by Hermes, never by the plugin.

| Key | Adds | Get one |
|---|---|---|
| UNSPLASH_ACCESS_KEY | Unsplash photos (only the Access Key; the secret key is never used) | unsplash.com/developers |
| PEXELS_API_KEY | Pexels photos | pexels.com/api |
| TAVILY_API_KEY | Tavily image search | tavily.com |

🔍 How it works

flowchart LR
  A["You ask"] --> B["The model writes blocks<br/>as JSON"]
  B --> C["The plugin checks them<br/>and stores the board"]
  C --> D["The reply carries a tag<br/>::blocks{id}"]
  D --> E["Hermes Desktop fetches<br/>the board and draws it"]
  C -. "Telegram, CLI" .-> F["Readable Markdown<br/>instead"]

• The model learns the blocks from the show_blocks tool's description, and gets a short drawing rule in its
  system prompt.
• Tools fetch real data: market_history for prices, find_photos for photos. So numbers and pictures come from
  sources, not memory.
• Every block is validated before it's stored, and the model gets exact errors to fix. Desktop's readers also
  skip anything malformed, so a bad block never breaks the rest of the board.
• Outside Desktop (Telegram, the CLI and other chat apps), the same board is sent as readable Markdown.

🔒 Privacy, network and security

What leaves your machine (only when the model calls the tool that needs it):

| When | Who is contacted |
|---|---|
| Market prices | Yahoo Finance (query1.finance.yahoo.com) |
| Photos | Wikimedia Commons and Openverse APIs; Unsplash, Pexels or Tavily only if you added their keys; your configured web search; up to 256 KB of the top result pages (read only to their </head>) for their preview pictures; each photo host, briefly, to check the picture loads |

What stays local:

• Boards are saved as JSON in your profile, under plugin-data/hermes-genui/, with checklist ticks next to them.
• There's no telemetry and no account.

What the plugin adds to the model's context:

• a short drawing rule in the system prompt (under 1,500 characters);
• the show_blocks tool description (the block catalog);
• a one-line reminder next to messages that ask for something visual. This is added only in Desktop.

Safety by design:

• The model never sends code. It sends data only: no HTML, scripts or SVG. Every block is drawn by the plugin's
  own components.
• Photos are fetched over https only, from public addresses only.
  • DNS is pinned against rebinding.
  • Each redirect is checked again, up to 3.
  • Size and time limits apply.
• Buttons can't send slash commands, shell lines or @file references.
• Calculator formulas are read by a small parser with no eval and with operation budgets.
• Board size, depth, text length and counts are all capped.

Found a security issue? Please see SECURITY.md.

🛠️ Troubleshooting

| You see | Do this |
|---|---|
| The plugin isn't in Capabilities → Plugins | Switch the profile selector at the top to the profile you installed it in. |
| "Couldn't load this block" | Run hermes gateway restart after installing or updating. |
| A board arrives as text | You're outside Hermes Desktop (Telegram, the CLI), which is by design. In Desktop, check the plugin's Desktop switch. |
| Red "Unnamed call" rows, or tool_call 'calls' is not valid JSON | Tool search is on. Run hermes config set tools.tool_search.enabled off (add -p <profile>), then start a new chat. |
| The model built an HTML widget instead | Start a new chat, or ask "draw it with blocks". Tool search being on can also cause this; see Install. |
| Few or slow photos | Add a photo key above. A source that fails is skipped automatically and retried after 10 minutes. |

🤝 Contributing

New recipes, fixes and new blocks are all welcome. See CONTRIBUTING.md for:

• the layout;
• how to add a recipe, a data tool or a block type;
• the checks every pull request must pass.

New block types start as an issue first. Each one costs space in the model's guide on every message, so we agree on
the design before the code.

💛 Credits

Built by Sagar Mainkar, with Claude as a pair programmer. Made for the
Hermes Agent community.

Sagar also makes Odie, an open-source (MIT) personal assistant that lives
on your Mac and talks to you on your phone, with this kind of UI built in.

Released under the MIT licence.

README from the reviewed commit 43de4cd; it updates when the author re-pins.