webgraft
A Hermes Agent plugin for web answers built from pages that were really read.
- Reads pages and documents on your machine for
web_extract(HTML, PDF, Word, PowerPoint, Excel, OpenDocument, EPUB, RTF, CSV): no API key, no extraction service. - Reports HTTP errors and bot walls as errors, so the model never takes them for content.
- Lists the pages the answer really used, under the answer, with clickable links. Pages are taken from the tool results, never from the model's text, so the list cannot name a page that was not read.
- Pushes the model to open pages instead of answering from search snippets: a bundled skill and a short reminder.
- Keeps several pages within the model's context: one URL per
web_extractcall. Long pages are left to Hermes's own window, whichsetupcan widen with your consent. See What the model sees of a long page.
A real run, fully local: Gemma 4 E4B in LM Studio, DuckDuckGo search, pages read by webgraft.
> On each Nous Portal plan, how much RAM and disk does the included Hermes Cloud agent get,
and what does each plan cost?
* **Free:** $0, pay as you go (No included cloud agent)
* **Plus:** $20 / month (4 GB RAM, 25 GB disk)
* **Super:** $100 / month (4 GB RAM, 100 GB disk)
* **Ultra:** $200 / month (8 GB RAM, 100 GB disk)
Sources (opened pages):
- Hermes Cloud — Always-On AI Agent Hosting: https://portal.nousresearch.com/cloud
- Nous Portal Subscription: Plans, Credits, and Hermes Agent Setup — Agentic Bo...: https://agenticbotsitter.com/guides/nous-portal-subscription-hermes-agent/
Links are written out in full rather than as Markdown links: the Hermes CLI prints replies as plain text, where a bare link stays clickable (Ctrl+click in Windows Terminal) and Desktop and chat apps link it too.
The footer is in English or French. By default it follows Hermes's display.language, like Hermes's own notices; set language: answer to follow the language of the reply instead.
Contents
- Why
- How a page is read
- What the model sees of a long page
- Which pages are listed
- Install
- Settings
- What it does, for review
- Other plugins that rewrite replies
- Limits
- Dependencies
- Development
- License
Why
A web answer can go wrong in three places, and the model cannot see any of them.
-
The page reader drops the data. Main-text extractors discard what looks like page furniture.
- On the Nous Portal home page, trafilatura keeps the text but drops the plan list with each plan's Cloud agent size ("up to 4 GB RAM and 25 GB disk", "up to 8 GB RAM and 100 GB disk"). Without it, the answer above could not be given from that page.
-
The page is not a page. A 404 or a bot wall ("Automated bot check in progress") read as content gives the model nothing, or worse, a page of the wrong text.
- Bot walls often answer HTTP 200.
-
The model answers from snippets or memory. Small models in particular stop at the search results when a snippet holds a number, or fill gaps from memory.
- In tests, a small model gave a version number as the latest release with the releases page open, where no such version appears.
webgraft fixes the first two, pushes against the third, and shows which pages the answer came from so you can check it yourself.
How a page is read
-
What the site says about language models. If the site publishes an
llms.txtthat lists a Markdown (.md) version of the page, that version is read instead of the HTML: cleaner text, and what the site asked for.- The file is fetched once per hour per site and checked: plain text that opens with a
#title, never HTML (many sites answer any address with their home page). - When the site has an
llms.txtbut no.mdversion of the page, a line at the end of the page points the model to it. - A page that asks not to be used by AI comes back as an error, not as content:
noaiin theX-Robots-Tagheader or in a robots meta tag, or the TDM Reservation Protocol (tdm-reservation: 1, header or meta tag). robots.txtis not read: it addresses crawlers, and webgraft reads only the pages the user's agent asks for.
- The file is fetched once per hour per site and checked: plain text that opens with a
-
Download it, with Hermes's own HTTP client and its User-Agent (
HermesAgent). An HTTP error (403, 404, 410, 429, 5xx) is the site's answer and is reported as such: webgraft never tries to get past it. -
Documents (PDF, Office, OpenDocument, EPUB, RTF, CSV): Markdown, tables included, with anydoc, the converter Hermes's own
read_fileuses (itsdoc-extractextra, installed bysetupor on first use).- It runs in a separate process stopped after 60 seconds (and, on Linux, limited to 2 GB of address space and run with 4 threads), so a crafted document cannot hang Hermes or take over the machine.
- A scanned PDF (page images, no text layer) is read by local OCR when it is installed: RapidOCR, no language model, about 1 second per page; the text comes with a note that it was read by OCR. Without OCR, the error names the scanned pages.
HTML: main text with trafilatura.
-
With Crawl4AI installed, a page the site served (status 200, no bot wall) is also opened in a headless browser (the Chromium Hermes installs), with the same
HermesAgentUser-Agent.- Lines of the browser view that hold values missing from the main text, such as spec table rows, are appended under "More values from this page".
- Pages that trafilatura cannot read, because scripts build them (Vue, React...), use the browser view instead: the browser waits for the page to load, then one more second for its scripts to fetch their data.
- The browser only completes what the site served. A page the site refused (403, 429...) or a bot wall is never re-read in the browser: that would be getting past the site's answer.
-
Errors come from the HTTP status only: a 404, 410, 403 or 5xx is reported as such.
- A page that answers 200 is never called an error because of its text, since a table, a price or a blog post can hold "404" or "not found".
- Bot walls can answer 200, so a 200 page is checked for one: the response headers (Cloudflare marks its challenges with
cf-mitigated: challenge), then the text, where a challenge page's own phrases ("Just a moment...", "Checking your browser", "Automated bot check") count on a page of any length, and words an article also uses ("captcha", "access denied") only on a short page that does not declare itself an article in its HTML head. - Only a real HTTP error is reported in Hermes's
HTTP <code> from originform, as Hermes's own backends write it. Walls, AI opt-outs, unreadable documents and pages without text are reported with their reason alone. - What Hermes does next with a failed page is Hermes's decision: with
web.keyless_rescue: true(Hermes's default), a call where every page failed is retried through Hermes's keyless free tier, outside services that fetch the page again from their own servers. To keep pages that refused, or asked not to be used by AI, from being fetched that way, setweb.keyless_rescue: falseinconfig.yaml. - A wall that another backend returns as content is never listed as a source.
-
The page's links whose text is in the content are listed last, under "Links on this page" (40 at most): the products of a listing, the articles of an index, the sections of a documentation.
- Extraction keeps a link's text and drops its target, so without this a model that reads a category page sees names it cannot open and searches for their addresses instead.
- Navigation and footer links, which the extraction already dropped, stay out.
The whole text read (up to 2,000,000 characters, what Hermes stores for one page) goes to Hermes, links last. Hermes decides what the model sees of it.
What the model sees of a long page
Hermes decides, not webgraft. Small local models and models with a short context feel it most.
The window
Hermes shows the model at most web.extract_char_limit characters of each page: 15,000 by default, whichever backend read it. On a longer page the model gets the first 75% and the last 25% of that window.
The full text is saved to a file under HERMES_HOME/cache/web, with a note telling the model which read_file call pages through the rest. Small models rarely do it, so on a long document the answer can sit in the part the model never sees.
Widening it (optional, in setup)
hermes webgraft setup offers to raise web.extract_char_limit to 30,000. It is off unless you say yes (or pass --char-limit N).
- What you gain: the model sees twice as much of a long page inline.
- What it costs:
- each page can take about 7,500 tokens of context instead of 3,750;
- Hermes spills a tool result over 100,000 characters to disk and shows a short preview instead, with a lower threshold on small-context models (Tool-Result Spillover Budget). Below 64K tokens of context, a 30,000-character page can end up as a preview, so
setupdoes not offer the change whenmodel.context_lengthis set below that; - on a paid model, more input tokens per page.
- Undo: setup records the value it replaced, and
hermes webgraft uninstallputs it back. To set it by hand instead:hermes config set web.extract_char_limit 30000. The model can also ask for more on one call (char_limit).
Several URLs in one call
Each page gets its own window, but they all land in one tool result. Hermes spills a tool result over 100,000 characters to disk, with a lower threshold on small-context models, and shows the model a short preview plus the file's path for read_file (Tool-Result Spillover Budget). In Hermes 0.23.1 the thresholds are 15% of the context window for one result and 30% for a whole turn's tool output (tools/budget_config.py).
What webgraft does about it
-
One URL per call. When the model asks
web_extractfor several URLs, webgraft reads only the first one, the model's first choice (Hermes'spre_tool_callhook rewrites the call).- The result then lists the URLs left out (
webgraft_note, through thetransform_tool_resulthook), and the model opens the ones it still needs in their own calls. - Sending the whole call back with an error was tried first: it cost a round trip, and a small model (Ministral 3 3B) then twice opened its second choice and never the page it had ranked first.
- Setting:
one_url_per_call.
- The result then lists the URLs left out (
-
An empty search is not an absence. A
web_searchthat returns no result gets awebgraft_note: a self-hosted backend (SearXNG) answers nothing when the engines behind it rate-limit it, which several searches sent at once trigger, and Hermes reports that as a successful search with zero results.- The note says to search again one query at a time or to open a page that lists the thing, rather than conclude that it does not exist.
For Hermes
Requests for the Hermes maintainers, so plugins would not have to work around the harness:
- Which plugins rewrite replies. Hermes keeps the first non-empty
transform_llm_outputrewrite and gives a plugin no documented way to know whether another enabled plugin registers that hook. A public query (or a way to append rather than replace) would let a footer plugin step aside or combine instead of asking the user to choose. - Redirect checks behind a proxy. The security guide promises that redirect chains are re-validated at each hop, but the public
create_ssrf_safe_async_clientleaves proxied hops to the proxy. A public option or helper for that check would replace webgraft's own response hook. - The extract cache. The 20-minute extract cache is documented for users (Result caching) but not in the web provider plugin guide: a provider is not told that a cache hit can skip it, and the
cachedflag Hermes sets on a hit is dropped before the result reaches plugin hooks.
Which pages are listed
Every page web_extract returned with real content in this turn.
- A page opened in an earlier turn of the conversation is added under "Also used, opened earlier in this conversation" when the answer contains one of its values with a unit, a price or a version: an answer may rely on a page read a few turns before.
- When no page was opened, the footer says that the answer relies on search snippets only.
- When the turn searched and the answer holds a value (with a unit, a price or a version) that appears in the search snippets but on no page read in the conversation, a last line names it: "From search snippets, not from an opened page: 2617 MHz".
- That is a presence check on the number and its unit, not a judgement of the value.
Install
Requires Hermes Agent 0.22.6 or later.
hermes plugins install ElSerda/webgraft
hermes webgraft setup
Hermes asks before installing the plugin's one Python dependency (trafilatura).
After a later hermes plugins update webgraft, run hermes gateway restart if a gateway is running: on Hermes 0.23 chat sessions run the plugin code the gateway loaded, and an update does not reload it.
Then setup:
- chooses a search backend: your SearXNG instance (it checks that JSON output is enabled) or DuckDuckGo (
ddgs, built into Hermes, no key); - sets
web.extract_backend: webgraft, and installs Hermes's document converter (doc-extract) if missing; - offers Crawl4AI, which is optional;
- offers local OCR for scanned PDFs (RapidOCR), which is optional;
- offers to raise Hermes's
web.extract_char_limitto 30,000 (off unless you say yes; see Widening it); - offers to auto-load the bundled skill
webgraft:webgraft(it tells the model to open pages instead of answering from snippets, one URL per call; about 600 tokens per session); - tests one real page and one missing page.
Non-interactive: hermes webgraft setup --yes --search ddgs --crawl4ai --ocr --skill (add --char-limit 30000 to raise the window).
Check at any time with hermes webgraft status.
Uninstall
hermes webgraft uninstall
hermes plugins remove webgraft
uninstall puts back the web backends and the web.extract_char_limit that setup replaced (it records them in HERMES_HOME/plugin-data/webgraft), removes the skill from skills.auto_load, and deletes the Crawl4AI and OCR environments. Hermes's Chromium stays: it belongs to Hermes.
hermes plugins remove alone would leave web.extract_backend pointing at a backend that no longer exists.
Crawl4AI
Crawl4AI pulls about 35 packages, so it is not installed into Hermes's environment: setup creates a separate one with hermes pm (HERMES_HOME/environments/webgraft-crawl4ai).
webgraft downloads no browser. Crawl4AI drives the Chromium that Hermes's own browser tool uses, found in the order Hermes documents (Browser, "Which Chromium"). Hermes has no public function for it, so webgraft follows that order itself; on a test machine both pick the same executable:
browser.executable_pathinconfig.yaml,- then the Chromium installed by
hermes pm install chromium(pinned and checked by Hermes), - then
/etc/hermes/chromium-executable-path(Docker image), - then a
chromiumon PATH.
If there is none, setup offers hermes pm install chromium.
For each page, a fresh headless instance starts with a temporary profile and a DevTools port on 127.0.0.1, Crawl4AI connects to it, and it is stopped and its profile deleted when the page is read.
Without Crawl4AI, pages are read with trafilatura only, and lists and tables it takes for page furniture (the plan list above) stay out.
Local OCR
Scanned PDFs hold page images, not text. With OCR installed, webgraft renders the scanned pages (pypdfium2) and reads them with RapidOCR (PaddleOCR models in ONNX format, run by onnxruntime): no language model, no service, about 1 second per page, up to 30 pages per document.
Like Crawl4AI, it lives in its own hermes pm environment (HERMES_HOME/environments/webgraft-ocr, about 250 MB). On first use RapidOCR downloads its models (about 32 MB) from its official repository on modelscope.cn and checks each one against the SHA-256 it ships with.
Hermes's own read_file routes scanned pages to a vision model instead (vision_analyze); webgraft keeps that for what OCR cannot read, such as figures and handwriting.
Settings
Under Settings → Plugins in Hermes Desktop, or plugins.entries.webgraft.settings in config.yaml:
| Key | Default | Meaning |
|---|---|---|
| language | auto | Footer language: auto (Hermes's display.language; English for languages other than en/fr), answer (the reply's language), en or fr |
| show_sources | always | always: list the opened pages under the answer. never: no list (with another plugin that rewrites replies, see below). Older values: auto means always |
| remind | true | Add a short note to each user message (about 110 tokens), marked as a note and not the request: search snippets are not sources, open the pages, and open the pages a multi-URL call left out. When the message holds a URL, the note asks to open that page instead of searching |
| one_url_per_call | true | A web_extract call that asks for several URLs reads only the first; the result lists the others |
| llms_txt | true | When the site's llms.txt lists a Markdown version of the page, read that instead of the HTML |
| respect_ai_optout | true | A page that asks not to be used by AI (noai, tdm-reservation: 1) comes back as an error |
Search and page reading are separate settings: webgraft only reads pages. It works with any search backend.
What it does, for review
-
Hooks:
pre_llm_call(starts a record for the turn and, unlessremindis off, appends the short reminder to the user message)pre_tool_call(only forweb_extract: keeps the first URL of a call that asks for several whenone_url_per_callis on)transform_tool_result(adds the URLs left out to the result of such a call, and a note to aweb_searchresult that holds no result)post_tool_call(keeps the text ofweb_extractresults and the snippets ofweb_searchresults)transform_llm_output(appends the footer; the answer itself is not changed, so a streamed answer is not printed twice)
-
Web provider:
webgraft, extract only. -
CLI:
hermes webgraft status|setup|uninstall. -
Skill:
webgraft:webgraft, read-only, loaded only if you auto-load it or call it. -
Network: downloads the URLs the agent asks
web_extractto read, directly from this machine, with Hermes's HTTP client and its User-Agent (HermesAgent, the browser uses the same), and, withllms_txton, each site's/llms.txtonce per hour (plus the.mdversion of a page it lists).setupalso checks the SearXNG URL you give it and reads two test pages on iana.org; with OCR chosen, RapidOCR downloads its models once from modelscope.cn (SHA-256 checked).- Crawl4AI's browser loads the same URLs, runs their scripts and loads their page assets.
-
Private addresses:
- pages,
/llms.txtand.mdversions are fetched with Hermes's SSRF-safe client (tools.url_safety), which refuses private, loopback, link-local and cloud metadata addresses on every redirect hop; - with a proxy (
HTTP_PROXY,HTTPS_PROXY,ALL_PROXY), Hermes's client checks only direct connections and leaves each hop to the proxy: webgraft then checks the target of every redirect itself before it is followed. Which addresses the proxy itself can reach is the proxy's business; - a result names the page actually read in
url(the requested URL is inmetadata.sourceURL), so Hermes re-checks a redirect target against its SSRF and website policies; - the browser starts only after that download, on the address it reached, and its view is dropped when the page it ended on (after its own redirects and script navigation) is not a public address, or is on another site than the page downloaded;
- Limit: only the page the browser ends on is checked. What that page itself requests (images, scripts,
fetchcalls) and the browser's own DNS lookups are not, so a hostile page can make the browser send requests to your network, as any browser that opened it could. If that matters on your network, skip Crawl4AI insetup: pages are then read by the SSRF-safe client alone.
- pages,
-
Processes:
- one Crawl4AI child process per page, at most three at once in the whole Hermes process, each stopped after 60 seconds; it starts Hermes's Chromium headless (DevTools port on 127.0.0.1, temporary profile) and stops it before exiting;
- one child process per document (Hermes's own Python running
doc_reader.pywith anydoc), stopped after 60 seconds; - for a scanned PDF with OCR installed, one more child process (
ocr_runner.pyin the OCR environment), stopped after 100 seconds. - Each child is also stopped when Hermes ends the call first (
web.extract_timeout, 120 seconds by default) or the turn is interrupted. A Crawl4AI read is stopped with its Chromium; the browser profile and Crawl4AI's own log and cache live in a temporary folder that webgraft creates and removes, killed read or not. Folders of that kind left by an earlier run (Hermes stopped mid-read, or an older webgraft) are removed when they are more than an hour old, once per Hermes process, at the first browser read; only webgraft's ownwebgraft-browser-,webgraft-chromium-andwebgraft-crawl4ai-folders in the temp folder are touched. - Children start in the plugin's folder, and programs are run by full path (
taskkill) or looked up in the absolute folders ofPATHonly (hermes,chromium): on Windows, a program name alone is first looked up in the current folder. - These children get a minimal environment (path, home and temp folders, locale, proxy settings, and what each one needs), never Hermes's API keys.
setupandstatusalso run Hermes's ownhermes pmcommands, as Hermes would.
-
Writes:
web.search_backend,web.extract_backend,skills.auto_load,web.extract_char_limit(only if you accept it) and, for SearXNG,SEARXNG_URLin.env, all through Hermes's own config functions and only duringsetupanduninstall;- the values
setupreplaced, inHERMES_HOME/plugin-data/webgraft/previous.json; - the Chromium path, in
browser.jsonnext to it.
Nothing else is stored: page text lives in memory only, for at most 30 pages per conversation and 20 conversations, and so does which sites publish an
llms.txt(for an hour); all of it is gone when Hermes stops. -
Reads:
web.extract_char_limit(forsetupandstatus) anddisplay.languageinconfig.yaml;hermes pm env chromiumandbrowser.executable_path, duringsetup, to find Hermes's Chromium.
-
No telemetry, no credentials read, no self-update.
Other plugins that rewrite replies
Hermes runs every plugin's transform_llm_output hook but keeps the first non-empty rewrite of a reply (hooks guide), so two plugins that append a footer hide one another.
Hermes gives a plugin no documented way to know which other plugins register that hook, so webgraft does not guess. If you run another plugin that rewrites replies, choose which one shows: show_sources: never turns webgraft's footer off (page reading and the reminder keep working), or disable the other plugin. A public way to ask is among the requests for Hermes.
Limits
-
From Hermes itself (the harness), documented rather than worked around:
- One rewrite per tool result. Like replies, a
web_extractorweb_searchresult keeps only the first plugin rewrite (transform_tool_result). webgraft rewrites a result only to add its note (URLs left out, a page that could not be read, an empty search); when another plugin rewrites the same result, one of the two rewrites is lost, depending on load order. - Session ids.
pre_llm_callandtransform_llm_outputreceivesession_idonly, not theconversation_idthe plugin guide recommends for per-conversation state. When Hermes compresses a conversation into a new session (compression.in_place: false), webgraft starts over: pages read before do not appear under "opened earlier". - Proxies. Hermes's security guide says redirect chains are re-validated at each hop, but its public SSRF-safe client leaves proxied hops to the proxy. webgraft checks redirect targets itself (see "Private addresses"), and asks Hermes for a public way (requests for Hermes).
- One rewrite per tool result. Like replies, a
-
From Crawl4AI and Chromium: only the page the browser ends on is checked (see "Private addresses"). Hermes takes the same stance for its own local browser: with a local terminal backend it "can open
localhostand LAN addresses" and "reaches nothing the terminal can't" (Browser). Skip Crawl4AI insetupif that matters on your network. -
What is mechanical and what is an estimate.
- Mechanical: HTTP errors, document conversion and OCR,
llms.txt, AI opt-outs, one URL per call, the links of a page, the empty-search note, and the list of opened pages (from tool results). - Estimates: bot walls on pages that answer 200 (phrases and headers), the browser-view complement (which lines hold missing values), the "opened earlier" pages (shared values), and
language: answer. - Each estimate is additive, shown as such (a note, a heading), and can be turned off, except bot-wall detection, which removes a page and says why.
- Mechanical: HTTP errors, document conversion and OCR,
-
The footer says which pages the answer was built from. It does not check that each sentence of the answer matches them: that would take another model call per reply and still not be reliable.
-
An earlier page is matched on values with a unit, a price or a version; an answer that reuses only its prose is not linked to it.
-
Pages behind logins or strong bot protection cannot be read; they come back as errors.
-
Hermes refuses private and internal addresses (127.0.0.1, 10.x, 192.168.x, link-local) before any extraction backend runs: an agent reading untrusted pages must not be able to reach your network.
- webgraft applies the same rule to redirects, and to the page the browser ends on (see "Private addresses" above for what that does not cover).
- The result is "Blocked: URL targets a private or internal network address".
- To read pages on your own LAN, set
security.allow_private_urls: truein Hermes'sconfig.yaml; cloud metadata addresses stay blocked.
-
Hermes keeps
web_extractresults for 20 minutes (web.cache_ttl_minutes), across sessions, and a cached page does not go through webgraft again.web.cache_enabled: falseturns the cache off.
Dependencies
Declared in pyproject.toml, installed by Hermes inside its own constraints:
| Package | Range | Floor chosen because |
|---|---|---|
| trafilatura | >=1.10,<3 | oldest release that accepts lxml 6, which Hermes pins (tested 1.10 to 2.3) |
| rapidocr, onnxruntime, pypdfium2 | >=3.10,<4, >=1.20,<2, >=5,<6 | separate OCR environment, installed by setup; tested with RapidOCR 3.10, onnxruntime 1.31 and Python 3.14 |
| crawl4ai | >=0.9.1,<0.10 | separate environment, installed by setup; 0.9.0 and older need lxml 5, which has no Python 3.14 wheels (tested 0.9.1 to 0.9.5) |
Development
python tools/check.py
Runs everything locally, no cloud CI:
- for Python 3.11 and 3.14,
uvbuilds a throwaway environment with the dependencies frompyproject.toml(only releases at least 3 days old, the release-age quarantine the Hermes plugin guide recommends) and runs the tests; - then the Hermes on the machine runs
hermes plugins doctor . --ci(loads the plugin the way Hermes does) andhermes plugins validate . --install-deps(the catalog CI check).
--python 3.12 tests other versions, --skip-hermes runs the tests only.
The tests use made-up pages and need no network; the document tests start a child Python.
Documents are read with anydoc (firecrawl-anydoc), which Hermes provides, so it is not a plugin dependency; tools/check.py installs it in the test environment.
Built for the Nous Research Hermes hackathon (November 2026).
License
MIT
README from the reviewed commit 3d04f7f; it updates when the author re-pins.

ElSerda