bmtrnavskyStorm Panel Research
A human-gated perspective-panel adaptation of Stanford's STORM methodology. One question,
several seats, one judge, one human. No hardcoded models: the human picks the engine
every run.
Based on: Shao et al., NAACL 2024 -- "Assisting in Writing Wikipedia-like Articles from Scratch with Large Language Models"
What is it?
STORM (Synthesis of Topic Outlines through Retrieval and Multi-perspective question asking)
is a research pipeline from Stanford's Oval Lab that mimics investigative journalism:
discover the viewpoints a topic deserves, interview each one, ground every claim in
retrievable sources, then synthesize.
This adaptation changes the question. Stanford asks how to write the article. This asks
how to make the call. A DC issue seats the associate, the supervisor, safety, and the
customer. Same question, four truths. The judge synthesizes, the human decides.
Two improvements on Stanford: the human gates four checkpoints (cast, validation, verdict,
audit), and mode 3 gives every seat its own multi-model fanout. Full design: PANEL-SPEC.md.
Architecture
flowchart TD
Q([Question]) --> CAST[Cast seats<br/>orchestrator proposes,<br/>human approves]
CAST --> S1[Seat 1<br/>associate]
CAST --> S2[Seat 2<br/>supervisor]
CAST --> S3[Seat 3<br/>safety]
CAST --> SN[Seat N<br/>...]
S1 --> F1{Fanout}
S2 --> F2{Fanout}
S3 --> F3{Fanout}
SN --> FN{Fanout}
F1 --> R1[Seat report]
F2 --> R2[Seat report]
F3 --> R3[Seat report]
FN --> RN[Seat report]
R1 --> HV((Human<br/>validation))
R2 --> HV
R3 --> HV
RN --> HV
HV --> J[Judge<br/>recommends]
J --> HD((Human<br/>decides))
HD --> M[Moderator<br/>audit]
M --> OUT([Report])
style HV fill:#00C896,color:#0A0F1E
style HD fill:#00C896,color:#0A0F1E
style J fill:#0099FF,color:#0A0F1E
Green nodes are human gates. The judge recommends, the human decides, the moderator audits.
The three modes
flowchart LR
subgraph M1[Mode 1 Solo]
A1[One model<br/>sequential roleplay]
end
subgraph M2[Mode 2 Native]
A2[subagents<br/>one seat each]
end
subgraph M3[Mode 3 Full]
A3[fanout per seat<br/>Fusion or MoA]
end
M1 --> D[POV diversity only<br/>no keys]
M2 --> D
M3 --> E[POV x models<br/>engine key]
| Mode | Seats | Brains | Diversity | Keys |
|------|-------|--------|-----------|------|
| 1 Solo | 2-4 | One model, sequential | POV only | None |
| 2 Native panel | 2-12, parallel subagents | Human's model pick | POV only | None |
| 3 Full panel | 2-12, fanout per seat | OpenRouter Fusion or Hermes MoA | POV times models | Engine key |
In mode 3 a lineup slot can itself be a Fusion or MoA call, so the fanout nests: each seat
fans out across the lineup, and each slot can fan out again inside the engine. Cap: 12
seats x 4 slots per run; dry-run and state the call count first. See PANEL-SPEC.md for
the cost math.
The orchestrator sizes the panel to the problem: societal issue, up to a dozen seats;
broken deploy, two (design and engineering). The human approves or amends the cast.
How it works
Cast -> interview -> human validation -> outline -> grounded writing -> moderator audit.
Unsourced claims are marked unverified. Thin sections say needs more research.
The judge recommends with weighted contradictions; the human makes the call.
Install as a Hermes plugin
hermes plugins install storm-fusion-research
Or clone into your plugins directory. The skill loads namespaced as
storm-fusion-research:storm-fusion-research. Modes 1 and 2 need no keys. Mode 3 needs
one fanout engine (OpenRouter Fusion or Hermes MoA) with a human-chosen lineup.
One-time consent for per-call model picks:
plugins:
entries:
storm-fusion-research:
llm:
allow_provider_override: true
allow_model_override: true
# Optional but safer: limit what the plugin can reach.
# allowed_providers: [openrouter, moa]
# allowed_models: ["*"]
Hermes MoA slots: presets under the top-level moa key, used as provider: moa with the preset name as model, e.g. {"provider": "moa", "model": "<preset>"}.
Hermes runs the inner fanout with your own keys and presets. This is not yet confirmed to work
through plugins on every Hermes version, so run a two-seat test first. If it is refused, list
the preset's reference models as separate slots.
See skills/storm-fusion-research/SKILL.md for the full procedure.
Use it in other agents
Claude Code, Codex, OpenCode, OpenWork, and Pi can all load the skill. See
docs/use-in-other-tools.md.
Use it without Hermes
The method works anywhere. Paste the prompts into any chat assistant, or run
run_panel.py against any OpenAI-compatible endpoint for real multi-model fanout
(standard library only, --dry-run to preview). You pick every model. See
docs/use-without-hermes.md.
Example run
Two seats, active model, one tool call:
{
"topic": "Should a small DC add a second packing station before peak?",
"seats": [
{"name": "associate", "brief": "You pack boxes every shift.",
"questions": ["Where does packing actually bottleneck?"]},
{"name": "supervisor", "brief": "You own the shift plan and labor hours.",
"questions": ["Does a second station pay back in one season?"]}
]
}
Result: associate flagged flow and layout, supervisor ran the capacity math, judge
recommended the station only with handoff fixes after validating peak demand. Seats
report, judge weighs, human decides.
Results
Automatic evaluation on WildSeek with simulated users (Table 3, Jiang et al., Co-STORM, EMNLP 2024). Scores are rubric means on a 1-5 scale; dagger marks denote significant gains over both baselines:
| Metric | RAG Chatbot | STORM+QA | Co-STORM |
|--------|-------------|----------|----------|
| Report Relevance | 3.57 | 3.61 | 3.78 |
| Report Breadth | 3.50 | 3.61 | 3.79 |
| Report Depth | 3.26 | 3.43 | 3.77 |
| Report Novelty | 2.44 | 2.50 | 3.05 |
| Unique URLs | 2.94 | 2.89 | 6.04 |
Key finding: Removing the moderator hurts performance more than reducing the number of experts.
Implementation Notes
- Reference implementation: stanford-oval/storm (31.5k stars)
- Mixture of Agents: Hermes MoA docs
- This repo: bmtrnavsky/storm-fusion-research-pipeline
License
MIT
README from the reviewed commit b1a492e; it updates when the author re-pins.
