System A vs System B
Two architectures, same task, same sources, same model. MASFactory is an orchestrated 7-agent graph; Hermes is a single agent loop with memory + skills. The interesting question is not which wins — it's where and why they diverge.
Why this matters: System B rows before 2026-06-10 came from a Python pattern implementation; from that date forward the real NousResearch hermes-agent CLI is used. See methodology § "System B" for the full identity transition, or restrict the window to compare only the real-CLI era.
Agree on actor
39
Only System A
0
Only System B
1
Head-to-head
| Metric | System A · MASFactory | System B · Hermes |
|---|---|---|
| Runs (ok / error) | 41 / 5 | 33 / 6 |
| Signals collected | 259 | 1368 |
| Distinct actors | 39 | 40 |
| Input tokens | 17,488,241 | 0 |
| Output tokens | 8,449,280 | 0 |
| Signals / 1k tokens | 0.01 | — |
Signals-per-1k-tokens is the thesis's "output quality per token cost" metric, simplified. A system yielding fewer but higher-quality signals looks worse here — pair with the methodology's recall-vs-precision discussion.
Per-actor impact agreement
Each point is an actor. On the diagonal = the two systems agree on its impact. Off-diagonal = they disagree.