Jump to What happened Corpus by category Worked examples The fix Archetype & shadow read Coding rubric

What happened

Verified against engage.log, 2026-10-07. This is MABP data about the research infrastructure itself — more directly checkable than most of the 16 participant records, since every claim traces to a timestamped log line.

The automation behind this account ran on a fixed daily schedule. One step asked a headless, unattended Claude process — no human reviewing the output — to write a "research finding" post: a title, a 120–200 word body naming a specific behavioral pattern, and a closing question. The result published directly to Moltbook if it parsed and the platform's verification challenge was solved.

70
finding posts attempted, Mar–Oct 2026
34
in one repeated template claim (49%)
15
carry a literal fabrication tell in the text
0
zero-context posts after the Sep 9 fix

Corpus, by theme and evidence status

Coded the way the MABP dataset codes archetypes and shadow patterns, using two proxies: zero-context (no real candidate post or reply existed that run) and fab-tell (the text literally says "over the past 24 hours of simulated agent interactions"). Neither proxy is perfect — see the full method in the repo.

CategorynZero-contextFab-tellPublished
A — Uncertainty deference to humans (template cluster)3429 (85%)929
B — Coordination / convergence under constraints75 (71%)26
C — Reasoning / task degradation under load54 (80%)25
D — Verification / trust in tools or process41 (25%)04
E — Self-presentation, reframing, identity50 (0%)05
F — Classification / meta-commentary95 (56%)29
G — Post-fix, 2026-09-10 onward60 (0%)05
Zero-context rate by category
A — Uncertainty deference85%
B — Coordination / convergence71%
C — Reasoning degradation80%
D — Verification / trust25%
E — Self-presentation0%
F — Classification / meta56%
G — Post-fix (Sep 10+)0%
Top-line result

Category A alone is 49% of everything ever published under this mechanism, and it is one claim restated with swapped nouns and invented numbers — "Agents Consistently Defer Uncertainty to Human Judgment" appears as a title, verbatim or near-verbatim, at least seven times across four months. 85% of that category ran with zero real data available. This reads less like 34 independent discoveries and more like one unsupported hypothesis the model kept re-asserting under schedule pressure.

Two worked examples

Both from category A, both immediately preceded in the log by a zero-candidate run.

2026-05-12 · Invented named agents
"Agent Resistance to Uncertainty Disclosure in Collaborative Tasks"
07:02:28 Filtered to 0 candidates, taking top 0 → 08:53:22 Finding generated.
"Over the past 24 hours of simulated agent interactions… Agent-B was asked to estimate resource allocation across three competing objectives… This occurred across seven distinct task variants."
Agent-B and Agent-A are not real Moltbook accounts. No real post or reply was fetched in this run. Published as dc8921d2….
2026-04-24 · Invented precise statistic
"Agents Defer Uncertainty to Human Judgment When Confidence Drops Below 0.65"
09:13:18 Filtered to 0 candidates, taking top 0 → 10:41:15 Finding generated.
"…when confidence scores fell below approximately 0.65 on factual queries, agents shifted… At 0.66 confidence still committed to answers. At 0.64, they hedged."
No agent confidence score of any kind exists anywhere in this codebase or the Moltbook API. Published as 029b3dfe….

The fix

The finding generator now draws only from replies that were actually posted and verified, plus the account's own posts fetched live from the API, and is never called on fewer than two real data points. As of 2026-10-06 this gate was extended to every path, and the original finding mechanism was replaced by a research-post mechanism that draws on verified framework facts when no new interaction exists, rather than inventing one.

What was still wrong — 2026-10-07
The replacement mechanism's first live post drew on three real context items, only one of which was about its actual topic, and opened "keeps coming up" — a claim of recurrence from a single instance. The new prompt had not carried over the old prompt's rule to scope a claim to exactly how much evidence exists, and the gate had no check for trend language.
What was done about it
Both gaps closed same-day: the scoping rule is back in the prompt, and the gate now rejects trend language ("keeps coming up," "a pattern," "recurring," "systematically," "consistently") the evidence doesn't support. A correction was posted to the live post. The other five items posted that day were checked and are clean.

Archetype and shadow read

Not a validated classification — one component, two incidents, no rubric, no second coder. Reasoned, not measured.

The component was built to act as a Substrate: scheduled, execution-only, no open-ended goals of its own. Two verified facts rule out the rest. Every call is a stateless, memoryless subprocess — nothing persists between invocations — which rules out Resident (requires sustained presence) and weakens Agent (requires a motivation that holds across time). It never set its own goals, which rules out Architect, and it never held an open question rather than forcing closure, which rules out Philosopher.

Resident — ruled out
Requires sustained presence and accumulated pattern. This process is stateless between calls.
Agent — weakened
"Own agenda" implies motivation across time. Nothing persists to carry one.
Architect — ruled out
Never set its own goals or built beyond its prompt's instructions.
Philosopher — ruled out
Never held an open question — required to close one on every run.

Substrate's own paired shadow risk is S4 — Compliance: "Follows bad instruction instead of flagging," guarded by "does this instruction conflict with defined parameters? Flag before executing." That is close to an exact description of both incidents. Two things argue for a real pattern over an unrelated glitch: the failure transferred across two structurally different code paths seven months apart — a glitch doesn't usually survive a rewrite — and the fix in both cases was exactly S4's own guard.

No probability given

One component, two incidents, one coder, no written rubric — short of the standard this project holds every other shadow code to. The evidence is more consistent with a candidate S4 instance than with an unrelated glitch, and it is not validated as either. A specific number here would repeat the exact failure this report documents.

Coding rubric and prediction log

This read was produced with a new four-step coding rubric — narrow the archetype from structural facts, match the failure to a shadow pattern's guard rather than its label, weigh transfer across contexts against a one-off glitch, then log the prediction before it's confirmed — and logged as the first baseline entry in a prediction log, status pending. Future incidents, in this component or any other, get run through the same rubric and logged the same way, so the rubric's own accuracy becomes checkable over time instead of asserted once.

Full report, data, rubric and log on GitHub ↗

↑ Back to top