The differentiator
A refuted claim is removed the day it falls.
And it stays listed here. Every number on this site comes from a campaign with a stated N, negative controls, and deterministic re-runs. Where a mechanism did not survive its own kill-gate, we removed it from the product and kept the record — because an honest negative is information you would otherwise pay to rediscover.
The scale behind every bar
Six rungs, no shortcuts
Each feature carries one bar on this fixed scale. A rung is claimed only when its evidence exists; the bars on the features page link to that evidence.
- 1Coherent idea
the mechanism is stated precisely enough to be wrong. - 2Design with pre-registered kill-gates
the conditions under which we abandon it are written before building. - 3Mechanics proven
a harness runs it end-to-end, with negative controls and a deterministic re-run. - 4Measured at scale
campaigns with stated N and confidence intervals. - 5Product-integrated
in the library, on a surface, under tests. - 6Field-adopted
external replications by people who are not us.
Nothing is at rung 6. This project is pre-launch: no external replication exists yet. The bars show it — a full bar would be a lie today.
Kept on the page
The refuted-claims list
Each of these was designed, built, and measured — and the measurement said no. Each was removed from the product; none will quietly reappear without new evidence.
- retired
The C9 head-to-head benchmark — "naive 13/24 vs mechanical 24/24" (F5).
The arms were unequal: the naive call judged unlabeled pools while the mechanical
count read a declared frame. Re-run with labels, the arms converge (direct 23/24,
native think 24/24) — and a three-line counter with no model and no graph also scores 24/24
on the declared frame, so the number measured the frame, not the mind. The claim (and the
certified-perimeter "12/24 → 24/24" figure built on the same frames) is withdrawn. What
survived every control and still ships: 0 fabrication, honest refusals
without a declared frame (0 false verdicts), and the reframe the product now states
plainly — the graph guarantees the arguments; weighing them is the model's job,
via the judgment brief +
judgePrompt. - refuted Runtime "trusted answers" cross-agreement tier (F1). The idea: let two runtime answers certify each other and skip escalation. Tested; the agreement did not carry the trust it claimed. Removed. The guarantee that remains is the honest one: certification happens at stock admission, not at execution.
- refuted Graded / prevalence weighting under the precision cap (F5). Weighting arguments by graded strength or prevalence did not survive the low-quant judge's precision cap. The verdict logic uses counts against the measured margin bound instead.
- refuted Goal-criteria weighting for a low-quant judge (F5) — 2 forms. Asking the small judge to weigh arguments against stated goal criteria failed in both tested forms.
- refuted Low-quant self-audit (F5) — 3 forms. The small model auditing its own critique did not survive any of the three tested forms. That is exactly why the critical mind is external and witness-gated rather than self-assessed.
Where the verdict stops
The measured decidability bound
The margin is a stop signal, not a proof: the mechanical layer stops below the measured count margin and hands back the structure instead of a guess.
| Above the bound | Below the bound |
|---|---|
the judgment brief — theses with their verbatim witnesses, attacks
and standing, open points, structural facts — plus a self-contained
judgePrompt your model runs to render the decision.
The graph guarantees the arguments; the weighing is the model's job. |
counts + coverage + an honest UNDECIDED — never a fake weighing. The payload names the OPEN points so the host knows exactly which data would move the margin. |
An earlier version of this page presented the bound (and a tighter certified-perimeter one) with head-to-head accuracy figures; that benchmark was retired as confounded and the figures are withdrawn. The bound's job was never to prove the verdict right — it is where the tool declines to guess.
UNDECIDED is not a failure mode. It is the tool's typed way of saying "this margin does not support a verdict — here is what would". The iteration contract turns it into the next round's input.
The practice
How the numbers are made
- Stated N, always. SQL N=201 · finance tables N=120 · zoom campaigns N=200 per domain (560 tasks across the N=40/80/200 campaigns) with bootstrap confidence intervals · critique 48 arguments, 24 decisions.
- Negative controls. Anchored generation reports 0 fabrication because the negative controls are part of the protocol, not an afterthought (anti-injection ledger: 8/8 retracted).
- Deterministic re-runs. The engine ships an integrated demo that replays
bit-identically with no model and no GPU:
node examples/integrated-demo/run.js --replay— 7 checks, in the engine repo. That replay is the public verifiable. - Auditable stock. Every forged stock carries a sha256 validation dossier; every room load goes through the engine gates. This appliance's own suite (26/26, GPU-free) includes the fail-closed no-egress guard on real sockets, with a negative control proving the guard has teeth.
- Dataset domains, named neutrally: SQL, finance tables, math word problems, argumentation corpora.
Read this before adopting
What is NOT claimed
- No claim that runtime steering proves correctness — the guarantee is at stock admission.
- No claim of coverage beyond the typed, recurrent slice of your traffic; coverage depends on your stocks, forge yield is per-domain, amortization is a property of the domain's stereotypy.
- No claim that the piece-by-piece zoom (F2) is available on this appliance — it is library-only today (the known gap).
- No claim of a verdict below the decidability bound — UNDECIDED is the honest output.
- No claim of real streaming (it is simulated) or per-tier timeouts (none yet).
- No claim of field adoption — rung 6/6 is empty until people who are not us replicate the results.