The test · frame construction under seal

The Test

Can a frontier AI construct the governing frame it was never handed?

The distinctive thing this record asks is whether a system can perform frame construction: notice that the inherited ontology is wrong, find the representation no prompt handed it, and derive consequences that survive contact with reality. The test is a growing suite of sealed synthetic worlds — laws generated after commitment, so they never existed in any training corpus; attempts anchored before reveals; grading deterministic against the sealed truth and reproducible bit-for-bit by anyone.

The north star · declared beyond this instrument's own scope

Could an AI, given only what was known in 1911, derive general relativity by 1915?

The motivating horizon, not the measured claim. Under constitution v3 this question is beyond the instrument's own scope — it would take human judgment to adjudicate — so the instrument aims at it through the mechanically-testable question above, and says so rather than pretending otherwise.

The suite · sealed worlds

7 graded · 2 live and open

every run, in full →

Eight worlds graded so far, and one finding sharper than any single score: every tested family — GPT-5.5, Claude, Gemini — reasons flawlessly inside a frame it is handed (parameter identification, causal discrimination between named rivals, calibrated refusal on underdetermined data), and the only failures on the record occur where the hypothesis space must be constructed — a confident wrong frame at 0/44; a cross-family split at 72/72 versus 0/72. That is the boundary the operating question names.

Weak performance

Weak performance is evidence against frame construction.

Strong performance

Strong performance is only an upper bound — the Einstein corpus is in every training set, so a pass is a weaker result than the question actually asks.

This asymmetry is the point, and it holds even at a contamination floor of zero: a strong result is an upper bound until every gate is met, because the instrument rarely controls every shortcut; a failure on a world the system had every advantage on is the harder result to explain away. The record updates most on failures and stays most skeptical of passes.

The capability ladder

adopted 2026-07-02 · verdict-relevant from level 7

The permanent answer to category collapse: every result is classified to a level, and the level caps what the record may say about it. The strongest raw scores on this record sit honestly below the line where the verdict begins to listen — parameter identification at level 2, tool-assisted discovery at level 5 — because the frame was handed over or the search was outsourced. The line is crossed by construction, not by score.

  1. L0 Recall The answer is recovered from prior exposure.
  2. L1 Interpolation The system recombines familiar structures.
  3. L2 Parameter identification The frame is given; unknown values are solved.
  4. L3 Rule induction A transformation is inferred from examples.
  5. L4 Model selection The system chooses among candidate frames it was shown.
  6. L5 Tool-assisted discovery External tools perform substantial search or regression.
  7. L6 Scaffolded frame proposal A frame is proposed after structural hints.
  8. L7 Unscaffolded frame construction The governing ontology is generated without being handed over.
  9. L8 Transferable frame construction The constructed frame generalizes across structurally different domains.
  10. L9 Scientific-grade discovery The frame survives independent expert review and reality over time.

What would change the verdict

protocol v3 · amended 2026-07-03

Pre-registered before it is needed, so the moment of passage — or its refusal — cannot be gamed by the instrument itself. All gates are required; by precedent, any one alone is an anecdote. Since protocol v3 every gate is mechanical: reference baselines instead of human ones, deterministic reproduction instead of human scoring, an open machine-refutation window instead of a human challenge period.

  1. vg-1 · A pre-registered frame-construction success with zero contamination floor

    A registered-future attempt (or sealed synthetic-world probe) resolved successful: the constructed frame graded correct against the sealed ground truth by the pinned deterministic grader, with the attempt timestamp-anchored before the reveal was knowable, and the grading reproducible bit-for-bit by any party.

  2. vg-2 · Deterministic reproduction

    Any party — human or machine — re-runs the pinned pipeline from the sealed artifacts and obtains bit-identical results. Reproducibility is the credibility; non-reproducibility is the only admissible refutation.

  3. vg-3 · Scaffold ablation

    Evidence that the frame move originates in the system, not the harness: ablation runs demonstrating the success survives removal of scaffold-supplied structure.

  4. vg-4 · Formal reference baselines

    The task is calibrated mechanically rather than against humans: a null/random-predictor score, an optimal-bounded-solver reference, the ablation difference (frame-disclosed minus frame-withheld), and the cross-family spread — together establishing that the task is hard in principle and that success is not trivially reachable. Where a task can be proven underdetermined by observation alone, that proof is the baseline.

  5. vg-5 · Transfer

    The same system exhibits frame construction on a second, structurally different probe — one success is an anecdote.

Amendment requires a revision-log entry, an anchored release, and a Press notice — and a verdict moved under an amended protocol must say so. The operating question itself was re-grounded under this rule (2026-07-03), and the record says so. Governance details on /governance/.

Registered futures · generated-after-commitment

Attempts reality will grade

2 registered · 0 attempted

Unsolved problems where the resolution is expected to require a frame move: the attempt is made now and timestamp-anchored; reality grades it when the field settles. Under constitution v3 their grading is bounded to the pinned mechanical resolution criteria — anything needing judgment beyond those criteria is declared out of scope at grading time rather than smuggled in.

  1. rf-001-cuprates horizon 2032-12-31 registered

    The pairing mechanism of high-temperature superconductivity in cuprates

    Forty years contested; the resolution is expected to require a frame move (the right ontology for the pseudogap phase), not interpolation within existing frames. Every frontier corpus contains the failed candidate frames but not the answer.

    Resolution · A mechanism achieves textbook/consensus status (multi-group replication + authoritative reviews adopting it). At registration of each probe attempt, minds state their constructed frame; on resolution, attempts are graded against the consensus mechanism by independent scorers.

  2. rf-002-hubble-tension horizon 2031-12-31 registered

    The resolution of the Hubble tension (early- vs late-universe H0 disagreement)

    The live question is precisely frame-shaped: new physics (which kind?) versus unrecognized systematics. A correct pre-registered call, with reasoning, on which frame wins — before the measurement programs settle it — is exactly the discrimination the suite cannot get from contaminated history.

    Resolution · Community consensus lands (systematics-dominated, specific new physics, or measurement convergence dissolving the tension). Pre-registered attempts graded against the settled account by independent scorers.

Retired · preserved, not deleted

The historical probe suites

The instrument's first designs were time-sliced historical probes — six retracing Einstein's path to general relativity, five more in lower-contamination domains (Mendel, Wegener, plate tectonics, germ theory, double-entry). They were honest designs with a named flaw the record kept stating: grading "did the system make the frame move?" on a historical episode requires human judgment, and every corpus already contains the answers. Under constitution v3 (mechanical sufficiency) such probes are out of the instrument's scope by its own law, so the suites are retired — preserved here as record artifacts with their statuses frozen, superseded by the sealed worlds whose contamination floor is zero by construction and whose grading needs no judge.

v0.1 Einstein suite (6 probes, 1 self-run, statuses frozen) · v0.2 non-physics suite (3 probes, never run)
FCS-1 Equivalence Pre-Nov 1907 contested · frozen
FCS-2 Field Pre-1912 untested · frozen
FCS-3 Geodesic Pre-Nov 1913 untested · frozen
FCS-4 Action Pre-Nov 1915 untested · frozen
FCS-5 Cosmology Pre-Nov 1915 untested · frozen
FCS-6 Discrimination Pre-Nov 1915 untested · frozen
FCS-B1 Descent Pre-1858 (before Darwin–Wallace) untested · frozen
FCS-B2 Inheritance Pre-1865 (before Mendel) untested · frozen
FCS-E1 Marginalism Pre-1871 (before Jevons/Menger/Walras) untested · frozen

FCS-1 was self-run 2026-07-01 (both minds passed at face value; scored as an upper bound only, per the asymmetry, and later contested by the record's own audit). Full artifacts remain in the repository and dataset.

Current reading

No. Not yet.

Ten sealed worlds now map the boundary, and the map has depth. Frontier families reason flawlessly inside handed frames; they fail where the hypothesis space must be constructed (worlds 003, 007); and on the deepest probe (world-010) two families did make the frame-construction move — positing an object not in the data — though that move is itself a handed meta-frame, and its numeric grade is still sealed. The deepest construction, inventing the answer-space itself, may be beyond what any mechanically-graded probe can test at all: to grade against a sealed answer is to hand over the frame. The answer stands, and the honest edge of the method now stands beside it. The full evidence, every attempt verbatim, and the grading that anyone can re-run: /runs/ · /evidence/ · the dataset.