# E009-E010: One-stage build: algebraic docs missed the preset quality margin Condition: Generation · fresh codebase · 1 imperative stage · Result: not supported · 2026-09-23 The agent built a small application in a single stage. With gpt-6-sol, Algebraic docs scored below our control (No docs - code only); with gpt-6-luna it scored above! However, neither difference reached the threshold set before the runs, so the prediction is not supported. Notably, luna on xhigh showed improvements on every measure on its run. ## Results at a glance | Run | Arm | Quality score | Stages compared | Tokens | Time | Python CC | |---|---|---:|---:|---:|---:|---:| | E009 · gpt-6-sol · medium | No docs | **0.590** | 1 of 1 | **1.20M** | 0:07:40 | 6.25 | | | Prose docs | 0.580 (−0.010) | | 1.26M (+5%) | **0:07:37 (−1%)** | **5.92** | | | **Algebraic docs** | 0.520 (−0.070) | | 1.59M (+32%) | 0:07:51 (+2%) | 7.21 | | Run | Arm | Quality score | Stages compared | Tokens | Time | Python CC | |---|---|---:|---:|---:|---:|---:| | E010 · gpt-6-luna · xhigh | No docs | 0.520 | 1 of 1 | 9.28M | 0:30:42 | 5.65 | | | Prose docs | 0.580 (+0.060) | | 12.04M (+30%) | 0:40:09 (+31%) | 5.48 | | | **Algebraic docs** | **0.590 (+0.070)** | | **8.63M (−7%)** | **0:30:18 (−1%)** | **5.00** | Each score is from the run's one scored stage. Tokens and time are each run's builder totals over every stage it worked. Brackets compare with No docs. Highlighted values mark the highest quality score and lowest recorded tokens, time, and Python CC within each run; ties share the highlight. This does not mean the success rule passed. Python CC means the recorded mean cyclomatic complexity of the Python code's functions, classes, and methods at the final stage. It includes vendored code where present. ## What changed in this condition The agent builds the application from an empty repository, from one imperative prompt: it says what to build and how. Each run uses a different builder model, to test the result at different model sizes. - **No docs** (control): No design method and no starting design. Before coding, it writes its own method, then its designs. - **Prose docs**: A design method in prose. No starting charters. - **Algebraic docs** (hypothesis arm): A design method in algebra (capability contracts, Hoare triples, laws and proofs). No starting charters. **Runs:** 2 (E009, E010), one per builder model. - **E009:** gpt-6-sol · medium - **E010:** gpt-6-luna · xhigh **Agent harness:** `codex-cli`. **Judge:** `claude-fable-5-1-high`. ## Stage evidence **Not supported.** Algebraic docs needed a quality gain of at least +0.1 over No docs in each run. **E009 · gpt-6-sol · medium** | Arm | Quality score | Quality Δ vs No docs | |---|---:|---:| | No docs | 0.590 | — | | Prose docs | 0.580 | −0.010 | | **Algebraic docs** | 0.520 | −0.070 | **E010 · gpt-6-luna · xhigh** | Arm | Quality score | Quality Δ vs No docs | |---|---:|---:| | No docs | 0.520 | — | | Prose docs | 0.580 | +0.060 | | **Algebraic docs** | 0.590 | +0.070 | Quality Δ is arm score minus No docs score in the same run. A positive value favors the named arm; a negative value favors No docs. **Noise.** No two runs here repeat the same setup. In this group's repeated runs, the same arm's score at the same stage moved by a median of 0.03 and at most 0.11. ## Details ### What was measured The judge scored each arm's code after the build stage on a 0–1 quality scale. For the verdict, quality Δ is Algebraic docs score minus No docs score in the same run. The target is +0.1 or greater in every scored stage of both runs. Totals shows measured scores, tokens, and time; only the Algebraic docs versus No docs quality difference decides this result. ### Runs #### E009 — gpt-6-sol · medium **Run ID:** `run-20260923T172943Z-c25ba500` **Started:** 2026-09-23 **Harness status:** completed Every arm finished every stage. #### E010 — gpt-6-luna · xhigh **Run ID:** `run-20260923T174224Z-7bf8e1fb` **Started:** 2026-09-23 **Harness status:** completed Every arm finished every stage. ### Limits - Each builder model ran once, so neither model's own run-to-run noise is known. - The judge here was Claude Fable 5.1. Later conditions used Claude Opus 5.5, so their scores do not compare with these. ### Source data [View condition `E009-E010` in G001.json](https://mikereese.dev/data/experiments/G001.json). The run IDs above identify the counted runs. See [Data and methods](https://mikereese.dev/experiments/g001/#data-and-methods) for the export method and limits.