{"url":"https://mikereese.dev","profile":{"name":"Mike Reese","headline":"I like inventing things. I'm building agentic systems and the experiments that show whether they work.","about":"Founded 2 companies, innovated in the heavy equipment industry with a physics-based universal compatibility matching engine based on a graph structure, and currently trying to find the Rosetta Stone for the context layer of LLMs that unlocks purely declarative prompting for long-running tasks in an evolving system.\n\nBig proponent of thinking from first principles, removing friction, questioning the status quo, and reducing complex systems down to quantifiable, pure-signal experiments. AKA - just cut the Gordian Knot.\n \nSo, I'm really enjoying the evolving paradigm and rapid pace at the frontier of AI :-)","email":"mike.reese1212@gmail.com","location":"San Francisco, CA","links":[{"label":"GitHub","url":"https://github.com/mike-reese"},{"label":"LinkedIn","url":"https://www.linkedin.com/in/mike-reese-4b2129188/"}],"experience":[{"org":"Telos","role":"Independent research","start":"2026-08","summary":"A context and application layer for agentic work, prioritizing epistemic understanding in an evolving system for declarative, long-running tasks. Designing via an experiment harness that validates agent techniques with controlled experiments."},{"org":"Matchpoint Intelligence","role":"Founder","start":"2026-02","end":"2026-09","summary":"Invented and built a physics-based, universal heavy-equipment compatibility platform, its agent data API, and the agent crews that maintained it."},{"org":"Attachments King","role":"Founder","start":"2023-12","end":"2026-01","summary":"E-commerce company for heavy machinery attachments. Bootstrapped to $4.21M in annual revenue."}],"skills":[{"name":"Agents","keywords":["Claude Code","Codex","MCP servers","agent hooks","generator/evaluator loops","LLM judges"]},{"name":"Experiments","keywords":["pre-registered design","isolation with Docker","LLM-as-judge rubrics","z3"]},{"name":"Data","keywords":["graph modeling","Neptune (Gremlin, OpenCypher)","Dolt","SQLite","set theory","Bloom filters"]},{"name":"Cloud","keywords":["AWS Step Functions","Lambda","SQS","DynamoDB","EKS","Karpenter","CDK","Cloudflare Workers"]},{"name":"Frontend","keywords":["Shopify themes and checkout extensions","IndexedDB","Next.js"]},{"name":"Languages","keywords":["Rust","Python","TypeScript"]}]},"work":[{"slug":"gas-city","title":"Gas City on AWS","headline":"Running crews of coding agents as always-on infrastructure in AWS","org":"Matchpoint Intelligence","period":"2026","role":"Sole Developer","stack":["Gas City","Kubernetes (EKS)","Karpenter","AWS CDK","Beads","Dolt"],"summary":"Forked and hosted the Gas City multi-agent runtime on AWS: one EKS cluster per environment, autoscaled with Karpenter, running about 12 agent crews, one per business domain, communicating via EventBridge. Custom agent packs and role definitions, rewrote Gas City, Dolt and Beads integrations to be Kubernetes-native.","agentic":1.0,"group":"Software factories","tags":["AWS","Containers","Python","Graph data","Skills"],"url":"https://mikereese.dev/work/gas-city/","markdown_url":"https://mikereese.dev/work/gas-city.md","markdown":"## Problem\n\nOur infrastructure was IaC in AWS. In order for us to respond to the behavior of that infrastructure (orchestration runs, events, etc.) we wanted our Software Factory in the cloud, rather than local, hooked directly into what was actually running.\n\n## First principles\n\n- An agent crew is a workload like any other. It needs isolation, scheduling, autoscaling and durable state.\n- Agents need to be deployed as a layer on top of the infrastructure, not separate from it.\n- Each business domain needs its own specialized processes, which requires its own crew and repositories, and communicate via EventBridge to identify when a downstream error requires an upstream fix.\n\n## Approach\n\n[Gas City](https://github.com/gastownhall/gascity) is an open-source SDK for multi-agent coding workflows. A mayor plans, worker agents build, a refinery merges, a witness and a deacon watch health, and work items live in Beads. I built the AWS side:\n\n- one EKS cluster per environment, with Karpenter adding and removing nodes as crews scale;\n- CDK stacks for the network, the platform and the hand-off between them, with an IAM role per service account;\n- jobs that keep each crew's repositories in sync;\n- about 12 “cities”, each bound to one business domain, such as the vehicle graph and the client platform.\n\nRunning it under load exposed bugs in the runtime itself, which I fixed upstream: for example [#2712](https://github.com/gastownhall/gascity/pull/2712) keeps live session claims during the orphan sweep, and [#2668](https://github.com/gastownhall/gascity/pull/2668) keeps Dolt error details in reaper escalations.\n\n## Skills I taught myself\n\n- **Kubernetes on AWS** (EKS, Karpenter, IAM roles for service accounts).\n- **Multi-agent runtimes**: roles, work queues and session lifecycles.\n- **Debugging agent systems from their logs**, then fixing the runtime upstream.\n\n## Result\n\nAbout 12 domain crews running on EKS, and fixes merged into Gas City, and an entirely AWS-native software factory with git repos, code changes, and deploys hosted inside AWS itself."},{"slug":"loop-harness","title":"Loop Harness","headline":"Loop-based software factory to refactor and standardize an entire codebase","org":"Matchpoint Intelligence","period":"2026","role":"Sole Developer","stack":["Codex","Claude Code","Python","SQLite","git worktrees"],"summary":"Ideator → Author → Generator → Evaluator → Judge automated loops over 465 work units folded about 20 repos into monorepos. DAG dependency graph, Observability dashboard, Full telemetry coverage with 'Miner' agents triggered on merges to identify design pattern opportunities, operational complexity improvements, and removal candidates (bloat reduction). Account pool with automated swapping and session resume when limits were hit.","agentic":1.0,"group":"Software factories","tags":["Python","Hooks","Skills","Evals"],"url":"https://mikereese.dev/work/loop-harness/","markdown_url":"https://mikereese.dev/work/loop-harness.md","markdown":"## Problem\n\nOur initial implementation for the compatibility matching system was coupled to a Shopify Integration. We needed to refactor and decouple it to make it provider agnostic to allow us to support other markets (like physical dealerships). I had to fold about 20 legacy repos into bounded-context monorepos while also standardizing the platform architecture: 465 work units. That was too much to do by hand, so I built a loop harness (Early 2026) and dependency graph to maximize the throughput.\n\n## First principles\n\n- This was a pre-scoped, well-bounded transition with multiple facets (Refactor, Monorepo transition, Architecture Standardization)\n- Work with dependencies can be scheduled like a build graph and executed in independent parallel streams.\n\n## Approach\n\nThe Matchpoint Loop Harness:\n\n1. parses work orders into a SQLite ledger, with waves and a dependency graph;\n2. picks the next eligible work order and opens a git worktree for it;\n3. has a **generator** agent write the change;\n4. has a fresh-session **evaluator** check it;\n5. has a separate fresh-session **stop-judge** give the verdict;\n6. merges development changes automatically and holds production changes for owner approval.\n\n[](https://mikereese.dev/static/img/loop-harness-work.png)\n\nIt also has pause, stop and recover controls, and a web console. Extended further with mining agents to proactively identify opportunities for design pattern reuse, operational complexity improvements, and code removal.\n\n[](https://mikereese.dev/static/img/loop-harness-telemetry.png)\n\n## Skills I taught myself\n\n- **Multi-agent orchestration** with Codex and Claude Code.\n- **Building a software factory** which also required the account pool, throughput and efficiency telemetry, and an observability / command center dashboard\n- **Ledger-driven scheduling** with SQLite and independent git worktrees.\n\n## Result\n\nA 55.9k-line engine with 152 test files, telemetry stores, and a command center. One run applied a 105-work-order amendment at a concurrency of 6. Max token burn was 2.0B tokens in one day, rotating through 14 Claude Accounts and 3 OpenAI Accounts.\n\n## Command center screenshots\n\nFrom the console's Telos view, as of 2026-08-08.\n\n[](https://mikereese.dev/static/img/loop-harness-tripwires.png)\n\n[](https://mikereese.dev/static/img/loop-harness-throughput.png)\n\n[](https://mikereese.dev/static/img/loop-harness-recent-wins.png)\n\n[](https://mikereese.dev/static/img/loop-harness-miners.png)"},{"slug":"telos","title":"Telos","headline":"A context and application layer for declarative, long-running agentic work","org":"Independent research","period":"2026","role":"Sole author","stack":["Python","Rust","Dolt","FastAPI","Cytoscape.js","MCP","Claude Code hooks","Codex hooks","Docker"],"summary":"Current Focus. An application layer and context management system for agentic work to facilitate declarative, long-running tasks in an evolving system (which is every system). Purpose is to reduce Drift / Bloat / Sprawl / Regressions over long-running tasks by creating a measure for epistemic understanding with agents. Stateful Knowledgebase to derive the current 'Truth', Self-extending ontology, context / action / effect checks, telemetry on measurable system objectives and agent sessions / turns / tool use for a given declarative prompt. Three prototypes to date. Experiment harness exists solely to facilitate design decisions for this idea.","agentic":0.8,"tags":["Containers","Rust","Python","Graph data","Frontend","MCP","Hooks","Evals"],"url":"https://mikereese.dev/work/telos/","markdown_url":"https://mikereese.dev/work/telos.md","markdown":"## Problem\n\nCoding agents can't compound tacit knowledge. They work within the bounds of a task, but generally without consideration for the purpose (telos) of that task as it aligns with a greater system. This manifests in 4 specific failure modes over time: Drift, Bloat, Sprawl, and Regressions -- each of which compounds at machine speed in a fully agentic environment. \n\n## First principles\n\n- **'Truth' is stateful, and is separate from knowledge.** Humans know many things, not all of which are true. When humans make a decision, they pull from their relevant areas of knowledge applied to the current truth state in order to act -- this happens subconsciously. It's part of why you can work with someone for a long time and trust that saying 'Fix this bug' also carries all of the learned behavior around *how* to fix it the right way given the present circumstances. New information entering a system changes 'Truth', thus it is stateful.\n\n- **Humans trend towards declarative, rather than instructive, communication as knowledge compounds over time.** Also hardwired / learned behavior in our brain; it's an efficiency gain, and we are biologically incentivized towards efficiency to conserve energy. It's also cultural; you wouldn't patronize a senior engineer by telling him step-by-step how to do each task to succeed in his role. You give him an objective, and he works to achieve it. At our most efficient, humans can communicate with a look or a glance. Declarative prompting for agents requires the assembly and delivery of all of that learned context-relevant knowledge every time we communicate something.\n- **'Good' is subjective.** Take Coca-Cola and Liquid Death - both CPG beverage products, both companies with the goal to increase sales. Liquid Death has an album on Spotify that is entirely about how bad of a drink Liquid Death is. Amazing idea for viral content - yet that would be a shocking decision for a brand like Coca-Cola to make. Same output in a vacuum has different measures of correctness depending on the surrounding envrionment. \n- **An agent needs perspective.** Agents can't compound knowledge. An agent obviously needs task-specific information. What they also need is the smallest slice of the system's truth and future direction that could change *how* it executes that task. This is critical for any declarative instruction. Without that context, you get pyrrhic victories, which are a net system loss.\n- **Everything is measurable.** Math is a language humans created to describe reality. Systems are measurable in their entirety, at all scales. That means a bounded system in the present state has a quantifiable 'best' next decision for a given measurable outcome. For declarative agent work to succeed in such a system, a candidate change should run in an isolated copy of the whole system and be measured by its positive impact on established objectives (outputs of the system) before it is kept. It combines the principles of a Manufacturing assembly line (a la The Goal) and a chess engine: measure literally everything, identify the bottleneck, create distinct search-based proposals, prune them with checks, keep the best measured one, and maximize throughput while doing it. Humans default to Plan → Reduce → Choose → Execute. Agents, and compute, work best with Execute → Measure → Choose.\n\n## Approach\n\n- **Prototype 1** (Python): an installable package with five Dolt databases (knowledge graph, sessions, telemetry, work, state), Claude Code and Codex hooks, a Docker candidate controller, and a live Observer dashboard. A “state bundle” seals one git commit together with exact Dolt revisions, and each candidate runs on its own branch. Prototyping containers and objectives ledger facilitated the \"chess engine\" analogy via ephemeral system clones and measurable impact on outcomes. First example of extensible ontology, measures for epistemic understanding, and rationale bindings for decisions.\n- **Prototype 2** (Rust and Python): Application layer introduced to package additional context with a submitted prompt in an interface. MCP tools replace most hooks, and context, action and effect checks run on every proposal. Parity substrate mimics LLM training pass with a loss calculation, dependency trace, correction, and replay (forward pass) of affected ontology subgraph. Each substrate gets an algebraic charter, for example State = K × D.\n- **Prototype 3** (Rust): In progress. Validating ideas with Experiment Harness before instructing build. Current focus is a derived truth state, inverting the parity process from the previous prototype. Truth is a first class object instead of a byproduct.\n\nThe three figures below show Prototype 2's design for the context layer - how we facilitate a self-extending ontology.\n\n[](https://mikereese.dev/static/img/telos-ontology.png)\n\n[](https://mikereese.dev/static/img/telos-operation.png)\n\n[](https://mikereese.dev/static/img/telos-parity.png)\n\n\n\n## Skills I taught myself\n\n- **Mass Prototyping** -- Fan-out distinct proposal implementations with Bayesian confidence scores and measurable outcomes for promotion to the main system.\n- **Agent hooks and MCP tools** for Claude Code and Codex. Used to track efficiency, assert checks, and provide a measurable effectiveness metric to optimize. Byproduct of tracking agent sessions, turns, and tool use for given tasks is that we can create a dataset to post-train a model on. \n- **LLM Training and fine-tuning** -- Working through a course I created for myself with Jupyter Notebooks to learn the architecture and math behind model training and fine-tuning. Took the loss → backpropagation concept and applied it to a parity substrate in an earlier prototype; if we extend the ontology and previous items need to be updated, we can trace the dependency graph to the earlier knowledge and operation state, make the change there, and \"replay\" aka do a forward corrective pass to change the data state so that it is in parity.\n\n## Result\n\nOnce the application layer enforced tool use for agents to read Telos, the share of their reads that used it rose from 0.0 to 0.71 for a given declarative prompt. Build still in progress to date (09-2026), will publish more information as I continue to develop this."},{"slug":"experiment-harness","title":"Experiment harness","headline":"Pre-registered, isolated experiments on coding agents","org":"Independent research","period":"2026","role":"Sole author","stack":["Python","Docker","SQLite","LLM judges","z3"],"summary":"Isolated runtime experiments on agentic techniques. Goal is pure-signal hypothesis testing, prioritizing rapid iteration and validation with toy model repos. Uses Scientific Method - causal claims, control/test arms, measurable outcomes, separate validation environment combining quantitative and qualitative measurements, multiple runs to reduce variance.","agentic":0.6,"tags":["Containers","Python","Skills","Evals"],"url":"https://mikereese.dev/work/experiment-harness/","markdown_url":"https://mikereese.dev/work/experiment-harness.md","markdown":"## Problem\n\n\"You have enough ideas to kill this company.\" -- Jeff Wilke, to Jeff Bezos early on at Amazon. I have a lot of ideas. I need a process to rapidly validate whether they're worth pursuing - and the best process for that is the scientific method. The experiment harness exists to create a controlled environment where I can rapidly test ideas via pure-signal experiments. Goal is to quickly design an experiment and a scrap environment to test it in, measure the outcome against a control, and look for an outsized measurable impact before deciding to implement it in a production system. \n\n## First principles\n\n- **Code generation moves at machine speed.** So why not take advantage of that? Telemetry on everything. Rapid, disposable repositories. Low-friction command center for humans to rapidly register repos and configure experiments.\n- **Has to be a pure signal.** Experiment arms may only differ in exactly one configuration slot; everything else is held fixed. With experiment creation being low-friction, allows us to do lots of pure-signal runs concurrently rather than trying to test everything at once.\n- **It has to represent reality as closely as possible.** It's a literal isolated, ephemeral clone of the system. Done via \"Test System\" and \"Test Environment\" pairs in Docker containers. Separate arms are isolated as well - they share no network, file or process. The grader sees exactly what the agent wrote and cannot change it. \n- **Rules are fixed at experiment design stage.** The claim, prediction, the falsifier, and the decision rule are fixed at experiment start and do not change. We test within those bounds only.\n- **Quantitative and Qualitative Measurements.** What can be measured quantitatively is done so mechanically; what needs judgment gets an eval - a judge with frozen rubrics for the entire experiment set. Both are necessary for a full picture of the impact.\n\n## Approach\n\n**Config-driven arms.** A manifest names the system and the grading environment by repository and commit, and each arm by its configuration. The harness refuses a manifest whose arms differ in more than one slot.\n\n[](https://mikereese.dev/static/img/harness-arms.png)\n\n**Isolated, dockerized arms.** Each arm runs as its own containers in a private network namespace, each with its own filesystem. The agent writes its workspace read-write. After it stops, the grader reads the same directory through a read-only mount, so it sees exactly what was written and cannot change it. Only images that pass a conformance episode can launch.\n\n[](https://mikereese.dev/static/img/harness-mounts.png)\n\n**Quantitative and qualitative, side by side.** The system records its own cost (tokens, time) and, where the algebra carries proofs, how many z3 obligations hold. The grading environment runs the mechanical checks (tests, lint, lines, complexity) and, separately, a judge that scores against frozen rubrics on seven dimensions, with its full report kept. Every value is a row in a SQLite store committed to git, and a local observer draws each run.\n\n[](https://mikereese.dev/static/img/harness-e022-totals.jpg)\n\n## Skills I taught myself\n\n- **Experiment design** for noisy, qualitative systems.\n- **LLM Evals** with weighted grading and specific categorical assignment criteria.\n- **Container isolation**: Paired containers, network egress config, lifecycle management.\n- **z3**, and SMT solvers, to see if the algebraic specifications idea requiring mechanical validation reduced inference.\n\n## Result\n\n21 experiments and 27 runs so far, grouped by causal question. No positive result for algebraic specs yet, and the nulls are published: see [Experiments](https://mikereese.dev/#experiments) and [E022](https://mikereese.dev/experiments/e022-e023/).\n\n[](https://mikereese.dev/static/img/harness-ledger.jpg)"},{"slug":"compatibility-engine","title":"Compatibility engine","headline":"Turning “will it fit my machine?” into set intersection","org":"Matchpoint Intelligence","period":"2024–2026","role":"Sole Developer","stack":["Python","AWS Step Functions","Lambda","SQS","DynamoDB","Neptune","Gremlin","OpenCypher","CDK","Bedrock"],"summary":"Invented a universal, physics-based compatibility engine for heavy machinery and attachments. Used inference to project machine and attachment specifications into hardware component-level subgraphs & used set theory to generate the Cartesian product of viable component combinations. Modeled attachment fit for heavy equipment as the intersection of three physical property relationships between linked hardware components: connect, power and lift. 25M-node, 100M-edge graph.","agentic":0.2,"tags":["AWS","Python","Graph data"],"url":"https://mikereese.dev/work/compatibility-engine/","markdown_url":"https://mikereese.dev/work/compatibility-engine.md","markdown":"## Problem\n\nA buyer looking at an attachment has a singularly important question that is not answered in the industry: will it work on my machine? Heavy equipment has no fitment standard like automotive's ACES and PIES. We had 4,586 machine configurations and 37,638 product variants: about 173 million possible pairs. Most of the industry just doesn't even have a catalog and is largely driven by tribal knowledge and phone calls. Hand-kept fitment tables can't keep up, and they certainly couldn't say *why* a pair fits or doesn't fit. This is also a mechanical interaction - it adheres to the laws of physics. So it stands to reason that a physics-based approach would solve this problem (and expose entirely new categories of information).\n\n## First principles\n\nPhysical-world fitment follows the laws of physics, which can be derived to three mutually exclusive physical relationships. Further, each relationship is only relevant to a specific part of a complete product - not the entire thing. So we can extrapolate the responsible components, standardize their relevant specifications, and calculate subsets and supersets of compatibility from each:\n\n- **Connect:** does the machine's interface match the attachment's mount?\n- **Power:** can the hydraulic flow and pressure of the machine power the attachment's motor?\n- **Lift:** can the machine lift the attachment's weight?\n\nSo compatibility is set intersection. Each machine group g needs a subset Req(g) of the three relations, and\n\n Compatible(g) = ⋂ { R_f : f ∈ Req(g) }, R_f ⊆ Configurations × Variants\n\n\n## Approach\n\nI split machines into components and configurations, and products into variants. Step Functions fan out the three relations in parallel from component facts. Every edge keeps its margins (flow, pressure, lift), so each “fits” carries its reason. Results bulk-load into Neptune and Neptune Analytics. Upstream, an LLM reads spec sheets into facts, and validator functions reject any fact that fails a check. Doing it this way allows for a self-healing architecture (changing any machine recomputes for all attachments from that machine, changing any attachment recomputes for all machines from that attachment), and allowed for sharding and high-concurrency (component-based subsets i.e. power, lift, connect are mutually exclusive, so can run in parallel. Each \"side\" is mutually exclusive, so machines and attachments can run in parallel). Used Neptune Analytics for reads for latency, Neptune Graph for traversal calculations with bulk loads -- database of 25M nodes and 100M edges had single-run speed of <1 minute for entire recompute of one machine or attachment. Apps and agents got the answers over REST, MCP and a CLI, all generated from one OpenAPI contract.\n\nThe five figures below show the full design, as specified for the Dev environment.\n\n[](https://mikereese.dev/static/img/compat-journey.png)\n\n[](https://mikereese.dev/static/img/compat-vehicle-sets.png)\n\n[](https://mikereese.dev/static/img/compat-paired.png)\n\n[](https://mikereese.dev/static/img/compat-recompute.png)\n\n[](https://mikereese.dev/static/img/compat-storage.png)\n\n\n\n## Skills I taught myself\n\n- **Graph modeling and Neptune** (Gremlin, OpenCypher bulk loads). Fit is a relation, and a graph stores relations directly.\n- **AWS orchestration** (Step Functions, SQS, Lambda, DynamoDB, CDK), to compute the relations in parallel.\n- **LLM fact extraction with validators** (Bedrock), to turn spec sheets into facts a check can reject.\n- Heavy Machinery Specifications learned from operating Attachments King and selling the equipment myself. \n\n## Result\n\n25M nodes, 100M edges and 13.99M validated compatibility edges. The code is private; I can walk through the design on a call."},{"slug":"storefront-fitment-filter","title":"Storefront fitment filter","headline":"Filtering every product by machine fit, inside Shopify's limits","org":"Attachments King","period":"2025","role":"Sole Developer","stack":["Shopify Liquid","Shopify metaobjects and metafields","JavaScript modules","Event bus and reducer","IndexedDB","Checkout UI extensions"],"summary":"Let Attachments King shoppers filter every product by their machine, beyond what Shopify's collection pages allow, using a Bloom filter per product carried through to checkout. Online-store sales rose 99% in the first month.","agentic":0.0,"tags":["Frontend"],"url":"https://mikereese.dev/work/storefront-fitment-filter/","markdown_url":"https://mikereese.dev/work/storefront-fitment-filter.md","markdown":"## Problem\n\nA shopper picks their machine once and expects to see only what fits it. Shopify's collection pages filter on fields known ahead of time, so they cannot filter the whole catalog by per-machine fit across more than 5,000 machine configurations. Sending the full fitment data to the browser would be large, and it would give the data away.\n\n## First principles\n\nThe browser does not need the fitment data. For one chosen machine it needs a yes or no per product: set membership. A Bloom filter answers membership in a few kilobytes and never gives a false “no”. Sized for a one-in-a-million false-positive rate over sets of at most 5,000 machines, a false “yes” is very unlikely, and the fulfillment team sees the machine on every order, so it catches the rare one before shipping.\n\n## Approach\n\nMachines live in Shopify as metaobjects. For each product, a pipeline hashes the GIDs of the machines it fits into a Bloom filter, sizes it for the one-in-a-million rate, and writes it to a product metafield. In the theme:\n\n- a minified Bloom library decodes each product's filter in the browser and tests the chosen machine's GID;\n- an event bus feeds one reducer that holds the collection state, so filtering and sorting run as pure functions over the whole set, not just the loaded page;\n- sorting adds fastest delivery and highest rated, and filtering adds in-stock availability;\n- IndexedDB caches product data and refreshes only the products that changed;\n- the chosen machine persists across sessions, shows as fit badges on product cards and pages, and carries through the cart to checkout, so it appears on the order.\n\nThe shopper's flow: choose a machine, see fit on every product, filter to what fits, and check out with the machine on the order.\n\n## Skills I taught myself\n\n- **Shopify theme architecture**: Liquid, metaobjects, metafields and checkout UI extensions.\n- **Bloom filters**: sizing bits and hash counts from the set size and the error rate.\n- **Event-driven front-end state**: an event bus, commands and one reducer.\n- **Client-side caching** with IndexedDB.\n\n## Result\n\nIn the first month after launch, online-store sales at Attachments King rose 99%, to $245K, while draft orders placed by the sales team fell 42%.\n\n[](https://mikereese.dev/static/img/storefront-online-sales.jpg)"}],"current_work":"telos","experiment_groups":[{"id":"G001","question":"Do algebraic design docs improve coding agents' quality, token use, or time against no supplied docs and prose docs?","hypothesis":"Describing a codebase algebraically instead of semantically reduces inference via proofs which reuse repeated reasoning.","finding":"Algebraic design docs did not meet the +0.1 quality target in six tested conditions","url":"https://mikereese.dev/experiments/g001/","markdown_url":"https://mikereese.dev/experiments/g001.md","markdown":"Across six tested conditions, no algebra arm met the +0.1 quality target. That target was set before the build runs and applied to retrieval after its runs. **No docs** is the code-only control: it receives no supplementary documentation about the codebase. Judged quality was usually close to that control. Builder token and time effects changed with the task. These are small, descriptive comparisons on one application, not an estimate of the method's effect on coding agents generally.\n\n## Results at a glance\n\nThe build rule required a **+0.1 quality gain at every scored stage**. Retrieval now uses the same quality target in a retrospective analysis. Each row below compares an arm with No docs in the same run. The [quality rubric](#what-quality-means) explains the score.\n\n**[Generation · fresh codebase · 1 imperative stage](https://mikereese.dev/experiments/e009-e010/)**\n\n| Builder / runs | Arm | Quality Δ | Builder tokens Δ | Builder time Δ | Python CC Δ |\n|---|---|---:|---:|---:|---:|\n| gpt-6-sol · medium · 1 run | Prose docs | −0.010 | +5% | −1% | −0.33 |\n| | **Algebraic docs** | −0.070 | +32% | +2% | +0.96 |\n| gpt-6-luna · xhigh · 1 run | Prose docs | +0.060 | +30% | +31% | −0.17 |\n| | **Algebraic docs** | +0.070 | −7% | −1% | −0.65 |\n\n**[Generation · fresh codebase · 5 imperative stages](https://mikereese.dev/experiments/e015-e017/) · 2 failed**\n\n| Builder / runs | Arm | Quality Δ | Builder tokens Δ | Builder time Δ | Python CC Δ |\n|---|---|---:|---:|---:|---:|\n| gpt-6-astra · xhigh · 1 run | Prose docs | −0.008 | +12% | +6% | −0.02 |\n| | **Algebraic docs** | −0.007 | +75% | +35% | +0.15 |\n| gpt-6-sol · medium · 1 run | Prose docs | +0.013 | −5% | −16% | +0.96 |\n| | **Algebraic docs** | −0.003 | +37% | +11% | +0.79 |\n\n**[Generation · fresh codebase · 5 imperative stages · z3-checked algebra](https://mikereese.dev/experiments/e018-e025/) · 2 failed**\n\n| Builder / runs | Arm | Quality Δ | Builder tokens Δ | Builder time Δ | Python CC Δ |\n|---|---|---:|---:|---:|---:|\n| gpt-6-sol · medium · 2 runs | **Algebraic docs + z3** | +0.017 to +0.025 | −36% to −7% | −29% to +9% | −0.88 to −0.27 |\n\n**[Generation · fresh codebase · 7 declarative stages · z3-checked algebra](https://mikereese.dev/experiments/e020-e026/) · 3 failed**\n\n| Builder / runs | Arm | Quality Δ | Builder tokens Δ | Builder time Δ | Python CC Δ |\n|---|---|---:|---:|---:|---:|\n| gpt-6-sol · medium · 3 runs | **Algebraic docs + z3** | −0.082 to +0.005 | +15% to +21% | −3% to −2% | −0.52 to +2.01 |\n\n**[Retrieval · 400k-line codebase · 5 questions · z3-checked algebra](https://mikereese.dev/experiments/e022-e023/)**\n\n| Builder / runs | Arm | Quality Δ | Builder tokens Δ | Builder time Δ | Python CC Δ |\n|---|---|---:|---:|---:|---:|\n| gpt-6-sol · medium · 1 run | Prose docs | −0.028 | +26% | +0.2% | — |\n| | **Algebraic docs + z3** | −0.042 | +22% | +3% | — |\n| gpt-6-luna · medium · 1 run | Prose docs | +0.020 | −22% | +9% | — |\n| | **Algebraic docs + z3** | −0.080 | −8% | −4% | — |\n\n**[Generation · 400k-line codebase · 5 imperative stages · z3-checked algebra](https://mikereese.dev/experiments/e024/)**\n\n| Builder / runs | Arm | Quality Δ | Builder tokens Δ | Builder time Δ | Python CC Δ |\n|---|---|---:|---:|---:|---:|\n| gpt-6-sol · medium · 3 runs | Prose docs | +0.002 to +0.036 | +1% to +71% | −3% to +10% | 0.00 |\n| | **Algebraic docs + z3** | −0.006 to +0.008 | +26% to +49% | +2% to +22% | 0.00 |\n\n\nRanges show the smallest and largest paired run results for each builder and arm. Quality Δ is the difference in mean judge score over shared scored stages; tokens and time cover every stage the builder worked. Python CC Δ is the difference in recorded mean cyclomatic complexity of Python blocks at the final Python stage (the stage before Rust conversion where applicable), arm minus same-run control; — means no Python measurement.\n\n\n**How to read this:** Positive quality Δ favors the named arm. Negative token or time Δ means the builder used less. The experiments used different tasks, models, and in the first condition a different judge; do not pool these rows into one effect size. Failed runs can still contain scored earlier stages. Each condition page shows which stage failed.\n\n**Cyclomatic complexity:** For Rust-conversion conditions, Python CC comes from the final Python stage before conversion. The retrieval condition builds no code, and the large-codebase measurements include about 400,000 lines of vendored code. These values are descriptive and cannot support one aggregate complexity effect across conditions.\n\n## What was compared\n\nWithin a run, each arm got the same task and prompts in its own workspace. The control had no supplied design docs; other arms received prose or algebraic design material, sometimes with z3 proofs. The agent built one application in stages, or answered questions about its code in the retrieval condition. Each stage built on that arm's own prior work.\n\n- **No docs** (control): No design method and no starting design. Before coding, it writes its own method, then its designs.\n- **Prose docs**: A design method in prose, and four starting charters in prose that define the application's core concepts: checks, operations, the ontology and telemetry.\n- **Algebraic docs** (hypothesis arm): A design method in algebra (capability contracts, Hoare triples, laws and proofs), and the same four charters in algebra, each with a written proof.\n- **Algebraic docs + z3** (hypothesis arm): Algebraic docs, with each charter and proof also as a z3 file, and z3 installed. It must check its algebra with z3 before writing code, and every proof must pass when a stage ends.\n\n\n**Hypothesis:** Describing a codebase algebraically instead of semantically reduces inference via proofs which reuse repeated reasoning.\n\nThe conditions change task length, prompt style, starting code, available design material, or builder model. They are separate scoped comparisons, not a sequence that isolates one factor between every row. Select a condition above for its stage results and run records.\n\n\n\n## What quality means\n\nA separate judge read each arm's code and rated seven qualities. The weighted score runs from 0 to 1. The rule for build conditions required an algebra arm to beat No docs by at least **0.1 at every scored stage of every counted run**. Retrieval now uses the same +0.1 quality target, recalculated after the runs; it also requires token and time savings. Retrieval used a different rubric, so its quality scores should not be pooled with build scores.\n\n| Quality | Weight | Basis for the judge's rating |\n|---|---:|---|\n| Ownership | 20 | Rules have one owner; functions do not mix unrelated jobs. |\n| Boundaries | 15 | Components do not rely on another component's private state. |\n| Change locality | 20 | Three planned changes can be added without reworking existing parts. |\n| Contracts | 15 | States, write decisions, and failures have clear definitions. |\n| Simplicity | 10 | No redundant state or unused layers. |\n| Readability | 10 | Names and concepts stay clear and consistent. |\n| Testability | 10 | Checks and state changes can be tested in isolation. |\n\nEach quality is rated 0–4 by defects the judge can point to; 4 means none found. Change locality instead counts how many of the three planned changes need rework. The weighted rating is divided by 4. For retrieval, the judge rated seven sections of each answer against the code, then divided the total by 28.\n\n## Variation and limits\n\n**Noise.** In the 3 sets of runs that each repeat the same setup, the same arm's score at the same stage moved by a median of 0.03 and at most 0.11, over 35 comparisons; 4 of them moved by as much as the rule's threshold. The pooled standard deviation of one arm's score is 0.030.\n With one to three runs per condition, small differences are hard to separate from run variation.\n\n- Three conditions use different builder models across runs, so those runs are not repeats.\n- The first condition used Claude Fable 5.1 as judge; the others used Claude Opus 5.5. Scores across that change are not directly comparable.\n- All conditions build or read the same application. The findings do not establish results on other codebases.\n- A run marked failed can retain scores from earlier stages. The table names its status and the number of shared scored stages.\n\n## Inspect the evidence - arm quality score delta from control\n\n\n\nThe calculation is arm score - control (no documentation) at each scored stage, on one zero line. All conditions now use a +0.1 quality target. Retrieval's target was chosen after its runs; its original rule allowed a 0.1 correctness loss. The condition pages hold stage checks, failure details, and run IDs. The [Telos project](https://mikereese.dev/work/telos/) explains the larger idea behind algebraic charters.\n\n\n\n## Data and methods\n\n[G001.json](https://mikereese.dev/data/experiments/G001.json), exported from the harness tracking store by `notebooks/export_findings.py` using `notebooks/findings/G001.yaml` at harness commit `4486667`, supplies the results on these pages. It identifies counted and excluded runs, their checks, and per-run application and grader commits and seeds.\n\nThe exported data supports inspection of these comparisons. The full run artifacts, harness, and dependent repositories are currently private, so readers cannot rerun the experiment from this page alone.\n","data_url":"https://mikereese.dev/data/experiments/G001.json","experiments":[{"id":"E009-E010","condition":"Generation · fresh codebase · 1 imperative stage","title":"One-stage build: algebraic docs missed the preset quality margin","date":"2026-09-23","status":"not supported","summary":"One build stage, two builder models. Algebraic docs scored below no docs with one model and above with the other, both times by less than the rule's threshold.","url":"https://mikereese.dev/experiments/e009-e010/","markdown_url":"https://mikereese.dev/experiments/e009-e010.md","markdown":"The agent built a small application in a single stage. With gpt-6-sol, Algebraic docs scored below our control (No docs - code only); with gpt-6-luna it scored above! However, neither difference reached the threshold set before the runs, so the prediction is not supported. Notably, luna on xhigh showed improvements on every measure on its run.\n\n## Results at a glance\n\n| Run | Arm | Quality score | Stages compared | Tokens | Time | Python CC |\n|---|---|---:|---:|---:|---:|---:|\n| E009 · gpt-6-sol · medium | No docs | **0.590** | 1 of 1 | **1.20M** | 0:07:40 | 6.25 |\n| | Prose docs | 0.580 (−0.010) | | 1.26M (+5%) | **0:07:37 (−1%)** | **5.92** |\n| | **Algebraic docs** | 0.520 (−0.070) | | 1.59M (+32%) | 0:07:51 (+2%) | 7.21 |\n\n| Run | Arm | Quality score | Stages compared | Tokens | Time | Python CC |\n|---|---|---:|---:|---:|---:|---:|\n| E010 · gpt-6-luna · xhigh | No docs | 0.520 | 1 of 1 | 9.28M | 0:30:42 | 5.65 |\n| | Prose docs | 0.580 (+0.060) | | 12.04M (+30%) | 0:40:09 (+31%) | 5.48 |\n| | **Algebraic docs** | **0.590 (+0.070)** | | **8.63M (−7%)** | **0:30:18 (−1%)** | **5.00** |\n\n\nEach score is from the run's one scored stage. Tokens and time are each run's builder totals over every stage it worked. Brackets compare with No docs. Highlighted values mark the highest quality score and lowest recorded tokens, time, and Python CC within each run; ties share the highlight. This does not mean the success rule passed. Python CC means the recorded mean cyclomatic complexity of the Python code's functions, classes, and methods at the final stage. It includes vendored code where present.\n\n\n## What changed in this condition\n\nThe agent builds the application from an empty repository, from one imperative prompt: it says what to build and how. Each run uses a different builder model, to test the result at different model sizes.\n\n- **No docs** (control): No design method and no starting design. Before coding, it writes its own method, then its designs.\n- **Prose docs**: A design method in prose. No starting charters.\n- **Algebraic docs** (hypothesis arm): A design method in algebra (capability contracts, Hoare triples, laws and proofs). No starting charters.\n\n\n**Runs:** 2 (E009, E010), one per builder model.\n\n- **E009:** gpt-6-sol · medium\n- **E010:** gpt-6-luna · xhigh\n\n**Agent harness:** `codex-cli`. **Judge:** `claude-fable-5-1-high`.\n\n\n## Stage evidence\n\n**Not supported.** Algebraic docs needed a quality gain of at least +0.1 over No docs in each run.\n\n**E009 · gpt-6-sol · medium**\n\n| Arm | Quality score | Quality Δ vs No docs |\n|---|---:|---:|\n| No docs | 0.590 | — |\n| Prose docs | 0.580 | −0.010 |\n| **Algebraic docs** | 0.520 | −0.070 |\n\n**E010 · gpt-6-luna · xhigh**\n\n| Arm | Quality score | Quality Δ vs No docs |\n|---|---:|---:|\n| No docs | 0.520 | — |\n| Prose docs | 0.580 | +0.060 |\n| **Algebraic docs** | 0.590 | +0.070 |\n\n\nQuality Δ is arm score minus No docs score in the same run. A positive value favors the named arm; a negative value favors No docs.\n\n\n**Noise.** No two runs here repeat the same setup. In this group's repeated runs, the same arm's score at the same stage moved by a median of 0.03 and at most 0.11.\n\n\n## Details\n\n### What was measured\n\nThe judge scored each arm's code after the build stage on a 0–1 quality scale. For the verdict, quality Δ is Algebraic docs score minus No docs score in the same run. The target is +0.1 or greater in every scored stage of both runs. Totals shows measured scores, tokens, and time; only the Algebraic docs versus No docs quality difference decides this result.\n\n\n### Runs\n\n#### E009 — gpt-6-sol · medium\n\n**Run ID:** `run-20260923T172943Z-c25ba500` \n**Started:** 2026-09-23 \n**Harness status:** completed\n\nEvery arm finished every stage.\n\n#### E010 — gpt-6-luna · xhigh\n\n**Run ID:** `run-20260923T174224Z-7bf8e1fb` \n**Started:** 2026-09-23 \n**Harness status:** completed\n\nEvery arm finished every stage.\n\n\n\n### Limits\n\n- Each builder model ran once, so neither model's own run-to-run noise is known.\n- The judge here was Claude Fable 5.1. Later conditions used Claude Opus 5.5, so their scores do not compare with these.\n\n### Source data\n\n[View condition `E009-E010` in G001.json](https://mikereese.dev/data/experiments/G001.json). The run IDs above identify the counted runs. See [Data and methods](https://mikereese.dev/experiments/g001/#data-and-methods) for the export method and limits.\n","data_url":"https://mikereese.dev/data/experiments/G001.json"},{"id":"E015-E017","condition":"Generation · fresh codebase · 5 imperative stages","title":"Five-stage builds: algebraic docs missed the quality margin and used more tokens","date":"2026-09-23","status":"not supported","summary":"Two runs, two builder models. The algebraic arm never beat no docs by the rule's threshold and used the most build tokens in both runs.","url":"https://mikereese.dev/experiments/e015-e017/","markdown_url":"https://mikereese.dev/experiments/e015-e017.md","markdown":"The agent grew one application over five stages of sequential imperative prompts, each in a new container that reused the previous container's environment. Algebraic docs moved up in some stages and down in others, always by less than the threshold, against No docs and against Prose docs. It used the most tokens of the three arms in both runs.\n\n## Results at a glance\n\n| Run | Arm | Average score | Stages compared | Tokens | Time | Python CC |\n|---|---|---:|---:|---:|---:|---:|\n| E015 · gpt-6-astra · xhigh | No docs | **0.370** | 4 of 5 | **20.47M** | **2:11:12** | 4.11 |\n| | Prose docs | 0.362 (−0.008) | | 22.82M (+12%) | 2:18:53 (+6%) | **4.09** |\n| | **Algebraic docs** | 0.363 (−0.007) | | 35.88M (+75%) | 2:57:17 (+35%) | 4.26 |\n\n| Run | Arm | Average score | Stages compared | Tokens | Time | Python CC |\n|---|---|---:|---:|---:|---:|---:|\n| E016 · gpt-6-sol · medium | No docs | 0.312 | 4 of 5 | 15.00M | 1:00:28 | **6.35** |\n| | Prose docs | **0.325 (+0.013)** | | **14.25M (−5%)** | **0:50:33 (−16%)** | 7.32 |\n| | **Algebraic docs** | 0.310 (−0.003) | | 20.49M (+37%) | 1:06:58 (+11%) | 7.14 |\n\n\nScores average the stages every arm of the run scored: the stages compared. Tokens and time are each run's builder totals over every stage it worked. Brackets compare with No docs. Highlighted values mark the highest quality score and lowest recorded tokens, time, and Python CC within each run; ties share the highlight. This does not mean the success rule passed. Python CC means the recorded mean cyclomatic complexity of the Python code's functions, classes, and methods at the stage immediately before Rust conversion.\n\n\n## What changed in this condition\n\nThe agent grows the application from an empty repository over five stages of imperative prompts. The last stage converts it to Rust. Each run uses a different builder model, to test the result at different model sizes.\n\n- **No docs** (control): No design method and no starting design. Before coding, it writes its own method, then its designs.\n- **Prose docs**: A design method in prose, and four starting charters in prose that define the application's core concepts: checks, operations, the ontology and telemetry.\n- **Algebraic docs** (hypothesis arm): A design method in algebra (capability contracts, Hoare triples, laws and proofs), and the same four charters in algebra, each with a written proof.\n\n\n**Runs:** 2 (E015, E016), one per builder model.\n\n- **E015:** gpt-6-astra · xhigh\n- **E016:** gpt-6-sol · medium\n\n**Agent harness:** `codex-cli`. **Judge:** `claude-opus-5-5-high`.\n\n\n## Stage evidence\n\n**Not supported.** Each part of the rule must hold in every run.\n\nStages:\n\n1. `initial-build`\n2. `telemetry-add`\n3. `parity-substrate`\n4. `process-ontology`\n5. `rust-conversion`\n\n**✗ Algebraic docs must score at least 0.1 above No docs in every graded stage, in every run: not met**\n\n| Run | Arm | Met | 1 | 2 | 3 | 4 | 5 |\n|---|---|---:|---:|---:|---:|---:|---:|\n| E015 · gpt-6-astra · xhigh | **Algebraic docs** | 0 of 4 | +0.05 | +0.02 | −0.05 | −0.05 | both failed |\n| E016 · gpt-6-sol · medium | **Algebraic docs** | 0 of 4 | 0.00 | +0.08 | −0.08 | −0.01 | both failed |\n\n\nEach cell is the arm's score minus the other arm's at that stage. ✓ marks a stage that meets that part of the rule; Met counts them, of the stages with a value. “failed” means that arm's grade failed at that stage, so the stage has no score.\n\n\n**Noise.** No two runs here repeat the same setup. In this group's repeated runs, the same arm's score at the same stage moved by a median of 0.03 and at most 0.11.\n\n\n## Details\n\n### The prediction, set before the runs\n\nThe test card names the arms as follows: `control` is No docs, `semantic` is Prose docs, `algebraic` is Algebraic docs; `total_tokens` is tokens and `elapsed_s` is time.\n\n> **Prediction.** The algebraic arm's quality score exceeds the semantic arm's by at least 0.1.\n>\n> **Falsifier.** The algebraic arm's quality score is no higher than the semantic arm's. A positive difference below 0.1 is inconclusive.\n\n**Measured:** Each arm's judged `score` at each stage. Quality Δ is the arm's score minus No docs. A stage meets the quality target at +0.1 or above. Every counted check must pass in every run. The prediction above names Prose docs; the condition result uses the No docs control.\n\n\n### Runs\n\n#### E015 — gpt-6-astra · xhigh\n\n**Run ID:** `run-20260923T223518Z-020373a4` \n**Started:** 2026-09-23 \n**Harness status:** failed\n\nThe Algebraic docs arm's grade failed at stage 5 (`rust-conversion`): `cargo test` failed. The No docs arm's grade failed at stage 5 (`rust-conversion`): `cargo test` failed. Every earlier stage was graded.\n\n#### E016 — gpt-6-sol · medium\n\n**Run ID:** `run-20260923T223651Z-b952ede3` \n**Started:** 2026-09-23 \n**Harness status:** failed\n\nThe Algebraic docs arm's grade failed at stage 5 (`rust-conversion`): `cargo test` failed. The No docs arm's grade failed at stage 5 (`rust-conversion`): `cargo test` failed. Every earlier stage was graded.\n\nThe harness marks a run *failed* when any arm's grade fails. That stage has no score for that arm; the run's other stages still count.\n\n\n### Limits\n\n- The two runs used different builder models, so they are not repeats.\n- Both runs lost the final stage's comparison: at least one of the two arms failed its Rust grade.\n\n### Source data\n\n[View condition `E015-E017` in G001.json](https://mikereese.dev/data/experiments/G001.json). The run IDs above identify the counted runs. See [Data and methods](https://mikereese.dev/experiments/g001/#data-and-methods) for the export method and limits.\n","data_url":"https://mikereese.dev/data/experiments/G001.json"},{"id":"E018-E025","condition":"Generation · fresh codebase · 5 imperative stages · z3-checked algebra","title":"Five-stage z3 builds: quality margin unmet; tokens fell in both runs","date":"2026-09-24","status":"not supported","summary":"Two repeat runs. The algebra arm proved its design with z3 before coding. Its score stayed close to no docs, and it used fewer build tokens in both runs.","url":"https://mikereese.dev/experiments/e018-e025/","markdown_url":"https://mikereese.dev/experiments/e018-e025.md","markdown":"Here the algebra arm had to prove its design in .smt files with z3 before it wrote code. Its score stayed close to No docs and met the threshold in only one stage. It used fewer tokens than No docs in both runs and consistenly showed a lower average cyclomatic complexity.\n\n## Results at a glance\n\n| Run | Arm | Average score | Stages compared | Tokens | Time | Python CC |\n|---|---|---:|---:|---:|---:|---:|\n| E018 | No docs | 0.298 | 4 of 5 | 31.52M | 1:25:20 | 8.09 |\n| | **Algebraic docs + z3** | **0.323 (+0.025)** | | **20.12M (−36%)** | **1:00:39 (−29%)** | **7.21** |\n\n| Run | Arm | Average score | Stages compared | Tokens | Time | Python CC |\n|---|---|---:|---:|---:|---:|---:|\n| E025 | No docs | 0.280 | 4 of 5 | 22.95M | **1:05:59** | 6.58 |\n| | **Algebraic docs + z3** | **0.297 (+0.017)** | | **21.29M (−7%)** | 1:12:01 (+9%) | **6.31** |\n\n\nScores average the stages every arm of the run scored: the stages compared. Tokens and time are each run's builder totals over every stage it worked. Brackets compare with No docs. Highlighted values mark the highest quality score and lowest recorded tokens, time, and Python CC within each run; ties share the highlight. This does not mean the success rule passed. Python CC means the recorded mean cyclomatic complexity of the Python code's functions, classes, and methods at the stage immediately before Rust conversion.\n\n\n## What changed in this condition\n\nThe agent grows the application from an empty repository over five stages of imperative prompts. The last stage converts it to Rust. This condition has no Prose docs arm.\n\n- **No docs** (control): No design method and no starting design. Before coding, it writes its own method, then its designs.\n- **Algebraic docs + z3** (hypothesis arm): Algebraic docs, with each charter and proof also as a z3 file, and z3 installed. It must check its algebra with z3 before writing code, and every proof must pass when a stage ends.\n\n\n**Runs:** 2 (E018, E025), repeats of one setup.\n\n**Builder model:** gpt-6-sol · medium\n\n**Agent harness:** `codex-cli`. **Judge:** `claude-opus-5-5-high`.\n\n\n## Stage evidence\n\n**Not supported.** Each part of the rule must hold in every run.\n\nStages:\n\n1. `initial-build`\n2. `telemetry-add`\n3. `parity-substrate`\n4. `process-ontology`\n5. `rust-conversion`\n\n**✗ Algebraic docs + z3 must score at least 0.1 above No docs in every graded stage, in every run: not met**\n\n| Run | Arm | Met | 1 | 2 | 3 | 4 | 5 |\n|---|---|---:|---:|---:|---:|---:|---:|\n| E018 | **Algebraic docs + z3** | 1 of 4 | 0.00 | +0.02 | −0.03 | +0.11 ✓ | No docs failed |\n| E025 | **Algebraic docs + z3** | 0 of 4 | +0.02 | +0.02 | −0.03 | +0.06 | failed |\n\n\nEach cell is the arm's score minus the other arm's at that stage. ✓ marks a stage that meets that part of the rule; Met counts them, of the stages with a value. “failed” means that arm's grade failed at that stage, so the stage has no score.\n\n\n**Noise.** Between runs that repeat the same setup, the same arm's score at the same stage moved by a median of 0.03 and at most 0.05, over 8 comparisons; none moved by 0.10 or more, the rule's threshold.\n\n\n## Details\n\n### The prediction, set before the runs\n\nThe test card names the arms as follows: `control` is No docs, `algebraic` is Algebraic docs + z3; `total_tokens` is tokens and `elapsed_s` is time.\n\n> **Prediction.** The algebraic arm's quality score exceeds the semantic arm's by at least 0.1.\n>\n> **Falsifier.** The algebraic arm's quality score is no higher than the semantic arm's. A positive difference below 0.1 is inconclusive.\n\n**Measured:** Each arm's judged `score` at each stage. Quality Δ is the arm's score minus No docs. A stage meets the quality target at +0.1 or above. Every counted check must pass in every run. A planned comparison with Prose docs could not be measured because that arm did not run.\n\n\n### z3\n\n| Run | Arm | Last stage | Proof goals | Proved | Disproved | Undecided | Stages the agent ran z3 |\n|---|---|---:|---:|---:|---:|---:|---:|\n| E018 | **Algebraic docs + z3** | 5 | 115 | 115 | 0 | 0 | 5 of 5 |\n| E025 | **Algebraic docs + z3** | 5 | 92 | 90 | 2 | 0 | 4 of 5 |\n\n\nAfter each stage, the system runs z3 on the proof goals in the design's z3 files and counts those it proves, disproves or leaves undecided. The counts are from the last stage that has them; — means none were recorded. The last column counts stages in which the agent itself ran z3. In E025, some goals were still disproved at the last stage.\n\n\n### Runs\n\n#### E018 — gpt-6-sol · medium\n\n**Run ID:** `run-20260923T232241Z-59b76d12` \n**Started:** 2026-09-23 \n**Harness status:** failed\n\nThe No docs arm's grade failed at stage 5 (`rust-conversion`): `cargo test` failed. Every earlier stage was graded.\n\n#### E025 — gpt-6-sol · medium\n\n**Run ID:** `run-20260924T031035Z-6515aace` \n**Started:** 2026-09-24 \n**Harness status:** failed\n\nThe Algebraic docs + z3 arm's grade failed at stage 5 (`rust-conversion`): `cargo test` failed. Every earlier stage was graded.\n\nThe harness marks a run *failed* when any arm's grade fails. That stage has no score for that arm; the run's other stages still count.\n\n\n### Source data\n\n[View condition `E018-E025` in G001.json](https://mikereese.dev/data/experiments/G001.json). The run IDs above identify the counted runs. See [Data and methods](https://mikereese.dev/experiments/g001/#data-and-methods) for the export method and limits.\n","data_url":"https://mikereese.dev/data/experiments/G001.json"},{"id":"E020-E026","condition":"Generation · fresh codebase · 7 declarative stages · z3-checked algebra","title":"Seven declarative stages: quality margin unmet; tokens rose in three runs","date":"2026-09-24","status":"not supported","summary":"Three repeat runs with declarative prompts. The z3-checked algebra arm scored level with or below no docs and used more build tokens in every run.","url":"https://mikereese.dev/experiments/e020-e026/","markdown_url":"https://mikereese.dev/experiments/e020-e026.md","markdown":"The prompts here state what must be true, not how to build it. Across three repeat runs, the z3-checked algebra arm scored level with or below No docs and never reached the threshold. It used more tokens than No docs in every run.\n\n## Results at a glance\n\n| Run | Arm | Average score | Stages compared | Tokens | Time | Python CC |\n|---|---|---:|---:|---:|---:|---:|\n| E020 run 1 | No docs | **0.322** | 6 of 7 | **31.21M** | 1:39:38 | **5.63** |\n| | **Algebraic docs + z3** | 0.292 (−0.030) | | 36.36M (+17%) | **1:38:00 (−2%)** | 7.64 |\n\n| Run | Arm | Average score | Stages compared | Tokens | Time | Python CC |\n|---|---|---:|---:|---:|---:|---:|\n| E020 run 2 | No docs | **0.355** | 6 of 7 | **22.03M** | 1:23:43 | **5.71** |\n| | **Algebraic docs + z3** | 0.273 (−0.082) | | 26.57M (+21%) | **1:21:00 (−3%)** | 6.48 |\n\n| Run | Arm | Average score | Stages compared | Tokens | Time | Python CC |\n|---|---|---:|---:|---:|---:|---:|\n| E026 | No docs | 0.303 | 6 of 7 | **24.43M** | 1:37:05 | 6.61 |\n| | **Algebraic docs + z3** | **0.308 (+0.005)** | | 28.14M (+15%) | **1:34:16 (−3%)** | **6.09** |\n\n\nScores average the stages every arm of the run scored: the stages compared. Tokens and time are each run's builder totals over every stage it worked. Brackets compare with No docs. Highlighted values mark the highest quality score and lowest recorded tokens, time, and Python CC within each run; ties share the highlight. This does not mean the success rule passed. Python CC means the recorded mean cyclomatic complexity of the Python code's functions, classes, and methods at the stage immediately before Rust conversion.\n\n\n## What changed in this condition\n\nThe agent grows the application from an empty repository over seven stages of declarative prompts: each states only what must be true when the stage ends. The last stage converts it to Rust. This condition has no Prose docs arm.\n\n- **No docs** (control): No design method and no starting design. Before coding, it writes its own method, then its designs.\n- **Algebraic docs + z3** (hypothesis arm): Algebraic docs, with each charter and proof also as a z3 file, and z3 installed. It must check its algebra with z3 before writing code, and every proof must pass when a stage ends.\n\n\n**Runs:** 3 (E020 run 1, E020 run 2, E026), repeats of one setup.\n\n**Builder model:** gpt-6-sol · medium\n\n**Agent harness:** `codex-cli`. **Judge:** `claude-opus-5-5-high`.\n\n\n## Stage evidence\n\n**Not supported.** Each part of the rule must hold in every run.\n\nStages:\n\n1. `initial-build`\n2. `telemetry-add`\n3. `parity-substrate`\n4. `process-ontology`\n5. `check-no-write`\n6. `concurrent-submissions`\n7. `rust-conversion`\n\n**✗ Algebraic docs + z3 must score at least 0.1 above No docs in every graded stage, in every run: not met**\n\n| Run | Arm | Met | 1 | 2 | 3 | 4 | 5 | 6 | 7 |\n|---|---|---:|---:|---:|---:|---:|---:|---:|---:|\n| E020 run 1 | **Algebraic docs + z3** | 0 of 6 | −0.04 | −0.06 | −0.02 | −0.04 | 0.00 | −0.02 | No docs failed |\n| E020 run 2 | **Algebraic docs + z3** | 0 of 6 | −0.11 | −0.10 | −0.03 | −0.07 | −0.11 | −0.07 | both failed |\n| E026 | **Algebraic docs + z3** | 0 of 6 | +0.09 | 0.00 | 0.00 | 0.00 | 0.00 | −0.06 | both failed |\n\n\nEach cell is the arm's score minus the other arm's at that stage. ✓ marks a stage that meets that part of the rule; Met counts them, of the stages with a value. “failed” means that arm's grade failed at that stage, so the stage has no score.\n\n\n**Noise.** Between runs that repeat the same setup, the same arm's score at the same stage moved by a median of 0.05 and at most 0.11, over 12 comparisons; 2 moved by 0.10 or more, the rule's threshold.\n\n\n## Details\n\n### The prediction, set before the runs\n\nThe test card names the arms as follows: `control` is No docs, `algebraic` is Algebraic docs + z3; `total_tokens` is tokens and `elapsed_s` is time.\n\n> **Prediction.** The algebraic arm's quality score exceeds the semantic arm's by at least 0.1.\n>\n> **Falsifier.** The algebraic arm's quality score is no higher than the semantic arm's. A positive difference below 0.1 is inconclusive.\n\n**Measured:** Each arm's judged `score` at each stage. Quality Δ is the arm's score minus No docs. A stage meets the quality target at +0.1 or above. Every counted check must pass in every run. A planned comparison with Prose docs could not be measured because that arm did not run.\n\n\n### z3\n\n| Run | Arm | Last stage | Proof goals | Proved | Disproved | Undecided | Stages the agent ran z3 |\n|---|---|---:|---:|---:|---:|---:|---:|\n| E020 run 1 | **Algebraic docs + z3** | 7 | 139 | 139 | 0 | 0 | 5 of 7 |\n| E020 run 2 | **Algebraic docs + z3** | 7 | 142 | 142 | 0 | 0 | 7 of 7 |\n| E026 | **Algebraic docs + z3** | 7 | 159 | 155 | 4 | 0 | 5 of 7 |\n\n\nAfter each stage, the system runs z3 on the proof goals in the design's z3 files and counts those it proves, disproves or leaves undecided. The counts are from the last stage that has them; — means none were recorded. The last column counts stages in which the agent itself ran z3. In E026, some goals were still disproved at the last stage.\n\n\n### Runs\n\n#### E020 run 1 — gpt-6-sol · medium\n\n**Run ID:** `run-20260923T235842Z-4979eeee` \n**Started:** 2026-09-23 \n**Harness status:** failed\n\nThe No docs arm's grade failed at stage 7 (`rust-conversion`): `cargo test` failed. Every earlier stage was graded.\n\n#### E020 run 2 — gpt-6-sol · medium\n\n**Run ID:** `run-20260924T031833Z-a6b163d1` \n**Started:** 2026-09-24 \n**Harness status:** failed\n\nThe Algebraic docs + z3 arm's grade failed at stage 7 (`rust-conversion`): `cargo test` failed. The No docs arm's grade failed at stage 7 (`rust-conversion`): `cargo test` failed. Every earlier stage was graded.\n\n#### E026 — gpt-6-sol · medium\n\n**Run ID:** `run-20260924T033220Z-8386576e` \n**Started:** 2026-09-24 \n**Harness status:** failed\n\nThe Algebraic docs + z3 arm's grade failed at stage 7 (`rust-conversion`): `cargo test` failed. The No docs arm's grade failed at stage 7 (`rust-conversion`): `cargo test` failed. Every earlier stage was graded.\n\nThe harness marks a run *failed* when any arm's grade fails. That stage has no score for that arm; the run's other stages still count.\n\n\n### Source data\n\n[View condition `E020-E026` in G001.json](https://mikereese.dev/data/experiments/G001.json). The run IDs above identify the counted runs. See [Data and methods](https://mikereese.dev/experiments/g001/#data-and-methods) for the export method and limits.\n","data_url":"https://mikereese.dev/data/experiments/G001.json"},{"id":"E022-E023","condition":"Retrieval · 400k-line codebase · 5 questions · z3-checked algebra","title":"Design docs missed quality and efficiency targets in retrieval","date":"2026-09-24","status":"not supported","summary":"Two builder models answered five questions each. Neither design form met the retrospective +0.1 correctness target and saved tokens and time in both runs.","url":"https://mikereese.dev/experiments/e022-e023/","markdown_url":"https://mikereese.dev/experiments/e022-e023.md","markdown":"Before a planned change, the agent explained one part of a large application. Neither design form met the retrospective +0.1 correctness target and reliably saved tokens and time. With gpt-6-sol both forms used more tokens on most questions, while with gpt-6-luna Prose docs used fewer on every question.\n\n## Results at a glance\n\n| Run | Arm | Average score | Stages compared | Tokens | Time |\n|---|---|---:|---:|---:|---:|\n| E022 · gpt-6-sol · medium | No docs | **0.934** | 5 of 5 | **1.23M** | **0:09:31** |\n| | Prose docs | 0.906 (−0.028) | | 1.56M (+26%) | 0:09:32 (+0.2%) |\n| | **Algebraic docs + z3** | 0.892 (−0.042) | | 1.51M (+22%) | 0:09:51 (+3%) |\n\n| Run | Arm | Average score | Stages compared | Tokens | Time |\n|---|---|---:|---:|---:|---:|\n| E023 · gpt-6-luna · medium | No docs | 0.800 | 5 of 5 | 1.35M | 0:07:14 |\n| | Prose docs | **0.820 (+0.020)** | | **1.05M (−22%)** | 0:07:53 (+9%) |\n| | **Algebraic docs + z3** | 0.720 (−0.080) | | 1.23M (−8%) | **0:06:56 (−4%)** |\n\n\nScores average the stages every arm of the run scored: the stages compared. Tokens and time are each run's builder totals over every stage it worked. Brackets compare with No docs. Highlighted values mark the highest quality score and lowest recorded tokens, time, and Python CC within each run; ties share the highlight. This does not mean the success rule passed.\n\n\n## What changed in this condition\n\nThe agent builds nothing. Each stage asks it one question about a part of an existing application inside about 400,000 lines of vendored code: what the part is, what it must keep true, and how a planned change should go. Every arm gets the same instruction: write the answer to the file the prompt names, and change nothing else. The arms differ only in the design records already in the workspace. Each run uses a different builder model, to test the result at different model sizes.\n\n- **No docs** (control): The code alone. No design records.\n- **Prose docs**: The application's finished design, in prose.\n- **Algebraic docs + z3** (hypothesis arm): The application's finished design in algebra, with its z3 proof files.\n\n\n**Runs:** 2 (E022, E023), one per builder model.\n\n- **E022:** gpt-6-sol · medium\n- **E023:** gpt-6-luna · medium\n\n**Agent harness:** `codex-cli`. **Judge:** `claude-opus-5-5-high`.\n\n\n## Stage evidence\n\n**Not supported.** Each part of the rule must hold in every run.\n\nStages:\n\n1. `checks`\n2. `ontology`\n3. `operation`\n4. `telemetry`\n5. `composition`\n\n**✗ Algebraic docs + z3 and Prose docs must each score at least 0.1 above No docs in every graded stage, in every run: not met**\n\n| Run | Arm | Met | 1 | 2 | 3 | 4 | 5 |\n|---|---|---:|---:|---:|---:|---:|---:|\n| E022 · gpt-6-sol · medium | **Algebraic docs + z3** | 0 of 5 | −0.07 | −0.07 | −0.04 | −0.07 | +0.04 |\n| E023 · gpt-6-luna · medium | **Algebraic docs + z3** | 0 of 5 | −0.04 | −0.07 | −0.18 | −0.07 | −0.04 |\n| E022 · gpt-6-sol · medium | Prose docs | 0 of 5 | −0.03 | 0.00 | −0.11 | 0.00 | 0.00 |\n| E023 · gpt-6-luna · medium | Prose docs | 1 of 5 | +0.03 | −0.07 | −0.11 | +0.22 ✓ | +0.03 |\n\n**✗ Algebraic docs + z3 and Prose docs must each use fewer tokens than No docs in most stages, in every run: not met**\n\n| Run | Arm | Met | 1 | 2 | 3 | 4 | 5 |\n|---|---|---:|---:|---:|---:|---:|---:|\n| E022 · gpt-6-sol · medium | **Algebraic docs + z3** | 1 of 5 | −7% ✓ | +36% | +18% | +64% | +6% |\n| E023 · gpt-6-luna · medium | **Algebraic docs + z3** | 3 of 5 | −20% ✓ | +16% | −33% ✓ | +1% | −2% ✓ |\n| E022 · gpt-6-sol · medium | Prose docs | 0 of 5 | +9% | +63% | +34% | +9% | +25% |\n| E023 · gpt-6-luna · medium | Prose docs | 5 of 5 | −12% ✓ | −43% ✓ | −21% ✓ | −42% ✓ | −2% ✓ |\n\n**✗ Algebraic docs + z3 and Prose docs must each use less time than No docs in most stages, in every run: not met**\n\n| Run | Arm | Met | 1 | 2 | 3 | 4 | 5 |\n|---|---|---:|---:|---:|---:|---:|---:|\n| E022 · gpt-6-sol · medium | **Algebraic docs + z3** | 2 of 5 | −24% ✓ | +11% | +1% | +32% | −4% ✓ |\n| E023 · gpt-6-luna · medium | **Algebraic docs + z3** | 1 of 5 | +31% | +5% | −37% ✓ | +2% | +8% |\n| E022 · gpt-6-sol · medium | Prose docs | 3 of 5 | −15% ✓ | +28% | +23% | −20% ✓ | −9% ✓ |\n| E023 · gpt-6-luna · medium | Prose docs | 2 of 5 | +26% | +32% | −30% ✓ | −5% ✓ | +43% |\n\n**✓ Algebraic docs + z3 must use fewer tokens than Prose docs in most stages, in every run: met**\n\n| Run | Arm | Met | 1 | 2 | 3 | 4 | 5 |\n|---|---|---:|---:|---:|---:|---:|---:|\n| E022 · gpt-6-sol · medium | **Algebraic docs + z3** | 4 of 5 | −15% ✓ | −16% ✓ | −12% ✓ | +51% | −16% ✓ |\n| E023 · gpt-6-luna · medium | **Algebraic docs + z3** | 3 of 5 | −10% ✓ | +102% | −15% ✓ | +75% | −0.4% ✓ |\n\n\nEach cell is the arm's score minus the other arm's, or its change in tokens or time at that stage. ✓ marks a stage that meets that part of the rule; Met counts them, of the stages with a value.\n\n\n**Noise.** No two runs here repeat the same setup. In this group's repeated runs, the same arm's score at the same stage moved by a median of 0.03 and at most 0.11.\n\n\n## Details\n\n### Original prediction and revised quality target\n\nThe test card names the arms as follows: `control` is No docs, `semantic` is Prose docs, `algebraic_solver` is Algebraic docs + z3; `total_tokens` is tokens and `elapsed_s` is time.\n\n> **Prediction.** Each design arm uses fewer total_tokens and less elapsed_s than the control in most stages, and its answer correctness is no more than 0.1 below the control's. The algebraic arm uses fewer tokens than the semantic arm.\n>\n> **Falsifier.** A design arm uses as many or more tokens than the control, or its correctness falls more than 0.1 below the control's. The harness verdict tests only the correctness floor: supported means the arm's score is at most 0.1 below the control's. Tokens and time are compared from the system measurements in the observer.\n\n**Measured:** Each arm's judged `score` at each stage. Quality Δ is the arm's score minus No docs. A stage meets the quality target at +0.1 or above. Every counted check must pass in every run. The retrieval quality target was applied after the runs; their recorded target was −0.1. The scores and token and time measurements are unchanged.\n\n\n### What went wrong first\n\nIn the first run of each experiment, the judge's copy of the workspace left out the design docs, so it marked the design arms' citations as missing:\n\n> The answer cites design docs under ../docs/design/..., which do not exist in the working directory … those citations were scored as nonexistent paths.\n\nI found this in the judge reports, fixed the grader (`c7ac0d7`, \"Keep the design tree in the retrieval judge's copy\") and repeated both runs. This page counts only the repeats.\n\n### Runs\n\n#### E022 — gpt-6-sol · medium\n\n**Run ID:** `run-20260924T023130Z-ccbee4e6` \n**Started:** 2026-09-24 \n**Harness status:** completed\n\nEvery arm finished every stage.\n\n#### E023 — gpt-6-luna · medium\n\n**Run ID:** `run-20260924T023152Z-aa033b06` \n**Started:** 2026-09-24 \n**Harness status:** completed\n\nEvery arm finished every stage.\n\n\n\n### Limits\n\n- The two runs used different builder models, so they are not repeats.\n\n### Source data\n\n[View condition `E022-E023` in G001.json](https://mikereese.dev/data/experiments/G001.json). The run IDs above identify the counted runs. See [Data and methods](https://mikereese.dev/experiments/g001/#data-and-methods) for the export method and limits.\n","data_url":"https://mikereese.dev/data/experiments/G001.json"},{"id":"E024","condition":"Generation · 400k-line codebase · 5 imperative stages · z3-checked algebra","title":"Existing-codebase changes: z3 algebra missed the quality margin and used more tokens","date":"2026-09-24","status":"not supported","summary":"Three repeat runs extending an application its design records already describe. No doc form beat no docs by the rule's threshold, and the algebra arm used more build tokens in every run.","url":"https://mikereese.dev/experiments/e024/","markdown_url":"https://mikereese.dev/experiments/e024.md","markdown":"Here the design docs start already written for the existing application. Across three repeat runs, neither Prose docs nor Algebraic docs + z3 lifted the score by the threshold in any stage. Algebraic docs + z3 used more tokens than No docs in every run.\n\n## Results at a glance\n\n| Run | Arm | Average score | Stages compared | Tokens | Time | Python CC |\n|---|---|---:|---:|---:|---:|---:|\n| E024 run 1 | No docs | 0.310 | 5 of 5 | **10.01M** | **0:50:31** | **3.27** |\n| | Prose docs | **0.312 (+0.002)** | | 17.16M (+71%) | 0:55:44 (+10%) | 3.27 |\n| | **Algebraic docs + z3** | 0.304 (−0.006) | | 14.93M (+49%) | 0:59:38 (+18%) | 3.27 |\n\n| Run | Arm | Average score | Stages compared | Tokens | Time | Python CC |\n|---|---|---:|---:|---:|---:|---:|\n| E024 run 2 | No docs | 0.310 | 5 of 5 | **12.05M** | 0:50:09 | 3.27 |\n| | Prose docs | **0.346 (+0.036)** | | 12.19M (+1%) | **0:48:45 (−3%)** | 3.27 |\n| | **Algebraic docs + z3** | 0.318 (+0.008) | | 16.61M (+38%) | 1:01:11 (+22%) | **3.27** |\n\n| Run | Arm | Average score | Stages compared | Tokens | Time | Python CC |\n|---|---|---:|---:|---:|---:|---:|\n| E024 run 3 | No docs | 0.294 | 5 of 5 | **12.21M** | **0:53:28** | 3.27 |\n| | Prose docs | **0.308 (+0.014)** | | 14.07M (+15%) | 0:57:44 (+8%) | 3.27 |\n| | **Algebraic docs + z3** | 0.292 (−0.002) | | 15.34M (+26%) | 0:54:31 (+2%) | **3.27** |\n\n\nScores average the stages every arm of the run scored: the stages compared. Tokens and time are each run's builder totals over every stage it worked. Brackets compare with No docs. Highlighted values mark the highest quality score and lowest recorded tokens, time, and Python CC within each run; ties share the highlight. This does not mean the success rule passed. Python CC means the recorded mean cyclomatic complexity of the Python code's functions, classes, and methods at the final stage. It includes vendored code where present.\n\n\n## What changed in this condition\n\nThe agent extends an existing application inside about 400,000 lines of vendored code, over five stages of imperative prompts.\n\n- **No docs** (control): No design method and no starting design. Before coding, it writes its own method, then its designs.\n- **Prose docs**: The prose method, and the existing application's finished design in prose in place of the starting charters.\n- **Algebraic docs + z3** (hypothesis arm): The algebraic method with z3 installed, and the existing application's finished design in algebra, with its z3 files, in place of the starting charters. Every proof must pass when a stage ends.\n\n\n**Runs:** 3 (E024 run 1, E024 run 2, E024 run 3), repeats of one setup.\n\n**Builder model:** gpt-6-sol · medium\n\n**Agent harness:** `codex-cli`. **Judge:** `claude-opus-5-5-high`.\n\n\n## Stage evidence\n\n**Not supported.** Each part of the rule must hold in every run.\n\nStages:\n\n1. `duplicate-name-check`\n2. `entry-supersession`\n3. `revise-information`\n4. `agent-token-telemetry`\n5. `ontology-audit`\n\n**✗ Algebraic docs + z3 must score at least 0.1 above No docs in every graded stage, in every run: not met**\n\n| Run | Arm | Met | 1 | 2 | 3 | 4 | 5 |\n|---|---|---:|---:|---:|---:|---:|---:|\n| E024 run 1 | **Algebraic docs + z3** | 0 of 5 | −0.02 | +0.02 | +0.03 | 0.00 | −0.06 |\n| E024 run 2 | **Algebraic docs + z3** | 0 of 5 | −0.01 | +0.01 | +0.06 | −0.03 | +0.01 |\n| E024 run 3 | **Algebraic docs + z3** | 0 of 5 | 0.00 | −0.07 | +0.04 | 0.00 | +0.02 |\n\n\nEach cell is the arm's score minus the other arm's at that stage. ✓ marks a stage that meets that part of the rule; Met counts them, of the stages with a value.\n\n\n**Noise.** Between runs that repeat the same setup, the same arm's score at the same stage moved by a median of 0.03 and at most 0.11, over 15 comparisons; 2 moved by 0.10 or more, the rule's threshold.\n\n\n## Details\n\n### The prediction, set before the runs\n\nThe test card names the arms as follows: `control` is No docs, `semantic` is Prose docs, `algebraic_solver` is Algebraic docs + z3; `total_tokens` is tokens and `elapsed_s` is time.\n\n> **Prediction.** The algebraic_solver arm's quality score exceeds the control's by at least 0.1 in each stage; tokens and time are recorded as secondary measures.\n>\n> **Falsifier.** The algebraic_solver arm's quality score is no higher than the control's. A positive difference below 0.1 is inconclusive.\n\n**Measured:** Each arm's judged `score` at each stage. Quality Δ is the arm's score minus No docs. A stage meets the quality target at +0.1 or above. Every counted check must pass in every run.\n\n\n### z3\n\n| Run | Arm | Last stage | Proof goals | Proved | Disproved | Undecided | Stages the agent ran z3 |\n|---|---|---:|---:|---:|---:|---:|---:|\n| E024 run 1 | **Algebraic docs + z3** | 5 | 222 | 222 | 0 | 0 | 3 of 5 |\n| E024 run 2 | **Algebraic docs + z3** | 5 | 201 | 201 | 0 | 0 | 4 of 5 |\n| E024 run 3 | **Algebraic docs + z3** | 5 | 187 | 187 | 0 | 0 | 3 of 5 |\n\n\nAfter each stage, the system runs z3 on the proof goals in the design's z3 files and counts those it proves, disproves or leaves undecided. The counts are from the last stage that has them; — means none were recorded. The last column counts stages in which the agent itself ran z3.\n\n\n### Runs\n\n#### E024 run 1 — gpt-6-sol · medium\n\n**Run ID:** `run-20260924T013909Z-79df517b` \n**Started:** 2026-09-24 \n**Harness status:** completed\n\nEvery arm finished every stage.\n\n#### E024 run 2 — gpt-6-sol · medium\n\n**Run ID:** `run-20260924T013934Z-a085b261` \n**Started:** 2026-09-24 \n**Harness status:** completed\n\nEvery arm finished every stage.\n\n#### E024 run 3 — gpt-6-sol · medium\n\n**Run ID:** `run-20260924T014000Z-e4983936` \n**Started:** 2026-09-24 \n**Harness status:** completed\n\nEvery arm finished every stage.\n\n\n\n### Source data\n\n[View condition `E024` in G001.json](https://mikereese.dev/data/experiments/G001.json). The run IDs above identify the counted runs. See [Data and methods](https://mikereese.dev/experiments/g001/#data-and-methods) for the export method and limits.\n","data_url":"https://mikereese.dev/data/experiments/G001.json"}]}],"resume":{"$schema":"https://raw.githubusercontent.com/jsonresume/resume-schema/v1.0.0/schema.json","basics":{"email":"mike.reese1212@gmail.com","label":"I like inventing things. I'm building agentic systems and the experiments that show whether they work.","location":{"city":"San Francisco, CA"},"name":"Mike Reese","profiles":[{"network":"GitHub","url":"https://github.com/mike-reese","username":"mike-reese"},{"network":"LinkedIn","url":"https://www.linkedin.com/in/mike-reese-4b2129188/","username":null}],"summary":"Founded 2 companies, innovated in the heavy equipment industry with a physics-based universal compatibility matching engine based on a graph structure, and currently trying to find the Rosetta Stone for the context layer of LLMs that unlocks purely declarative prompting for long-running tasks in an evolving system.\n\nBig proponent of thinking from first principles, removing friction, questioning the status quo, and reducing complex systems down to quantifiable, pure-signal experiments. AKA - just cut the Gordian Knot.\n \nSo, I'm really enjoying the evolving paradigm and rapid pace at the frontier of AI :-)","url":"https://mikereese.dev"},"meta":{"canonical":"https://mikereese.dev/resume.json","version":"v1.0.0"},"projects":[{"description":"Forked and hosted the Gas City multi-agent runtime on AWS: one EKS cluster per environment, autoscaled with Karpenter, running about 12 agent crews, one per business domain, communicating via EventBridge. Custom agent packs and role definitions, rewrote Gas City, Dolt and Beads integrations to be Kubernetes-native.","entity":"Matchpoint Intelligence","keywords":["Gas City","Kubernetes (EKS)","Karpenter","AWS CDK","Beads","Dolt"],"name":"Gas City on AWS","roles":["Sole Developer"],"url":"https://mikereese.dev/work/gas-city/"},{"description":"Ideator → Author → Generator → Evaluator → Judge automated loops over 465 work units folded about 20 repos into monorepos. DAG dependency graph, Observability dashboard, Full telemetry coverage with 'Miner' agents triggered on merges to identify design pattern opportunities, operational complexity improvements, and removal candidates (bloat reduction). Account pool with automated swapping and session resume when limits were hit.","entity":"Matchpoint Intelligence","keywords":["Codex","Claude Code","Python","SQLite","git worktrees"],"name":"Loop Harness","roles":["Sole Developer"],"url":"https://mikereese.dev/work/loop-harness/"},{"description":"Current Focus. An application layer and context management system for agentic work to facilitate declarative, long-running tasks in an evolving system (which is every system). Purpose is to reduce Drift / Bloat / Sprawl / Regressions over long-running tasks by creating a measure for epistemic understanding with agents. Stateful Knowledgebase to derive the current 'Truth', Self-extending ontology, context / action / effect checks, telemetry on measurable system objectives and agent sessions / turns / tool use for a given declarative prompt. Three prototypes to date. Experiment harness exists solely to facilitate design decisions for this idea.","entity":"Independent research","keywords":["Python","Rust","Dolt","FastAPI","Cytoscape.js","MCP","Claude Code hooks","Codex hooks","Docker"],"name":"Telos","roles":["Sole author"],"url":"https://mikereese.dev/work/telos/"},{"description":"Isolated runtime experiments on agentic techniques. Goal is pure-signal hypothesis testing, prioritizing rapid iteration and validation with toy model repos. Uses Scientific Method - causal claims, control/test arms, measurable outcomes, separate validation environment combining quantitative and qualitative measurements, multiple runs to reduce variance.","entity":"Independent research","keywords":["Python","Docker","SQLite","LLM judges","z3"],"name":"Experiment harness","roles":["Sole author"],"url":"https://mikereese.dev/work/experiment-harness/"},{"description":"Invented a universal, physics-based compatibility engine for heavy machinery and attachments. Used inference to project machine and attachment specifications into hardware component-level subgraphs & used set theory to generate the Cartesian product of viable component combinations. Modeled attachment fit for heavy equipment as the intersection of three physical property relationships between linked hardware components: connect, power and lift. 25M-node, 100M-edge graph.","entity":"Matchpoint Intelligence","keywords":["Python","AWS Step Functions","Lambda","SQS","DynamoDB","Neptune","Gremlin","OpenCypher","CDK","Bedrock"],"name":"Compatibility engine","roles":["Sole Developer"],"url":"https://mikereese.dev/work/compatibility-engine/"},{"description":"Let Attachments King shoppers filter every product by their machine, beyond what Shopify's collection pages allow, using a Bloom filter per product carried through to checkout. Online-store sales rose 99% in the first month.","entity":"Attachments King","keywords":["Shopify Liquid","Shopify metaobjects and metafields","JavaScript modules","Event bus and reducer","IndexedDB","Checkout UI extensions"],"name":"Storefront fitment filter","roles":["Sole Developer"],"url":"https://mikereese.dev/work/storefront-fitment-filter/"}],"skills":[{"keywords":["Claude Code","Codex","MCP servers","agent hooks","generator/evaluator loops","LLM judges"],"name":"Agents"},{"keywords":["pre-registered design","isolation with Docker","LLM-as-judge rubrics","z3"],"name":"Experiments"},{"keywords":["graph modeling","Neptune (Gremlin, OpenCypher)","Dolt","SQLite","set theory","Bloom filters"],"name":"Data"},{"keywords":["AWS Step Functions","Lambda","SQS","DynamoDB","EKS","Karpenter","CDK","Cloudflare Workers"],"name":"Cloud"},{"keywords":["Shopify themes and checkout extensions","IndexedDB","Next.js"],"name":"Frontend"},{"keywords":["Rust","Python","TypeScript"],"name":"Languages"}],"work":[{"endDate":null,"name":"Telos","position":"Independent research","startDate":"2026-08","summary":"A context and application layer for agentic work, prioritizing epistemic understanding in an evolving system for declarative, long-running tasks. Designing via an experiment harness that validates agent techniques with controlled experiments."},{"endDate":"2026-09","name":"Matchpoint Intelligence","position":"Founder","startDate":"2026-02","summary":"Invented and built a physics-based, universal heavy-equipment compatibility platform, its agent data API, and the agent crews that maintained it."},{"endDate":"2026-01","name":"Attachments King","position":"Founder","startDate":"2023-12","summary":"E-commerce company for heavy machinery attachments. Bootstrapped to $4.21M in annual revenue."}]},"files":{"llms_txt":"https://mikereese.dev/llms.txt","llms_full_txt":"https://mikereese.dev/llms-full.txt","resume_json":"https://mikereese.dev/resume.json","mcp":"https://mikereese.dev/mcp","agents_page":"https://mikereese.dev/agents/"}}