The challenge

Three papers, one claim, one command each. The author thinks he is right and would rather be shown wrong than believed.

This page is the review. There is no venue, no referee and no gate behind the Efficient Thinking series; there is a public record, a reproduction that fails loud, and this invitation. It is written by an engineer from outside the research community, working with AI assistance, and it says so on the front page.

The claim under attack. This series separates the intelligence of a frozen model from the experience it can use and the experience it can acquire, shows each is measurable on its own, and finds that a learning plateau can belong to acquisition alone. Every number behind it comes from one experimental system — a frozen 7B model on synthetic pipeline-debugging problems with a verifier — and the papers say "can" for that reason. Whether it holds elsewhere is exactly what this page asks you to find out.

The three results, one sentence each

PaperWhat it claimsRead
VIIIA read-only linear head over a frozen model's own hidden state, fitted on its verified history, moves the quality–compute frontier: equal success at 2.3× less compute, +11 to +14 problems in 100 across four paired measurements of 300, and the model's own preference or a five-sample vote at the same decision do not reach it.paper
VIII-bFeeding the agent more of its own experience does not compound: the second generation's gain does not separate from the loop's run-to-run noise, because 49 to 86 percent of the new experience is byte-identical to the old — the policy returns to the same region.paper
VIII-cThat plateau was acquisition, not exhaustion: the same head form on states gathered where a second agent stood, or where the agent was made to look first, restores gains of VIII's size (35.4% and 41.2% against 26.8% from the agent's own experience and 22.0% alone) on 300 unseen problems. The second agent was the experiment, not the mechanism.paper

Figure 1 of Paper VIII: the quality–compute frontier, the paired effect everywhere it was measured, and the same-decision baselines

Figure 1 of VIII, rebuilt from the repository's own result files by scripts/fig1_frontier.py.

Published neighbours, as found

A first pass over about twenty-five abstracts, not a survey; the papers' §6a carries the reasoning. The closest objects to this series' instrument are OLIVIA (a frozen LLM's action choice as a linear bandit over its hidden state, learned online outside the model), HSRM (a small verifier-trained encoder over a frozen generator's hidden states) and Multi-Head Latent Control (control heads over frozen latent traces for a quality–cost trade-off); the closest to "selecting experience for a frozen agent" is ExpGraph (an external experience graph with utility-ranked retrieval); ReBalance shares the deep-state steering instrument for reasoning economy. The closest to the plateau is the echo-gap result on memory-based self-improvement; the closest to the lever are the scout paper (trained) and DORA (training-free); the baseline class is ExpeL. None of them separates using experience from acquiring it with the model and the mechanism held fixed. If you know work that does, that is the most valuable finding this page can receive: file it.

What you need

An Apple-silicon Mac with about 16 GB of free memory, Python 3.9 or later with mlx-lm (tested at 0.29.1 on mlx 0.29.3), and the repository. The model is Qwen/Qwen2.5-7B-Instruct in bf16, pulled on first use, about 15 GB. Nothing leaves your machine.

git clone https://github.com/LouayAlsakka/efficient-thinking && cd efficient-thinking

The commands

Each script prints what it expects beside what it got, refuses to print a table if any of its gates fails, and exits non-zero if the published result is not reproduced. Read the header of each before running it; the cost is stated there rather than discovered at hour nine.

PaperCommandWall clock
VIIIbash experience/reproduce_matched_compute.sh — the base given the head's compute against the head, n = 300, paired. An independent run on a second machine by someone who had not touched the work read +8.0 [+3.0, +13.3] against the published +9.3 [+3.7, +15.0].hours; see header
VIII-bbash experience/reproduce_8b_accumulation.sh — the second generation's rise and the third generation's stop, both checked; a reproduction that confirms only the rise has reproduced half a claim.hours; see header
VIII-c, our bytespython experience/g13b_score.py --dir experience/results/viiic --plan viiic7 --out /tmp/viiic.json — rescores all seven arms and 33 pairings from the 189 published episode files, named by hash in experience/results/viiic_LOCK_v3.json. Needs Python 3.9+ with numpy (pip install numpy); no model, no GPU.seconds
VIII-c, your bytesbash experience/reproduce_8c.sh — regenerates both task sets from their seeds and proves them disjoint by program signature, collects the base and forced-look arms, fits the head, plays three games on 300 unseen problems, and prints your green counts beside ours against the session's own 3.67-point bar. A fresh head will not hash-match ours; the claim is the counts.~9 h, measured
The from-scratch VIII-c script has completed one full run on a second machine (2026-09-27), and every arm came in within the bar. The run regenerates both task sets, collects states, fits the forced-look head from its own states (cross-validated 52.8% against the published 54.8%, permutation controls at chance), and plays three games per arm. Base 66 · 70 · 60 of 300 against the lock's 66 · 67 · 65 (means 21.78% vs 22.00%, 0.22 points apart); forced-look 126 · 121 · 116 against 122 · 126 · 123 (40.33% vs 41.22%, 0.89 apart); the effect re-derives at +18.55 points against the published +19.22. Every arm re-collects and re-plays, so the whole table is an independent re-draw judged against the session's 3.67-point bar, not against equality: the reproduced base arm alone spans 10 problems across its three games, which is the loop's own spread measured on another box. The falsifying window for the last game (91 to 157) was posted before its number (116) existed. The stage-6 output with each game's completion time is at experience/results/viiic_repro/2026-09-27_stage6_output.txt; the self-experience arm is skipped by the script because its head is a VIII-b artefact this repository does not publish. Wall clock about 4 h 20 min on an M-series Ultra.

Where to aim

The places the author thinks the results are weakest, stated by him so you do not have to guess. A finding that lands on one of these is expected; one that lands elsewhere is more valuable.

VIII

  1. An alternative explanation for the matched-compute frontier shift that is not the head.
  2. Leakage in how the experience was constructed: any path by which a scored problem's answer reached the head.
  3. A flaw in the paired statistics, in particular the treatment of the loop's run-to-run variance.
  4. A reason the second search structure (SQL repair) does not establish that the mechanism transfers.

VIII-b

  1. A reading of the second-generation gain that survives the harness's own null replicates.
  2. A duplication instrument under which the redundancy figures come out materially different.
  3. A same-family disjoint-problem arm that does show new experience helping, at the same decision count.
  4. Evidence that the plateau is the head form's, not the policy's.

VIII-c

  1. The acquisition rule on a second task family: if "look before you commit" during collection does not move a head there, the claim is one family's.
  2. The exchange rate: 132 forced-look groups matched 290 of the second agent's within noise; a measured ratio, in either direction, would change the paper's §5.
  3. The probe: the paper claims the held-out proxy tracks the amount of experience and not its origin; a proxy that recovers the game ordering across the four heads falsifies that.
  4. The noise floor: every comparison is read against the largest within-arm range of the session at n = 300; a demonstration that this bar is too permissive, or that session drift between the 09-25 bracket and the 09-26 arms is larger than the base's within-day stability, moves the interval counts.

How review works here

What should be tested next

The claim on this page has been tested one way; a theory survives by being tested from more directions than its author thought of. The experiments below are the ones the author intends to run next, registered in the series plan with each reading written before the run. Readers are asked for the ones the author has not thought of.

  1. The baseline VIII owes — done (VIII-e, 2026-10-01). Prompt retrieval of past successes at matched compute adds nothing and costs 5.9% more input; the head beats it by 13–15 points in every game. The value-function half is not yet run.
  2. Coverage or readability — done (VIII-d, 2026-10-01/02). Neither: the two selection rules do not differ within the design's ~5-point resolution, and a random draw from the same pool does as well as either. What mattered was which states were collected, not how they were selected afterwards.
  3. A second task family. The acquisition rule of VIII-c, "look before you commit" during collection, on SQL repair and then on a public verified family. If it does not move a head there, the claim is one family's.
  4. The exchange rate. How many forced-look groups equal how many second-agent groups; a measured ratio in either direction changes VIII-c §5.
  5. A third family — done (VIII-f, 2026-10-03). The head replicates on Yi-1.5-9B-Chat at 4-bit, +21 to +23 over base in three of three games. The claim now stands on three families; a fourth, and the quantisation check on family 1, are registered next.
  6. Steering built from the verifier's label — tested and stopped (VIII-g, 2026-10-03). The precondition named by the adaptive-steering authors is not met for this label at the candidate layers; the candidate ended at its first gate, published as a negative.
  7. A larger model. The same dissociation at 14B and 32B, to see whether the acquisition plateau and its remedy survive scale.

Propose one. If there is an experiment on these systems that would test the theory harder than the five above, say so as an issue: what to vary, what to hold fixed, and what result would count against the claim. The author selects from the proposals, registers the chosen ones with their readings written first, runs them on the pool of M3 Ultra Mac Studios that produced every result on this site, and publishes each result on this page either way, with the proposer credited as above. The machines are the author's and are not offered for outside use; what is offered is the run, and the record of it.

Findings so far

From the author's side: one reproduction (the VIII-c from-scratch run on a second machine, above, all arms within the bar), and since then five further measured arms on the plan — VIII-d, VIII-e, VIII-f, VIII-g, and VIII-h awaiting its second read — each registered before its run, each locked and hashed before scoring. None filed from outside yet. The series' own withdrawals are on the papers' pages and in the register, dated: a contaminated arm read 134 of 300 and was withdrawn; a bar computed without the arm under test moved a verdict from 2 of 3 to 1 of 3; a proxy ranked three heads backwards and was retired as a diagnostic. They are left visible because a challenge whose author hides his own failures is not one.

Author: the address on the front page · github.com/LouayAlsakka/efficient-thinking · Efficient Thinking