E6 Phase 5: classify 8/16 finished cells (no O1, all P4, no sieve rank)

- Apply Addendum 6 P-ladder on [1001,2000] probe (computed on CPU; eval.py still hardcodes [101,200])
- wd 0.01/0.1/0.3: O-PARTIAL + P4 (val EM 0.77-0.84 in-range); wd 1.0: O4 + P4
- Sieve rank: undefined in all cells (smallest composite pred has small factors; no P5(k)/P6)
- 960->967 in all cells is training-split memorization, not sieve evidence
- Verdict: data pressure did not change outcome class; walls algorithmic, not data-bound
- 4 cells running (wd1.0 seeds 1,2), 4 queued (wd3.0, batch-128) -> 08:30 safety-net cron

1 files changed, 175 insertions, 0 deletions

diff --git a/NOTES-E6.md b/NOTES-E6.md
new file mode 100644
index 0000000..06bc685
--- /dev/null
+++ b/NOTES-E6.md
@@ -0,0 +1,175 @@
+# E6 — dataset extension to [2, 1000] — analysis notes
+
+Sources: `runs/e6/<job>/<model>/seed<seed>/results.json` (eval run on ichi after each
+cell's training), `metrics.csv` (per-eval curves), and the locked interpretation clauses in
+`design/preregistration.md` **Addendum 6** (P-ladder / sieve-rank estimation) plus Addenda 1–4
+for the unchanged O/H/P operationalizations.
+
+**Batch status at analysis time (2026-08-17 ~22:45 UTC): 8 of 16 cells finished, 4 running,
+4 queued.** The sweep on ichi (`run_sweep`, concurrency 4) is still in its third batch
+(wd 1.0 seeds {1,2}); the fourth batch (wd 3.0 seed 0, batch-128 seed 0) has not started.
+This note classifies the 8 finished cells; the remaining 8 are reported as progress only and
+will be completed by the 08:30 safety-net cron.
+
+Constants used throughout: val EM is exact-match; "best" is the val-selected checkpoint
+(selection-holed), "last" is the unselected final checkpoint. **The E6 probe is exact-match on
+[1001, 2000]** (out of training range; range_end = 1000). The old [101, 200] probe is *in-range*
+for E6 and is reported only as an in-range generalization reference, NOT as the out-of-range
+probe. Probe was computed on voidlaptop (CPU) by loading `last.pt` with `map_location='cpu'`
+(the checkpoints were trained with CUDA + AMP on ichi; eval.py's own `_load` would try
+`device=cuda` and cannot run on this host).
+
+---
+
+## Per-cell classification (8 finished cells)
+
+| job | model | O | H | halt mean | val EM best | val EM last | train EM (max/final) | probe [1001,2000] acc | P | pred(960) | probe [101,200] (in-range) |
+|-----|-------|---|-----|-----------|-------------|-------------|----------------------|-----------------------|---|-----------|---------------------------|
+| e6-wd001 | rnn | O-PARTIAL | H1 | 2.00 | 0.783 | 0.770 | 1.000 / 1.000 | 0.040 (40/1000) | P4 | 967 | 0.94 |
+| e6-wd001 | transformer | O-PARTIAL | — | — | 0.840 | 0.803 | 1.000 / 1.000 | 0.007 (7/1000) | P4 | 967 | 0.91 |
+| e6-wd01 | rnn | O-PARTIAL | H1 | 2.00 | 0.830 | 0.807 | 1.000 / 1.000 | 0.049 (49/1000) | P4 | 967 | 0.90 |
+| e6-wd01 | transformer | O-PARTIAL | — | — | 0.827 | 0.807 | 1.000 / 1.000 | 0.007 (7/1000) | P4 | 967 | 0.92 |
+| e6-wd03 | rnn | O-PARTIAL | H1 | 2.03 | 0.823 | 0.787 | 1.000 / 1.000 | 0.039 (39/1000) | P4 | 967 | 0.95 |
+| e6-wd03 | transformer | O-PARTIAL | — | — | 0.840 | 0.787 | 1.000 / 1.000 | 0.008 (8/1000) | P4 | 967 | 0.92 |
+| e6-wd10 | rnn | O4 | H4 | 2.96 | 0.480 | 0.303 | 0.605 / 0.452 | 0.004 (4/1000) | P4 | 967 | 0.44 |
+| e6-wd10 | transformer | O4 | — | — | 0.797 | 0.677 | 0.987 / 0.890 | 0.004 (4/1000) | P4 | 967 | 0.73 |
+
+Progress (running / queued cells, at ~22:45 UTC):
+
+| cell | status | step | train EM | val EM |
+|------|--------|------|----------|--------|
+| e6-wd10 rnn s1 | running | 198000/200000 | 0.323 | 0.267 |
+| e6-wd10 rnn s2 | running | 187800 | 0.489 | 0.353 |
+| e6-wd10 transformer s1 | running | 180600 | 0.883 | 0.627 |
+| e6-wd10 transformer s2 | running | 177800 | 0.887 | 0.700 |
+| e6-wd30 rnn s0 | queued | — | — | — |
+| e6-wd30 transformer s0 | queued | — | — | — |
+| e6-b128 rnn s0 | queued | — | — | — |
+| e6-b128 transformer s0 | queued | — | — | — |
+
+---
+
+## O-codes — no O1 anywhere (no grokking)
+
+**LOCKED** (Addenda 1–4, unchanged): O1 = sharp transition after sustained train saturation;
+O-PARTIAL = train saturated, val never reaches 0.9, final val in (0.3, 0.9); O4 = train never
+sustained ≥ 0.95 for 10 evals (setup/optimization failure, no scientific reading).
+
+**Commentary.** The four low-wd cells (0.01, 0.1, 0.3 × both models) saturate train
+(train EM = 1.0, saturating from eval 2–63) and end with val EM 0.77–0.81 → **O-PARTIAL**. The
+two wd-1.0 cells are **O4**: the RNN's train EM never exceeds 0.605 (it cannot even memorize
+the range under wd 1.0), and the transformer peaks at 0.987 but never sustains ≥ 0.95 for 10
+consecutive evals. **No cell shows the O1 grokking signature.** This is the same headline as
+phases 1–3: increasing the dataset 10× did not produce the delayed generalization jump.
+
+**Caveat on the transformer at wd 1.0.** It reaches a 0.797 val EM *best* checkpoint (in-range)
+yet is O4 by the locked definition (train never sustained saturation). Do not read its 0.677
+final val as O-PARTIAL; the O4 branch fires first by construction.
+
+---
+
+## P-codes — all P4, and the sieve-rank ladder yields "no sieve"
+
+**LOCKED** (Addendum 6): probe range [1001, 2000]; P5(k) = errors concentrated on the rank-k
+signature set; P6 = exact (no probe misses); P3 threshold ≥ 90% on [1001, 2000]; P4/P2 keep
+their existing meanings. A model that internalized a k-prime sieve misses exactly the composites
+whose prime factors all exceed p_k; the smallest missed composite identifies k: 1147→k=10,
+1369→k=11, 1681→k=12, 1849→k=13, none→k≥14.
+
+**Commentary.** Every finished cell is **P4**: probe accuracy on [1001, 2000] is 0.4–4.9%
+(4–49 of 1000), and 574–597 of the 600 trivial inputs (even or multiple of 5) are wrong —
+i.e. the models fail "the next prime after an even number is odd" 95–99.5% of the time.
+Out-of-range transfer is, as in phases 1–3, essentially zero.
+
+**Sieve-rank finding.** No cell internalized a k-prime sieve. The diagnostic is the *smallest
+composite the model ever outputs as "prime"* in the probe range; a genuine rank-k sieve's
+smallest such composite is one of {1147, 1369, 1681, 1849} (or none, for k ≥ 14). Every cell
+instead emits small composites with small prime factors — e.g. wd-10 transformer predicts 9, 49,
+100, 110; wd-0.3 transformer predicts 49; wd-0.01 rnn predicts 100; the smallest *in-range*
+composite predictions are 1003 = 17·59, 1079 = 13·83, 1099 = 7·157, 1671 = 3·557. A model that
+outputs 3·557 or 7·157 as a prime has no divisibility structure at any rank. So the P-ladder
+resolves to **"no sieve" (k undefined) in all 8 cells** — P5(k) and P6 do not fire anywhere, and
+P3 (≥ 90%) is far from firing.
+
+**The 960→961 discriminator is uninformative here.** All 8 cells output **967** for n = 960
+(the correct answer, which is the "k ≥ 11" behaviour). But 960 sits in the *training* split
+(699 train / 300 val, seed 0), so this is memorization of the 960→967 pair, not sieve transfer.
+A true k=10 sieve would output 961 (it cannot see 31|961); these models get 960 right *because
+they memorized it*, while simultaneously failing 95–99.5% of out-of-range inputs. The
+discriminator is only meaningful once a cell is already P5/P6; no cell is.
+
+**In-range reference.** The old [101, 200] probe — which is *in-range* for E6 — scores 0.90–0.95
+at low wd (falling to 0.44–0.73 at wd 1.0). This is the same pattern phases 1–3 saw *on their
+out-of-range probe*: the model generalizes within its training distribution but transfers
+nothing beyond it. It is not a positive result, just a reminder that the correct E6 probe is
+[1001, 2000].
+
+---
+
+## H-codes — halting still collapsed / noisy
+
+**LOCKED** (Addenda 1–4): H1 = mean steps at the min_steps floor (2); H2 = pinned at K; H3 =
+intermediate + positive gap correlation; H4 = intermediate but no correlation.
+
+**Commentary.** The three low-wd RNN cells are **H1** (mean 2.00–2.03, collapsed to the
+min_steps floor), and the wd-1.0 RNN is **H4** (mean 2.96, noisy). Same as every prior phase:
+the ACT gate never learned a computation budget. K = 32 here (vs 20 in phases 1–3) makes no
+difference — the gate still bottoms out at the floor or hovers unstructured.
+
+---
+
+## E6 verdict vs phases 1–3
+
+**LOCKED** (Addendum 6): "an O1 anywhere = data pressure unlocked grokking; same codes = the
+task's walls are algorithmic, not data-bound."
+
+**Commentary.** No O1 anywhere, and out-of-range transfer is P4 in all 8 cells — so the
+"data pressure unlocked grokking" branch does **not** fire. The outcome *class* is unchanged:
+no grokking, zero out-of-range transfer, no divisibility algorithm. The one measurable shift is
+**in-range only**: the tied RNN, which was O4 everywhere at [2, 100] (val EM best ≤ 36.7%), is
+now O-PARTIAL at wd 0.01/0.1/0.3 (val EM best 0.78–0.83, final 0.77–0.81) — i.e. at 699
+training examples the RNN finally holds an in-range memorized/heuristic solution it could not
+hold at 69. That is a real, reportable measurement, but it is not grokking and it does not
+transfer. Reading the locked clause strictly, the headline is the algorithmic-bound branch:
+**more data eased in-range fitting but did not move the task's walls — the search/increment
+loop and divisibility test remain unlearnable as transferable structure under these dynamics.**
+
+**Caveats.** The clause is binary ("O1 anywhere" vs "same codes"); the measured pattern is
+"no O1, and the in-range RNN O-code improved one notch". We report the O-code shift as a
+measurement rather than forcing it into either binary branch. Seed 0 only for the wd sweep;
+the wd-1.0 seeds {1,2} (running) will tell whether the in-range RNN improvement is seed-stable.
+
+---
+
+## Caveats / known gaps
+
+1. **eval.py still hardcodes the [101, 200] probe.** Addendum 6 moves the E6 probe to
+   [1001, 2000] and adds the P5(k)/P6 ladder, but `src/eval.py::probe_report` still defaults to
+   `lo=101, hi=200` and only emits P1–P4. The [1001, 2000] numbers above were computed by a
+   standalone probe (load `last.pt` on CPU, greedy-decode over [1001, 2000]). `eval.py` should
+   be taught the E6 probe range + ladder before the remaining 8 cells are classified, otherwise
+   their committed `results.json` will again carry the in-range [101, 200] code under the P
+   field (harmless for O/H, misleading for P).
+2. **`--post-only` cannot run on this host for E6.** The checkpoints are `device=cuda`; eval's
+   `_load` honours `cfg.device='cuda'` and cannot run on the CPU-only voidlaptop. Re-scoring must
+   happen on ichi (which the sweep already does automatically after each cell trains — the 8
+   finished cells were eval'd there). The `runs/e6/summary.csv` will therefore not exist until
+   the sweep on ichi completes and is rsync'd back.
+3. **fp16 numerics.** E6 ran with AMP fp16 on CUDA (a deliberate, recorded change vs the CPU
+   fp32 baseline). All 8 cells sit far from any O/P/H boundary (e.g. probe acc 0.4–4.9% vs the
+   90% P3 line; val EM 0.77–0.81 vs the 0.9 O1 line), so the precision caveat in Addendum 6 does
+   not bind for any finished cell.
+4. **960 is in the training split**, so the 960→967 result is memorization, not sieve evidence
+   (see P-codes section).
+
+---
+
+## E7 next step (per Addendum 6)
+
+Long-horizon test: **20× budget (4M steps)** on the best E6 cells, per jayrup's decision. The
+candidate cells are the low-wd O-PARTIAL cells that hold the strongest in-range solution —
+e6-wd01 / e6-wd03 (rnn and transformer, val EM best 0.83–0.84, final 0.79–0.81) — with the
+wd-1.0 control for parity. Protocol locked in Addendum 7 post-E6. The E6 read so far gives no
+reason to expect the 20× horizon to unlock transfer (the wall is algorithmic, not a data or
+budget shortfall at the margins tested), but the long horizon is exactly the test that separates
+"slow algorithmic basin" from "no basin at all".