# E6 — dataset extension to [2, 1000] — analysis notes Sources: `runs/e6///seed/results.json` (eval run on ichi after each cell's training), `metrics.csv` (per-eval curves), and the locked interpretation clauses in `design/preregistration.md` **Addendum 6** (P-ladder / sieve-rank estimation) plus Addenda 1–4 for the unchanged O/H/P operationalizations. **Batch status at analysis time (2026-08-17 ~22:45 UTC): 8 of 16 cells finished, 4 running, 4 queued.** The sweep on ichi (`run_sweep`, concurrency 4) is still in its third batch (wd 1.0 seeds {1,2}); the fourth batch (wd 3.0 seed 0, batch-128 seed 0) has not started. This note classifies the 8 finished cells; the remaining 8 are reported as progress only and will be completed by the 08:30 safety-net cron. Constants used throughout: val EM is exact-match; "best" is the val-selected checkpoint (selection-holed), "last" is the unselected final checkpoint. **The E6 probe is exact-match on [1001, 2000]** (out of training range; range_end = 1000). The old [101, 200] probe is *in-range* for E6 and is reported only as an in-range generalization reference, NOT as the out-of-range probe. Probe was computed on voidlaptop (CPU) by loading `last.pt` with `map_location='cpu'` (the checkpoints were trained with CUDA + AMP on ichi; eval.py's own `_load` would try `device=cuda` and cannot run on this host). --- ## Per-cell classification (8 finished cells) | job | model | O | H | halt mean | val EM best | val EM last | train EM (max/final) | probe [1001,2000] acc | P | pred(960) | probe [101,200] (in-range) | |-----|-------|---|-----|-----------|-------------|-------------|----------------------|-----------------------|---|-----------|---------------------------| | e6-wd001 | rnn | O-PARTIAL | H1 | 2.00 | 0.783 | 0.770 | 1.000 / 1.000 | 0.040 (40/1000) | P4 | 967 | 0.94 | | e6-wd001 | transformer | O-PARTIAL | — | — | 0.840 | 0.803 | 1.000 / 1.000 | 0.007 (7/1000) | P4 | 967 | 0.91 | | e6-wd01 | rnn | O-PARTIAL | H1 | 2.00 | 0.830 | 0.807 | 1.000 / 1.000 | 0.049 (49/1000) | P4 | 967 | 0.90 | | e6-wd01 | transformer | O-PARTIAL | — | — | 0.827 | 0.807 | 1.000 / 1.000 | 0.007 (7/1000) | P4 | 967 | 0.92 | | e6-wd03 | rnn | O-PARTIAL | H1 | 2.03 | 0.823 | 0.787 | 1.000 / 1.000 | 0.039 (39/1000) | P4 | 967 | 0.95 | | e6-wd03 | transformer | O-PARTIAL | — | — | 0.840 | 0.787 | 1.000 / 1.000 | 0.008 (8/1000) | P4 | 967 | 0.92 | | e6-wd10 | rnn | O4 | H4 | 2.96 | 0.480 | 0.303 | 0.605 / 0.452 | 0.004 (4/1000) | P4 | 967 | 0.44 | | e6-wd10 | transformer | O4 | — | — | 0.797 | 0.677 | 0.987 / 0.890 | 0.004 (4/1000) | P4 | 967 | 0.73 | Progress (running / queued cells, at ~22:45 UTC): | cell | status | step | train EM | val EM | |------|--------|------|----------|--------| | e6-wd10 rnn s1 | running | 198000/200000 | 0.323 | 0.267 | | e6-wd10 rnn s2 | running | 187800 | 0.489 | 0.353 | | e6-wd10 transformer s1 | running | 180600 | 0.883 | 0.627 | | e6-wd10 transformer s2 | running | 177800 | 0.887 | 0.700 | | e6-wd30 rnn s0 | queued | — | — | — | | e6-wd30 transformer s0 | queued | — | — | — | | e6-b128 rnn s0 | queued | — | — | — | | e6-b128 transformer s0 | queued | — | — | — | --- ## O-codes — no O1 anywhere (no grokking) **LOCKED** (Addenda 1–4, unchanged): O1 = sharp transition after sustained train saturation; O-PARTIAL = train saturated, val never reaches 0.9, final val in (0.3, 0.9); O4 = train never sustained ≥ 0.95 for 10 evals (setup/optimization failure, no scientific reading). **Commentary.** The four low-wd cells (0.01, 0.1, 0.3 × both models) saturate train (train EM = 1.0, saturating from eval 2–63) and end with val EM 0.77–0.81 → **O-PARTIAL**. The two wd-1.0 cells are **O4**: the RNN's train EM never exceeds 0.605 (it cannot even memorize the range under wd 1.0), and the transformer peaks at 0.987 but never sustains ≥ 0.95 for 10 consecutive evals. **No cell shows the O1 grokking signature.** This is the same headline as phases 1–3: increasing the dataset 10× did not produce the delayed generalization jump. **Caveat on the transformer at wd 1.0.** It reaches a 0.797 val EM *best* checkpoint (in-range) yet is O4 by the locked definition (train never sustained saturation). Do not read its 0.677 final val as O-PARTIAL; the O4 branch fires first by construction. --- ## P-codes — all P4, and the sieve-rank ladder yields "no sieve" **LOCKED** (Addendum 6): probe range [1001, 2000]; P5(k) = errors concentrated on the rank-k signature set; P6 = exact (no probe misses); P3 threshold ≥ 90% on [1001, 2000]; P4/P2 keep their existing meanings. A model that internalized a k-prime sieve misses exactly the composites whose prime factors all exceed p_k; the smallest missed composite identifies k: 1147→k=10, 1369→k=11, 1681→k=12, 1849→k=13, none→k≥14. **Commentary.** Every finished cell is **P4**: probe accuracy on [1001, 2000] is 0.4–4.9% (4–49 of 1000), and 574–597 of the 600 trivial inputs (even or multiple of 5) are wrong — i.e. the models fail "the next prime after an even number is odd" 95–99.5% of the time. Out-of-range transfer is, as in phases 1–3, essentially zero. **Sieve-rank finding.** No cell internalized a k-prime sieve. The diagnostic is the *smallest composite the model ever outputs as "prime"* in the probe range; a genuine rank-k sieve's smallest such composite is one of {1147, 1369, 1681, 1849} (or none, for k ≥ 14). Every cell instead emits small composites with small prime factors — e.g. wd-10 transformer predicts 9, 49, 100, 110; wd-0.3 transformer predicts 49; wd-0.01 rnn predicts 100; the smallest *in-range* composite predictions are 1003 = 17·59, 1079 = 13·83, 1099 = 7·157, 1671 = 3·557. A model that outputs 3·557 or 7·157 as a prime has no divisibility structure at any rank. So the P-ladder resolves to **"no sieve" (k undefined) in all 8 cells** — P5(k) and P6 do not fire anywhere, and P3 (≥ 90%) is far from firing. **The 960→961 discriminator is uninformative here.** All 8 cells output **967** for n = 960 (the correct answer, which is the "k ≥ 11" behaviour). But 960 sits in the *training* split (699 train / 300 val, seed 0), so this is memorization of the 960→967 pair, not sieve transfer. A true k=10 sieve would output 961 (it cannot see 31|961); these models get 960 right *because they memorized it*, while simultaneously failing 95–99.5% of out-of-range inputs. The discriminator is only meaningful once a cell is already P5/P6; no cell is. **In-range reference.** The old [101, 200] probe — which is *in-range* for E6 — scores 0.90–0.95 at low wd (falling to 0.44–0.73 at wd 1.0). This is the same pattern phases 1–3 saw *on their out-of-range probe*: the model generalizes within its training distribution but transfers nothing beyond it. It is not a positive result, just a reminder that the correct E6 probe is [1001, 2000]. --- ## H-codes — halting still collapsed / noisy **LOCKED** (Addenda 1–4): H1 = mean steps at the min_steps floor (2); H2 = pinned at K; H3 = intermediate + positive gap correlation; H4 = intermediate but no correlation. **Commentary.** The three low-wd RNN cells are **H1** (mean 2.00–2.03, collapsed to the min_steps floor), and the wd-1.0 RNN is **H4** (mean 2.96, noisy). Same as every prior phase: the ACT gate never learned a computation budget. K = 32 here (vs 20 in phases 1–3) makes no difference — the gate still bottoms out at the floor or hovers unstructured. --- ## E6 verdict vs phases 1–3 **LOCKED** (Addendum 6): "an O1 anywhere = data pressure unlocked grokking; same codes = the task's walls are algorithmic, not data-bound." **Commentary.** No O1 anywhere, and out-of-range transfer is P4 in all 8 cells — so the "data pressure unlocked grokking" branch does **not** fire. The outcome *class* is unchanged: no grokking, zero out-of-range transfer, no divisibility algorithm. The one measurable shift is **in-range only**: the tied RNN, which was O4 everywhere at [2, 100] (val EM best ≤ 36.7%), is now O-PARTIAL at wd 0.01/0.1/0.3 (val EM best 0.78–0.83, final 0.77–0.81) — i.e. at 699 training examples the RNN finally holds an in-range memorized/heuristic solution it could not hold at 69. That is a real, reportable measurement, but it is not grokking and it does not transfer. Reading the locked clause strictly, the headline is the algorithmic-bound branch: **more data eased in-range fitting but did not move the task's walls — the search/increment loop and divisibility test remain unlearnable as transferable structure under these dynamics.** **Caveats.** The clause is binary ("O1 anywhere" vs "same codes"); the measured pattern is "no O1, and the in-range RNN O-code improved one notch". We report the O-code shift as a measurement rather than forcing it into either binary branch. Seed 0 only for the wd sweep; the wd-1.0 seeds {1,2} (running) will tell whether the in-range RNN improvement is seed-stable. --- ## Caveats / known gaps 1. **eval.py still hardcodes the [101, 200] probe.** Addendum 6 moves the E6 probe to [1001, 2000] and adds the P5(k)/P6 ladder, but `src/eval.py::probe_report` still defaults to `lo=101, hi=200` and only emits P1–P4. The [1001, 2000] numbers above were computed by a standalone probe (load `last.pt` on CPU, greedy-decode over [1001, 2000]). `eval.py` should be taught the E6 probe range + ladder before the remaining 8 cells are classified, otherwise their committed `results.json` will again carry the in-range [101, 200] code under the P field (harmless for O/H, misleading for P). 2. **`--post-only` cannot run on this host for E6.** The checkpoints are `device=cuda`; eval's `_load` honours `cfg.device='cuda'` and cannot run on the CPU-only voidlaptop. Re-scoring must happen on ichi (which the sweep already does automatically after each cell trains — the 8 finished cells were eval'd there). The `runs/e6/summary.csv` will therefore not exist until the sweep on ichi completes and is rsync'd back. 3. **fp16 numerics.** E6 ran with AMP fp16 on CUDA (a deliberate, recorded change vs the CPU fp32 baseline). All 8 cells sit far from any O/P/H boundary (e.g. probe acc 0.4–4.9% vs the 90% P3 line; val EM 0.77–0.81 vs the 0.9 O1 line), so the precision caveat in Addendum 6 does not bind for any finished cell. 4. **960 is in the training split**, so the 960→967 result is memorization, not sieve evidence (see P-codes section). --- ## E7 next step (per Addendum 6) Long-horizon test: **20× budget (4M steps)** on the best E6 cells, per jayrup's decision. The candidate cells are the low-wd O-PARTIAL cells that hold the strongest in-range solution — e6-wd01 / e6-wd03 (rnn and transformer, val EM best 0.83–0.84, final 0.79–0.81) — with the wd-1.0 control for parity. Protocol locked in Addendum 7 post-E6. The E6 read so far gives no reason to expect the 20× horizon to unlock transfer (the wall is algorithmic, not a data or budget shortfall at the margins tested), but the long horizon is exactly the test that separates "slow algorithmic basin" from "no basin at all".