diff options
| author | Void Agent <void@jayrup.hermes> | 2026-08-17 16:57:19 +0100 |
|---|---|---|
| committer | Void Agent <void@jayrup.hermes> | 2026-08-17 16:57:19 +0100 |
| commit | 604072df642b5789cfaf15483dc2aad3fca4f57b (patch) | |
| tree | 52590fd4d5e0610254c0f7f474a4d0a30f1b0388 /design/preregistration.md | |
| parent | 6c3788e008a2d5ab624b5eb752af1d553b09724a (diff) | |
public release prep: full preregistration mirror (addenda 1-6), public README, track per-phase summary CSVs
Diffstat (limited to 'design/preregistration.md')
| -rw-r--r-- | design/preregistration.md | 50 |
1 files changed, 50 insertions, 0 deletions
diff --git a/design/preregistration.md b/design/preregistration.md index 7efa8ad..07d0791 100644 --- a/design/preregistration.md +++ b/design/preregistration.md @@ -261,3 +261,53 @@ interpretation per Addendum 4 E5, unchanged. Batch = 11 runs: e4 (2), e3 (2), ints (3), isp (4). 2-way parallel, one thread per child, same eval cadence, metrics, and early stop as all previous phases. + +--- + +## Addendum 6 — E6: dataset extension to [2, 1000] (locked 2026-08-17, pre-run) + +**Question (Q3c):** does increasing the dataset 10× (range_end 100 → 1000) change the outcome +class of next-prime learning? Power et al. (2201.02177): "smaller datasets require increasing +amounts of optimization for generalization" — the inverse prediction: more data should pull any +grokking transition earlier. E6 is the SAME protocol as phases 1–3 (E1+E2 replication) with +exactly one variable changed: range_end 100 → 1000. + +**Protocol (all cells):** digits mode, task next_prime, train_frac 0.7, lr 1e-3 const, +max_train_steps 200_000, eval_every 200, ACT halting on (halt_penalty 0.01), d_model 128. +**Changed vs phases 1–3 (recorded, unavoidable):** +- `--max_steps 32` (K). Max prime gap below 1000 is 20 (887→907), so K=20 is the exact worst + case with zero margin. K=32 covers every in-range target. Rationale logged pre-run. +- `--device cuda` + AMP fp16 autocast (T1000, ichi). fp16 numerics differ from the CPU fp32 + baseline. The O/P/H classification is precision-robust; if any cell lands within ±2 points of + a classification boundary, the tiebreak re-run is fp32 on ichi CPU. +- Outputs up to 4 digits: covered by max_out_len 6; global layout is range-derived (bisect + + margin code, tested to 100k). + +**Cells (16):** +- wd sweep {0.01, 0.1, 0.3, 1.0, 3.0} × {rnn, transformer}, seed 0 (10) +- wd 1.0 replication × seeds {1, 2} × {rnn, transformer} (4) +- batch-size ablation: wd 1.0, seed 0, batch 128 × {rnn, transformer} (2) + +**Interpretation matrix — O/P/H codes unchanged (locked Addenda 1–4). Probe range is now +[1001, 2000]. New P-ladder (sieve-rank estimation, replaces the single P1 signature):** +- A model that internalized a k-prime sieve misses exactly the composites whose prime factors + all exceed p_k; the smallest missed composite in probe range identifies k unambiguously: + 1147→k=10, 1369→k=11, 1681→k=12, 1849→k=13, none→k≥14 (exact trial division to √n). +- In-range discriminator: the 960→961 case (961=31² is the ONLY composite below 1000 with all + factors ≥ 31) separates k=10 from k=11. +- P5(k) = errors concentrated on the rank-k signature set; P6 = exact (no probe misses). +- P3 threshold for the [1001, 2000] probe: ≥90% (a rank-10 sieve scores ~97% there). +- Locked reading: P5(k=10) would confirm the "first-10-primes heuristic" hypothesis; P6 would + show full divisibility transfer; P4/P2 keep their existing meanings. +- E6 outcome vs phase 1–3: an O1 anywhere = data pressure unlocked grokking; same codes = + the task's walls are algorithmic, not data-bound. + +**Compute:** ichi (T1000 + i7-8700), tmux batch, run_sweep concurrency 4, GPU jobs only. +Benchmarks pre-run: transformer 66 steps/s (AMP), rnn 31 steps/s → batch est. 2.5–4 h. + +**E7 (next, after E6 analysis):** long-horizon test — 20× budget (4M steps) on the best E6 +cells, per user decision; protocol locked in Addendum 7 post-E6. + +--- + +*Publication note (2026-08-17): this file is mirrored verbatim from the private research repo (prime-grokking/preregistration.md) at the moment the experiment repo went public; hashes cited above are research-repo commits. The experiment repo commit history preserves each mirror step.* |
