summaryrefslogtreecommitdiff
path: root/design/preregistration.md
diff options
context:
space:
mode:
Diffstat (limited to 'design/preregistration.md')
-rw-r--r--design/preregistration.md50
1 files changed, 50 insertions, 0 deletions
diff --git a/design/preregistration.md b/design/preregistration.md
index 7efa8ad..07d0791 100644
--- a/design/preregistration.md
+++ b/design/preregistration.md
@@ -261,3 +261,53 @@ interpretation per Addendum 4 E5, unchanged.
Batch = 11 runs: e4 (2), e3 (2), ints (3), isp (4). 2-way parallel, one thread per child,
same eval cadence, metrics, and early stop as all previous phases.
+
+---
+
+## Addendum 6 — E6: dataset extension to [2, 1000] (locked 2026-08-17, pre-run)
+
+**Question (Q3c):** does increasing the dataset 10× (range_end 100 → 1000) change the outcome
+class of next-prime learning? Power et al. (2201.02177): "smaller datasets require increasing
+amounts of optimization for generalization" — the inverse prediction: more data should pull any
+grokking transition earlier. E6 is the SAME protocol as phases 1–3 (E1+E2 replication) with
+exactly one variable changed: range_end 100 → 1000.
+
+**Protocol (all cells):** digits mode, task next_prime, train_frac 0.7, lr 1e-3 const,
+max_train_steps 200_000, eval_every 200, ACT halting on (halt_penalty 0.01), d_model 128.
+**Changed vs phases 1–3 (recorded, unavoidable):**
+- `--max_steps 32` (K). Max prime gap below 1000 is 20 (887→907), so K=20 is the exact worst
+ case with zero margin. K=32 covers every in-range target. Rationale logged pre-run.
+- `--device cuda` + AMP fp16 autocast (T1000, ichi). fp16 numerics differ from the CPU fp32
+ baseline. The O/P/H classification is precision-robust; if any cell lands within ±2 points of
+ a classification boundary, the tiebreak re-run is fp32 on ichi CPU.
+- Outputs up to 4 digits: covered by max_out_len 6; global layout is range-derived (bisect +
+ margin code, tested to 100k).
+
+**Cells (16):**
+- wd sweep {0.01, 0.1, 0.3, 1.0, 3.0} × {rnn, transformer}, seed 0 (10)
+- wd 1.0 replication × seeds {1, 2} × {rnn, transformer} (4)
+- batch-size ablation: wd 1.0, seed 0, batch 128 × {rnn, transformer} (2)
+
+**Interpretation matrix — O/P/H codes unchanged (locked Addenda 1–4). Probe range is now
+[1001, 2000]. New P-ladder (sieve-rank estimation, replaces the single P1 signature):**
+- A model that internalized a k-prime sieve misses exactly the composites whose prime factors
+ all exceed p_k; the smallest missed composite in probe range identifies k unambiguously:
+ 1147→k=10, 1369→k=11, 1681→k=12, 1849→k=13, none→k≥14 (exact trial division to √n).
+- In-range discriminator: the 960→961 case (961=31² is the ONLY composite below 1000 with all
+ factors ≥ 31) separates k=10 from k=11.
+- P5(k) = errors concentrated on the rank-k signature set; P6 = exact (no probe misses).
+- P3 threshold for the [1001, 2000] probe: ≥90% (a rank-10 sieve scores ~97% there).
+- Locked reading: P5(k=10) would confirm the "first-10-primes heuristic" hypothesis; P6 would
+ show full divisibility transfer; P4/P2 keep their existing meanings.
+- E6 outcome vs phase 1–3: an O1 anywhere = data pressure unlocked grokking; same codes =
+ the task's walls are algorithmic, not data-bound.
+
+**Compute:** ichi (T1000 + i7-8700), tmux batch, run_sweep concurrency 4, GPU jobs only.
+Benchmarks pre-run: transformer 66 steps/s (AMP), rnn 31 steps/s → batch est. 2.5–4 h.
+
+**E7 (next, after E6 analysis):** long-horizon test — 20× budget (4M steps) on the best E6
+cells, per user decision; protocol locked in Addendum 7 post-E6.
+
+---
+
+*Publication note (2026-08-17): this file is mirrored verbatim from the private research repo (prime-grokking/preregistration.md) at the moment the experiment repo went public; hashes cited above are research-repo commits. The experiment repo commit history preserves each mirror step.*