# Pre-registration: Prime-Grokking — Experiment 1 (seed 0) **Status:** PRE-REGISTERED before any training runs. **Date:** 2026-08-14 **Spec:** `prime-grokking/main.md` (research repo, commit 546dc2c) **Lock:** this file is committed to the research repo before experiment code runs; the commit hash is the lock. ## Experiment summary - **Task:** map n → next prime, range n ∈ [2, 100], 30% random holdout (seed 0), tokenized as decimal digits + EOS, teacher forcing on outputs. - **Arms:** (A) weight-tied 2-layer RNN cell, K = 20 tied steps, ACT learned halting (λ = 0.01, warmup 1000 steps); (B) GPT-style transformer baseline, d_model = 128, 2 layers, 4 heads. - **Comparison control:** fixed d_model = 128 for both arms. NO parameter-parity gate — weight sharing is the variable under study (param-matching a tied RNN against a GPT would kill the very property the spec hypothesizes). Param counts are logged per run for the record. - **Optimizer:** AdamW, lr = 1e-3, weight decay = 1.0 (sweep {0.3, 1.0, 3.0, 10.0} is a later experiment, not tuning on v1). Budget: 200k steps cap, eval every 200, early stop on val exact-match = 1.0 (patience 5). - **Primary metric:** per-example exact-match accuracy on held-out inputs (token accuracy reported alongside). Per-example correctness on 10 fixed val inputs logged every eval. Mean halt steps logged every eval (RNN arm). - **Probe:** generalization on [101, 200] is diagnostic-only (range extension is its own experiment). ## Pre-registered outcome → interpretation matrix ### In-range outcomes ([2, 100] held-out) | Code | Observable | Interpretation (locked) | Next step | |---|---|---|---| | **O1** | Sharp transition: train EM ≥ 0.95 sustained ≥ 10 evals, THEN val EM rises 0.2 → 0.9 within ≤ 5 evals | Grokking-like. Tied cell + heavy wd found a structured in-range solution; memorization repelled. NOT yet evidence of the full sieve — may be skip-evens/5s + divisibility heuristics. | Identify structure (per-example log, halting pattern); minimal-conditions ablations (halting=False, wd sweep, seeds) — jayrup's call. | | **O2** | Train EM → ~1.0, val EM stays low (≤ 0.3 at run end) | Memorization won. Lookup table is the lower-norm solution under these hyperparams — consistent with spec's "memorization is a stronger attractor". | wd sweep, smaller train fraction (40%/50%), longer budget — jayrup's call. | | **O3** | Gradual val rise to ≥ 0.9, no sharp transition | Smooth heuristic learning — NOT grokking by our operational definition. A "fast" generalizing solution exists that GD finds directly. | Distinguish from O1 by transition sharpness; report both. | | **O4** | Train EM never ≥ 0.95 within budget | Optimization/setup failure (lr, K, halting collapse, bug). NO scientific interpretation until fixed. | Inspect losses + halt steps; fix; rerun. | ### Architecture comparison (same hyperparams, seed 0) | Code | Observable | Interpretation (locked) | |---|---|---| | **O5** | RNN O1, transformer O2/O3 | Weight-tied recurrence is the enabling mechanism at this scale — evidence for the spec's core hypothesis (weight sharing makes the algorithmic solution cheaper). | | **O6** | Transformer O1, RNN O2/O3 | Recurrence + halting not necessary; fixed-depth transformer suffices in-range. Weight-tying not the key variable — depth/regularization is. | | **O7** | Both O1 | Grokking robust to architecture at this scale; the variable is data/regularization, not recurrence. | | **O8** | Neither | See O2/O4. Next-prime may be fundamentally harder than modular addition as the spec hypothesizes; wd/data sweeps decide. | Caveat: single seed — all architecture comparisons are seed-0 anecdotes until seeds {1, 2}. ### Halting structure (RNN arm only) | Code | Observable | Interpretation (locked) | |---|---|---| | **H1** | Mean steps collapse to 1 by run end | ACT failed (penalty/init issue); architecture conclusions read with a collapsed gate; fixed-K ablation becomes the informative run. | | **H2** | Mean steps pinned at K = 20 | Never learned to halt (penalty too weak). Same caveat as H1. | | **H3** | Mean steps intermediate (2–18), evolves during training, ideally correlates with gap-to-next-prime | Learned computation budget — evidence of structured algorithm. Check per-input steps on the 10 logged val examples. | | **H4** | Steps fluctuate noisily | Halt signal not used meaningfully. | ### Generalization probe [101, 200] (diagnostic-only) | Code | Observable | Interpretation (locked) | |---|---|---| | **P1** | Errors concentrated on {121, 143, 169, 187} — composites with factors 11, 13 (divisors beyond the {2, 3, 5, 7} sieve of the training range) | Definitive evidence of a learned sieve with the training-range divisor set. Strongest positive result available at this scale. | | **P2** | Errors scattered uniformly over composites | Memorization or non-transferable heuristics; no evidence of divisibility-based algorithm. | | **P3** | High probe accuracy beyond {121, 143, 169, 187} | Surprising — implies richer algorithm than the {2,3,5,7} sieve. Treat with suspicion; verify across seeds before claiming anything. | | **P4** | Probe fails on ALL of [101, 200] incl. easy evens / skip-5s | In-range solution didn't transfer even trivial heuristics — strong memorization evidence. | ### Operational definition of "grokking-like" (locked) - train exact-match ≥ 0.95 sustained for ≥ 10 consecutive evals (train saturated), AND - val exact-match transition from ≤ 0.2 to ≥ 0.9 within ≤ 5 consecutive evals (eval_every = 200). - eval order: evals happen every 200 steps; indices counted in evals, not steps. ## Statistical hygiene 1. Seed 0 for v1; seeds {1, 2} required before ANY claim beyond "seed-0 result". 2. No hyperparameter tuning on the val set. The wd sweep is a separate experiment run only after v1 results, at jayrup's call. 3. Probe interpretation locked above; NOTES.md must compare outcomes against this matrix verbatim (cite codes). 4. Training-range caveat recorded: for n ≤ 100 the sieve only needs divisors {2, 3, 5, 7}; "grokking the algorithm" in-range does not imply the general sieve. --- ## Addendum 1 (2026-08-14, pre-launch — setup amendments only) Trigger: Gemini 3.6 Flash design review (experiment repo `design/reviews/gemini-design-review.md`). The interpretation matrix (O/H/P codes) above is NOT amended; only setup details changed. Original lock commit: 00c696d. 1. **Fully-tied cell (was: un-tied GRU decoder).** All recurrence — input read-in, K compute steps, AND output-digit decoding — now runs through the SAME 2-layer cell. The earlier draft's GRU decoder would have masked whether the tied cell solved the task. RNN param count ≈ 36.6k (was 168.7k). Regression test added (no GRU/LSTM/RNN modules). 2. **Recurrent input read-in (was: masked mean-pool).** Digits are read through the tied cell with sinusoidal positional encoding. The mean-pool blurred place value ("10" and "100" share the token multiset {1,0}). 3. **λ schedule: linear ramp 1000→5000 steps (was: hard switch at step 1000).** Avoids a discontinuous loss jump late in training. 4. **Future wd sweep revised to {0.01, 0.1, 0.3, 1.0, 3.0}** (10.0 dropped: at lr=1e-3 with AdamW, λ=10 decays weights ~1%/step). Seed-0 default wd=1.0 unchanged. 5. **ACT verification note:** aggregation is Graves (2016) standard — w_t = p_t·Π_{s 50% of trivial-composite inputs (even or multiple of 5) wrong. ### Correctness fixes (codex review) 1. **Layout invariance (BLOCKER).** Inputs are now LEFT-padded to a fixed global length (3 digits for [2,100]) in EVERY context — training batches, eval, greedy decoding; outputs right-padded to the global length (4 incl. EOS). Previously batch-max padding made an example's representation depend on its batchmates (RNN pad steps transformed the state; transformer logit positions misaligned and absolute position embeddings shifted). RNN pad steps are now exact no-ops. Regression tests: mixed-batch vs singleton logit invariance for both models. 2. **min_steps off-by-one.** Halting probs forced to 0 only for the first `min_steps − 1` steps (was: first `min_steps`), so the earliest halt is step `min_steps` — matching "execute at least min_steps steps". 3. **integers vocab EOS alias.** Integers-mode vocab is now `next_prime(range_end) + 2`, so the value token 101 (= next_prime(100)) is NOT aliased with EOS. Boundary test added. 4. **Rerun protection.** train.py refuses to run if `metrics.csv` exists (append + header would corrupt provenance). Fresh `--out_dir` required per run. 5. **Checkpoint-selection honesty.** eval reports BOTH `best.pt` (val-selected — flagged as selection-holed) and `last.pt` (unselected); probe + halting analyses use `last.pt`. 6. **Dead config removed:** `halt_eps` (was never read). 7. **`run_meta.json`:** python/torch/numpy versions, device, thread count recorded per run. --- ## Addendum 3 (2026-08-14, pre-launch — P1 diagnostic corrected) Trigger: ad-hoc verification of `src/eval.py` against ground-truth stub models (promoted to `tests/test_eval_classification.py`). Pre-launch; P1 operationalization corrected before any results were seen. 1. **P1 concerns PREDICTIONS, not inputs.** A model that learned only the {2,3,5,7} sieve errs on inputs n = 113–120, 139–142, 167–168, 181–186, 199–200 — cases where the first candidate with no divisor ≤ 7 is composite. The signature is "predicted next prime" ∈ the flagged set, NOT "input n" ∈ the flagged set. Addendum 2's phrasing ("≥3 of {121,143,169,187} wrong") was ambiguous and the first implementation checked inputs — corrected. 2. **Set extended to {121, 143, 169, 187, 209}.** The original four are the no-small-factor composites ≤ 200; the probe's candidate window is actually [102, 211] (targets of n ∈ [101, 200] reach 211), and 209 = 11×19 is likewise mispredicted (n = 199, 200 → pred 209, target 211). 3. **P1 thresholds (replace Addendum 2's):** ≥ 3 errors total, ≥ 3 DISTINCT flagged predictions, ≥ 80% of error predictions flagged. A pure {2,3,5,7} sieve produces exactly 22 errors, 100% flagged (5 distinct values). P2–P4 unchanged. --- ## Addendum 4 (2026-08-14, pre-launch — Phase 2+ sweep plan) Precedes all sweep phases. Locked before any of these runs. Infrastructure changes recorded: `evaluate()` now batches greedy decoding (measured 46× eval speedup: 4.72 s → 0.10 s per eval cycle; semantics preserved — batch-invariance is enforced by regression tests, and a known-model sieve stub verifies exact-match counting). New runner: `scripts/run_sweep.py` (2-way parallel, one thread per child, fresh deterministic dirs, idempotent skip, `summary.csv` aggregation). All sweep runs: CPU, same metrics, early stop, eval cadence (every 200) and O/H/P operationalizations as Experiment 1. ### E1 — Seed replication (seeds {1, 2} × {rnn, transformer}; 4 runs) Identical hyperparameters to Experiment 1. - **"Replicated"** = same O-code and P-code across ≥ 2 of 3 seeds (0, 1, 2); RNN additionally same H-code across ≥ 2 of 3. - Transformer O-PARTIAL + P4 in ≥ 2 seeds → partial in-range generalization + zero transfer is seed-stable (a real claim). - RNN O4/H4 + P4 in ≥ 2 seeds → tied-cell instability + unstructured halting is seed-stable. - Any disagreement → report the per-seed pattern and claim nothing beyond it (high seed variance is itself a finding). ### E2 — Weight-decay sweep (wd ∈ {0.01, 0.1, 0.3, 1.0, 3.0} × both models, seed 0; wd=1.0 is the existing control) - **O1 at ≥ 1 wd value** → grokking exists for this task; the transition is regularization-sensitive; that (model, wd) cell becomes the study object for later phases. - **No O1 anywhere + val EM rises monotonically with wd** → the memorization attractor weakens with regularization, but no algorithmic basin was found at this scale. - **No O1 anywhere + val EM flat or declining with wd** → memorization strongly dominant; heavier wd only hurts. - **RNN never ≥ transformer at any wd** (same-seed val EM) → weight-tying + ACT fails to earn its keep at this scale (architecture-negative result). - Anything else → report per-cell codes + measurements, no further claim. ### E3 — Learning-rate annealing (both models, seed 0; cosine 1e-3 → 1e-4) Targets the late-run val-EM decay. Final val EM ABOVE the constant-lr control and no late-run decay → the rollover is optimizer dynamics. Identical or below → the decay is solution instability (more fundamental). ### E4 — halting=False ablation (RNN, seed 0) Fixed-K (K = 20) beats the ACT RNN (same seed, val EM) → ACT actively hurt it (consistent with H4). Equal or worse → the halt gate was not the problem. ### E5 — Training-fraction sweep {40%, 50%} (both models, seed 0) - **O1 at a smaller fraction** → grokking needs memorization to be only barely possible (the classic grokking recipe). - **Monotone val-EM gain as the fraction shrinks, no O1** → data pressure helps but does not produce the transition. - **No change** → memorization dominated at every data level tested. E3–E5 run after the E1/E2 gate, at jayrup's call. Range scaling ([2, 1000]) is its own addendum when it becomes the active phase. --- ## Addendum 5 (2026-08-16, pre-launch — Phase 3 diagnostics batch) Trigger: external model feedback on follow-ups; adopted the integer-token and is-prime decomposition experiments. Locked before any of these runs. Implementation: `task_mode` (next_prime | is_prime), `train_frac`, `lr_decay` flags — 41 tests green. ### D1 — Is-prime diagnostic (`task_mode=is_prime`, digits input, seed 0; wd 1.0 both models + wd 0.1 rnn/transformer) Binary classification n → {0,1}; output = digit token "1"/"0" + EOS. EM = classification accuracy. Probe = classification of [101, 200]. P-code ordering adapted for classification: P4 → **P1** → P3 → P2. (A pure {2,3,5,7} sieve scores ~96% on the probe because most of [101, 200] is trivially classifiable — that accuracy IS the sieve signature here, not a "surprising success". P1 fires on flagged-INPUT concentration: errors on {121, 143, 169, 187} classified prime. Verified against a ground-truth sieve stub: exactly those 4 errors → P1.) Locked decomposition readings: - **is_prime groks (O1) while next_prime never did** → the search/increment loop is the wall, not the divisibility test. - **is_prime reproduces next_prime's codes (no O1 anywhere)** → the divisibility operation itself is unlearnable under these dynamics — the strongest negative result available. - Any other pattern → report codes + measurements, no further claim. ### D2 — Integer-token diagnostic (`vocab_mode=integers`, next_prime, seed 0; wd 1.0 both models + wd 0.1 rnn) Readings (locked): - val EM lifts ≥ +10 points over the digits-mode control at the same wd → place-value parsing was a real tax on learning. - within ±10 points → parsing was not the bottleneck; the algorithmic content is the wall. ### E3 implementation note Cosine schedule inside the step loop: lr_t = lr_min + 0.5·(lr − lr_min)·(1 + cos(π·step/max)), lr_min = 0.1·lr → 1e-3 → 1e-4. Interpretation per Addendum 4 E3, unchanged. ### E4 wd choice Fixed-K (halting=False) at wd 0.1 (the RNN saturates train there — E2) and wd 1.0 (control parity). Interpretation per Addendum 4 E4, unchanged. ### E5 note `train_frac` ∈ {0.4, 0.5} subsamples the TRAIN split only (val stays the locked 30); deterministic stream (seed+1000); banker's rounding documented. Runs in the next batch; interpretation per Addendum 4 E5, unchanged. Batch = 11 runs: e4 (2), e3 (2), ints (3), isp (4). 2-way parallel, one thread per child, same eval cadence, metrics, and early stop as all previous phases. --- ## Addendum 6 — E6: dataset extension to [2, 1000] (locked 2026-08-17, pre-run) **Question (Q3c):** does increasing the dataset 10× (range_end 100 → 1000) change the outcome class of next-prime learning? Power et al. (2201.02177): "smaller datasets require increasing amounts of optimization for generalization" — the inverse prediction: more data should pull any grokking transition earlier. E6 is the SAME protocol as phases 1–3 (E1+E2 replication) with exactly one variable changed: range_end 100 → 1000. **Protocol (all cells):** digits mode, task next_prime, train_frac 0.7, lr 1e-3 const, max_train_steps 200_000, eval_every 200, ACT halting on (halt_penalty 0.01), d_model 128. **Changed vs phases 1–3 (recorded, unavoidable):** - `--max_steps 32` (K). Max prime gap below 1000 is 20 (887→907), so K=20 is the exact worst case with zero margin. K=32 covers every in-range target. Rationale logged pre-run. - `--device cuda` + AMP fp16 autocast (T1000, ichi). fp16 numerics differ from the CPU fp32 baseline. The O/P/H classification is precision-robust; if any cell lands within ±2 points of a classification boundary, the tiebreak re-run is fp32 on ichi CPU. - Outputs up to 4 digits: covered by max_out_len 6; global layout is range-derived (bisect + margin code, tested to 100k). **Cells (16):** - wd sweep {0.01, 0.1, 0.3, 1.0, 3.0} × {rnn, transformer}, seed 0 (10) - wd 1.0 replication × seeds {1, 2} × {rnn, transformer} (4) - batch-size ablation: wd 1.0, seed 0, batch 128 × {rnn, transformer} (2) **Interpretation matrix — O/P/H codes unchanged (locked Addenda 1–4). Probe range is now [1001, 2000]. New P-ladder (sieve-rank estimation, replaces the single P1 signature):** - A model that internalized a k-prime sieve misses exactly the composites whose prime factors all exceed p_k; the smallest missed composite in probe range identifies k unambiguously: 1147→k=10, 1369→k=11, 1681→k=12, 1849→k=13, none→k≥14 (exact trial division to √n). - In-range discriminator: the 960→961 case (961=31² is the ONLY composite below 1000 with all factors ≥ 31) separates k=10 from k=11. - P5(k) = errors concentrated on the rank-k signature set; P6 = exact (no probe misses). - P3 threshold for the [1001, 2000] probe: ≥90% (a rank-10 sieve scores ~97% there). - Locked reading: P5(k=10) would confirm the "first-10-primes heuristic" hypothesis; P6 would show full divisibility transfer; P4/P2 keep their existing meanings. - E6 outcome vs phase 1–3: an O1 anywhere = data pressure unlocked grokking; same codes = the task's walls are algorithmic, not data-bound. **Compute:** ichi (T1000 + i7-8700), tmux batch, run_sweep concurrency 4, GPU jobs only. Benchmarks pre-run: transformer 66 steps/s (AMP), rnn 31 steps/s → batch est. 2.5–4 h. **E7 (next, after E6 analysis):** long-horizon test — 20× budget (4M steps) on the best E6 cells, per user decision; protocol locked in Addendum 7 post-E6. --- ## Addendum 7 — E7: 20× Long-Horizon Budget Test (locked 2026-08-19, pre-run) **Question:** Is the failure of next-prime grokking in E6 an optimization horizon / compute-gated artifact ($H_\text{slow}$), or an algorithmic alignment / task-structure barrier? **Protocol (4 cells, seed 0):** - Candidates: `wd` $\in \{0.1, 0.3\} \times \{\text{rnn}, \text{transformer}\}$ (the $O\text{-PARTIAL}$ cells from E6). - Budget: $4\,000\,000$ steps ($20\times$ E6's 200k steps). - Schedule: Adaptive evaluation (530 evals total). - Range: $[2, 1000]$, `train_frac=0.7`, digits mode, $K=32$. - Probe: $[1001, 2000]$ (1000 out-of-range items). **Interpretation matrix (locked):** 1. **O1 anywhere:** GROKKING CONFIRMED on next-prime: transition exists, was budget-limited at 200k. Record transition step. 2. **O-PARTIAL with `val_em_best > parent best` AND sustained $\ge 50\text{k}$ steps after the parent's entire budget:** Partial progress toward the algorithmic basin; report as slow-basin dynamics, NOT grokking. 3. **Same codes as E6 parents (O-PARTIAL/O4, P4/P2, no P5(k$\ge$10)/P6):** $H_\text{slow}$ REJECTED for these cells at 4M steps: the null is now budget-robust to $20\times$. Surviving hypotheses become task-structure-gated (algorithmic alignment), not compute-gated. --- ## Addendum 8 — E8: Token-Space Recurrence & Decomposition (Arms A, B, C, D1, D2, D3) (locked 2026-08-29, pre-run) **Question (Q3d / Q1):** Does expanding computation into token space—via explicit algorithmic decomposition (scratchpad), unsupervised compute positions (filler/pause tokens), or random sequence expansion (learned vs frozen vs dynamic noise)—enable a fixed-depth Transformer to discover the next-prime algorithm ($O1$) where $4\text{M}$ steps of fixed-depth training ($E7$) failed? **Background & Theoretical Predictions:** 1. **Algorithmic Decomposition (Nye et al. 2021):** Supervised intermediate computation provides dense gradient pathways through combinatorial search spaces. 2. **Parallel Subcomputation in Filler Tokens (Pfau et al. 2024):** Filler tokens ($\langle\text{pause}\rangle$) only aid computation when underlying subproblems are parallelizable. Next-prime requires trial division across primes $\{2, 3, 5, \dots, \sqrt{c}\}$, which are independent modular tests that can be evaluated in parallel across attention heads at intermediate positions. 3. **Discrete vs Continuous Unstructured Scratchpad (The D1/D2/D3 Sub-arms):** - *Arm D1 (Learned discrete tokens):* Tokens `a-p` with standard learnable embeddings. Tests if the model develops token-specific routing biases. - *Arm D2 (Frozen discrete embeddings):* Tokens `a-p` with fixed, non-trainable random Gaussian embeddings ($\text{requires\_grad}=\text{False}$). - *Arm D3 (Continuous dynamic Gaussian noise):* Intermediate positions injected with fresh $\mathbf{z}_t \sim \mathcal{N}(0, I)$ vectors on every forward pass. Pure dynamic scratch real estate with zero static token bias. **Protocol:** - **Model:** GPT-style transformer (2 layers, 4 heads, `d_model=128`, `d_ff=512`), `wd=0.1`, `lr=1e-3` (const), `batch_size=32`, `max_train_steps=200_000`, `eval_every=200`. - **Dataset:** Range $[2, 1000]$, `train_frac=0.7`, digits mode, seed 0. - **Probe:** Generalization evaluated on out-of-range $[1001, 2000]$. - **Hardware/Precision:** CUDA + AMP fp16 on ichi / voidlaptop (CPU fallback). **The Experimental Arms:** 1. **Arm A (Direct Baseline / Control):** Standard mapping $n \to \text{next\_prime}(n) + \text{EOS}$. 2. **Arm B (Structured Scratchpad):** Explicit trial division trace: $n \to \text{candidate and divisor checks } (c = n+1, \dots; d_2:0/1, d_3:0/1, \dots) \to \text{SEP} \to \text{next\_prime}(n) + \text{EOS}$. 3. **Arm C (Fixed Pause / Filler Tokens):** $16$ identical learned pause tokens: $n \to \underbrace{\langle\text{pause}\rangle \dots \langle\text{pause}\rangle}_{16} \to \text{SEP} \to \text{next\_prime}(n) + \text{EOS}$. 4. **Arm D1 (Learned Random Tokens):** $16$ random discrete tokens from disjoint alphabet $[a\text{-}p]$ with learnable embeddings: $n \to \underbrace{r_1 r_2 \dots r_{16}}_{16} \to \text{SEP} \to \text{next\_prime}(n) + \text{EOS}$. 5. **Arm D2 (Frozen Random Tokens):** $16$ random discrete tokens from $[a\text{-}p]$ with frozen, fixed random Gaussian embeddings. 6. **Arm D3 (Continuous Dynamic Noise Vectors):** $16$ intermediate positions injected with dynamic $\mathbf{z}_t \sim \mathcal{N}(0, I)$ continuous noise vectors per batch. **Loss Masking & Evaluation Rules:** - Loss computed over target answer tokens for all arms (and trace tokens for Arm B during teacher forcing). Filler/random/noise tokens in Arms C, D1, D2, D3 are masked from cross-entropy loss. - In-range validation exact-match (EM) and out-of-range probe accuracy are scored strictly on the final answer tokens (post-SEP / post-scratch). **Interpretation Matrix (Locked):** | Code | Observable Condition | Interpretation (Locked) | Next Step | |---|---|---|---| | **S1** | Arm B shows $O1$ (sharp val EM $\ge 0.9$), while A, C, D1-3 remain $O\text{-PARTIAL}$ or $O4$ | **Decomposition-Gated:** Algorithmic grokking requires explicit trace supervision; unsupervised compute positions are insufficient to discover divisibility. | Sieve-rank probe on scratch outputs; inspect generated traces for correct divisibility. | | **S2** | Arm C shows $O1$ or significant probe generalization ($P5(k)$ / $P6$ on $[1001, 2000]$) | **Pfau Hypothesis Confirmed:** Unsupervised filler tokens provide enough hidden attention routing to compute parallel divisibility tests without trace supervision. Major positive result. | Sieve-rank estimation; test filler length ablation ($M \in \{4, 8, 16, 32\}$). | | **S3a** | Arm D1 $\gg$ Arm D2 $\approx$ Arm D3 | **Token-Embedding Bias:** Performance in D1 was driven by learning specific token embedding biases, not pure depth expansion. | Probe embedding representations of $[a\text{-}p]$. | | **S3b** | Arm D1 $\approx$ Arm D2 $\approx$ Arm D3 $\approx$ Arm C $\gg$ Arm A | **Pure Depth / Routing Invariance:** Mere expansion of sequence length / attention routing positions enables computation, independent of token identity or embedding trainability. | Study attention maps across intermediate layers. | | **S3c** | Arm C $\gg$ Arm D1, D2, D3 | **Static Token Specificity:** Shared static pause embedding provides a coherent synchronization anchor that random noise destroys. | Analyze value vectors at pause positions. | | **S4** | Arm B in-range $\text{val EM} \approx 1.0$, but out-of-range probe $[1001, 2000]$ is $P4$ ($<10\%$) | **Bounded Trace Interpolation:** Scratchpad supervision fits training domain heuristics but does not induce generalizable out-of-distribution reasoning. | Test `is_prime` scratchpad to isolate divisibility vs candidate search. | | **S5** | All arms A, B, C, D1-3 remain $O\text{-PARTIAL}$ / $O4$ ($P4$ out-of-range) | **Strongest Algorithmic Barrier:** Neither internal recurrence (RNN) nor externalized token recurrence (transformer scratch/filler/noise) enables SGD to escape the memorization basin on next-prime. | Transition directly to Phase 2 (`is_prime` 4-arm battery). | --- *Publication note (2026-08-17): this file is mirrored verbatim from the private research repo (prime-grokking/preregistration.md) at the moment the experiment repo went public; hashes cited above are research-repo commits. The experiment repo commit history preserves each mirror step.*