refine Addendum 8: expand Arm D into D1 (learned), D2 (frozen), and D3 (dynamic noise)

1 files changed, 17 insertions, 10 deletions

diff --git a/design/preregistration.md b/design/preregistration.md
index ad15994..c0294b2 100644
--- a/design/preregistration.md
+++ b/design/preregistration.md
@@ -328,14 +328,17 @@ cells, per user decision; protocol locked in Addendum 7 post-E6.
 
 ---
 
-## Addendum 8 — E8: Token-Space Recurrence & Decomposition (Arms A, B, C, D) (locked 2026-08-29, pre-run)
+## Addendum 8 — E8: Token-Space Recurrence & Decomposition (Arms A, B, C, D1, D2, D3) (locked 2026-08-29, pre-run)
 
-**Question (Q3d / Q1):** Does expanding computation into token space—via explicit algorithmic decomposition (scratchpad), unsupervised compute positions (filler/pause tokens), or random sequence expansion—enable a fixed-depth Transformer to discover the next-prime algorithm ($O1$) where $4\text{M}$ steps of fixed-depth training ($E7$) failed?
+**Question (Q3d / Q1):** Does expanding computation into token space—via explicit algorithmic decomposition (scratchpad), unsupervised compute positions (filler/pause tokens), or random sequence expansion (learned vs frozen vs dynamic noise)—enable a fixed-depth Transformer to discover the next-prime algorithm ($O1$) where $4\text{M}$ steps of fixed-depth training ($E7$) failed?
 
 **Background & Theoretical Predictions:**
 1. **Algorithmic Decomposition (Nye et al. 2021):** Supervised intermediate computation provides dense gradient pathways through combinatorial search spaces.
 2. **Parallel Subcomputation in Filler Tokens (Pfau et al. 2024):** Filler tokens ($\langle\text{pause}\rangle$) only aid computation when underlying subproblems are parallelizable. Next-prime requires trial division across primes $\{2, 3, 5, \dots, \sqrt{c}\}$, which are independent modular tests that can be evaluated in parallel across attention heads at intermediate positions.
-3. **Sequence Length / Depth Invariance:** Testing whether random non-digit tokens (Arm D) provide equivalent routing capacity without static token embeddings.
+3. **Discrete vs Continuous Unstructured Scratchpad (The D1/D2/D3 Sub-arms):**
+   - *Arm D1 (Learned discrete tokens):* Tokens `a-p` with standard learnable embeddings. Tests if the model develops token-specific routing biases.
+   - *Arm D2 (Frozen discrete embeddings):* Tokens `a-p` with fixed, non-trainable random Gaussian embeddings ($\text{requires\_grad}=\text{False}$).
+   - *Arm D3 (Continuous dynamic Gaussian noise):* Intermediate positions injected with fresh $\mathbf{z}_t \sim \mathcal{N}(0, I)$ vectors on every forward pass. Pure dynamic scratch real estate with zero static token bias.
 
 **Protocol:**
 - **Model:** GPT-style transformer (2 layers, 4 heads, `d_model=128`, `d_ff=512`), `wd=0.1`, `lr=1e-3` (const), `batch_size=32`, `max_train_steps=200_000`, `eval_every=200`.
@@ -343,28 +346,32 @@ cells, per user decision; protocol locked in Addendum 7 post-E6.
 - **Probe:** Generalization evaluated on out-of-range $[1001, 2000]$.
 - **Hardware/Precision:** CUDA + AMP fp16 on ichi / voidlaptop (CPU fallback).
 
-**The Four Experimental Arms:**
+**The Experimental Arms:**
 1. **Arm A (Direct Baseline / Control):** Standard mapping $n \to \text{next\_prime}(n) + \text{EOS}$.
 2. **Arm B (Structured Scratchpad):** Explicit trial division trace:
    $n \to \text{candidate and divisor checks } (c = n+1, \dots; d_2:0/1, d_3:0/1, \dots) \to \text{SEP} \to \text{next\_prime}(n) + \text{EOS}$.
-3. **Arm C (Fixed Pause / Filler Tokens):** $16$ identical pause tokens:
+3. **Arm C (Fixed Pause / Filler Tokens):** $16$ identical learned pause tokens:
    $n \to \underbrace{\langle\text{pause}\rangle \dots \langle\text{pause}\rangle}_{16} \to \text{SEP} \to \text{next\_prime}(n) + \text{EOS}$.
-4. **Arm D (Random Disjoint Tokens):** $16$ uniform random symbols from disjoint alphabet $[a\text{-}p]$:
+4. **Arm D1 (Learned Random Tokens):** $16$ random discrete tokens from disjoint alphabet $[a\text{-}p]$ with learnable embeddings:
    $n \to \underbrace{r_1 r_2 \dots r_{16}}_{16} \to \text{SEP} \to \text{next\_prime}(n) + \text{EOS}$.
+5. **Arm D2 (Frozen Random Tokens):** $16$ random discrete tokens from $[a\text{-}p]$ with frozen, fixed random Gaussian embeddings.
+6. **Arm D3 (Continuous Dynamic Noise Vectors):** $16$ intermediate positions injected with dynamic $\mathbf{z}_t \sim \mathcal{N}(0, I)$ continuous noise vectors per batch.
 
 **Loss Masking & Evaluation Rules:**
-- Loss computed over target answer tokens for all arms (and trace tokens for Arm B during teacher forcing). Filler/random tokens in Arms C/D are masked from cross-entropy loss.
+- Loss computed over target answer tokens for all arms (and trace tokens for Arm B during teacher forcing). Filler/random/noise tokens in Arms C, D1, D2, D3 are masked from cross-entropy loss.
 - In-range validation exact-match (EM) and out-of-range probe accuracy are scored strictly on the final answer tokens (post-SEP / post-scratch).
 
 **Interpretation Matrix (Locked):**
 
 | Code | Observable Condition | Interpretation (Locked) | Next Step |
 |---|---|---|---|
-| **S1** | Arm B shows $O1$ (sharp val EM $\ge 0.9$), while A, C, D remain $O\text{-PARTIAL}$ or $O4$ | **Decomposition-Gated:** Algorithmic grokking requires explicit trace supervision; unsupervised compute positions are insufficient to discover divisibility. | Sieve-rank probe on scratch outputs; inspect generated traces for correct divisibility. |
+| **S1** | Arm B shows $O1$ (sharp val EM $\ge 0.9$), while A, C, D1-3 remain $O\text{-PARTIAL}$ or $O4$ | **Decomposition-Gated:** Algorithmic grokking requires explicit trace supervision; unsupervised compute positions are insufficient to discover divisibility. | Sieve-rank probe on scratch outputs; inspect generated traces for correct divisibility. |
 | **S2** | Arm C shows $O1$ or significant probe generalization ($P5(k)$ / $P6$ on $[1001, 2000]$) | **Pfau Hypothesis Confirmed:** Unsupervised filler tokens provide enough hidden attention routing to compute parallel divisibility tests without trace supervision. Major positive result. | Sieve-rank estimation; test filler length ablation ($M \in \{4, 8, 16, 32\}$). |
-| **S3** | Arm D $\approx$ Arm C $\gg$ Arm A | **Depth/Position Invariance:** Sequence expansion enables attention routing independent of token identity. | Study attention maps across intermediate layers. |
+| **S3a** | Arm D1 $\gg$ Arm D2 $\approx$ Arm D3 | **Token-Embedding Bias:** Performance in D1 was driven by learning specific token embedding biases, not pure depth expansion. | Probe embedding representations of $[a\text{-}p]$. |
+| **S3b** | Arm D1 $\approx$ Arm D2 $\approx$ Arm D3 $\approx$ Arm C $\gg$ Arm A | **Pure Depth / Routing Invariance:** Mere expansion of sequence length / attention routing positions enables computation, independent of token identity or embedding trainability. | Study attention maps across intermediate layers. |
+| **S3c** | Arm C $\gg$ Arm D1, D2, D3 | **Static Token Specificity:** Shared static pause embedding provides a coherent synchronization anchor that random noise destroys. | Analyze value vectors at pause positions. |
 | **S4** | Arm B in-range $\text{val EM} \approx 1.0$, but out-of-range probe $[1001, 2000]$ is $P4$ ($<10\%$) | **Bounded Trace Interpolation:** Scratchpad supervision fits training domain heuristics but does not induce generalizable out-of-distribution reasoning. | Test `is_prime` scratchpad to isolate divisibility vs candidate search. |
-| **S5** | All arms A, B, C, D remain $O\text{-PARTIAL}$ / $O4$ ($P4$ out-of-range) | **Strongest Algorithmic Barrier:** Neither internal recurrence (RNN) nor externalized token recurrence (transformer scratch/filler) enables SGD to escape the memorization basin on next-prime. | Transition directly to Phase 2 (`is_prime` 4-arm battery). |
+| **S5** | All arms A, B, C, D1-3 remain $O\text{-PARTIAL}$ / $O4$ ($P4$ out-of-range) | **Strongest Algorithmic Barrier:** Neither internal recurrence (RNN) nor externalized token recurrence (transformer scratch/filler/noise) enables SGD to escape the memorization basin on next-prime. | Transition directly to Phase 2 (`is_prime` 4-arm battery). |
 
 ---