From 34734e1ab91253a33eddb1d6d77694a1367a7a9a Mon Sep 17 00:00:00 2001 From: CaptainJack2491 Date: Sat, 29 Aug 2026 23:13:40 +0100 Subject: refine Addendum 8: expand Arm D into D1 (learned), D2 (frozen), and D3 (dynamic noise) --- design/preregistration.md | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/design/preregistration.md b/design/preregistration.md index ad15994..c0294b2 100644 --- a/design/preregistration.md +++ b/design/preregistration.md @@ -328,14 +328,17 @@ cells, per user decision; protocol locked in Addendum 7 post-E6. --- -## Addendum 8 — E8: Token-Space Recurrence & Decomposition (Arms A, B, C, D) (locked 2026-08-29, pre-run) +## Addendum 8 — E8: Token-Space Recurrence & Decomposition (Arms A, B, C, D1, D2, D3) (locked 2026-08-29, pre-run) -**Question (Q3d / Q1):** Does expanding computation into token space—via explicit algorithmic decomposition (scratchpad), unsupervised compute positions (filler/pause tokens), or random sequence expansion—enable a fixed-depth Transformer to discover the next-prime algorithm ($O1$) where $4\text{M}$ steps of fixed-depth training ($E7$) failed? +**Question (Q3d / Q1):** Does expanding computation into token space—via explicit algorithmic decomposition (scratchpad), unsupervised compute positions (filler/pause tokens), or random sequence expansion (learned vs frozen vs dynamic noise)—enable a fixed-depth Transformer to discover the next-prime algorithm ($O1$) where $4\text{M}$ steps of fixed-depth training ($E7$) failed? **Background & Theoretical Predictions:** 1. **Algorithmic Decomposition (Nye et al. 2021):** Supervised intermediate computation provides dense gradient pathways through combinatorial search spaces. 2. **Parallel Subcomputation in Filler Tokens (Pfau et al. 2024):** Filler tokens ($\langle\text{pause}\rangle$) only aid computation when underlying subproblems are parallelizable. Next-prime requires trial division across primes $\{2, 3, 5, \dots, \sqrt{c}\}$, which are independent modular tests that can be evaluated in parallel across attention heads at intermediate positions. -3. **Sequence Length / Depth Invariance:** Testing whether random non-digit tokens (Arm D) provide equivalent routing capacity without static token embeddings. +3. **Discrete vs Continuous Unstructured Scratchpad (The D1/D2/D3 Sub-arms):** + - *Arm D1 (Learned discrete tokens):* Tokens `a-p` with standard learnable embeddings. Tests if the model develops token-specific routing biases. + - *Arm D2 (Frozen discrete embeddings):* Tokens `a-p` with fixed, non-trainable random Gaussian embeddings ($\text{requires\_grad}=\text{False}$). + - *Arm D3 (Continuous dynamic Gaussian noise):* Intermediate positions injected with fresh $\mathbf{z}_t \sim \mathcal{N}(0, I)$ vectors on every forward pass. Pure dynamic scratch real estate with zero static token bias. **Protocol:** - **Model:** GPT-style transformer (2 layers, 4 heads, `d_model=128`, `d_ff=512`), `wd=0.1`, `lr=1e-3` (const), `batch_size=32`, `max_train_steps=200_000`, `eval_every=200`. @@ -343,28 +346,32 @@ cells, per user decision; protocol locked in Addendum 7 post-E6. - **Probe:** Generalization evaluated on out-of-range $[1001, 2000]$. - **Hardware/Precision:** CUDA + AMP fp16 on ichi / voidlaptop (CPU fallback). -**The Four Experimental Arms:** +**The Experimental Arms:** 1. **Arm A (Direct Baseline / Control):** Standard mapping $n \to \text{next\_prime}(n) + \text{EOS}$. 2. **Arm B (Structured Scratchpad):** Explicit trial division trace: $n \to \text{candidate and divisor checks } (c = n+1, \dots; d_2:0/1, d_3:0/1, \dots) \to \text{SEP} \to \text{next\_prime}(n) + \text{EOS}$. -3. **Arm C (Fixed Pause / Filler Tokens):** $16$ identical pause tokens: +3. **Arm C (Fixed Pause / Filler Tokens):** $16$ identical learned pause tokens: $n \to \underbrace{\langle\text{pause}\rangle \dots \langle\text{pause}\rangle}_{16} \to \text{SEP} \to \text{next\_prime}(n) + \text{EOS}$. -4. **Arm D (Random Disjoint Tokens):** $16$ uniform random symbols from disjoint alphabet $[a\text{-}p]$: +4. **Arm D1 (Learned Random Tokens):** $16$ random discrete tokens from disjoint alphabet $[a\text{-}p]$ with learnable embeddings: $n \to \underbrace{r_1 r_2 \dots r_{16}}_{16} \to \text{SEP} \to \text{next\_prime}(n) + \text{EOS}$. +5. **Arm D2 (Frozen Random Tokens):** $16$ random discrete tokens from $[a\text{-}p]$ with frozen, fixed random Gaussian embeddings. +6. **Arm D3 (Continuous Dynamic Noise Vectors):** $16$ intermediate positions injected with dynamic $\mathbf{z}_t \sim \mathcal{N}(0, I)$ continuous noise vectors per batch. **Loss Masking & Evaluation Rules:** -- Loss computed over target answer tokens for all arms (and trace tokens for Arm B during teacher forcing). Filler/random tokens in Arms C/D are masked from cross-entropy loss. +- Loss computed over target answer tokens for all arms (and trace tokens for Arm B during teacher forcing). Filler/random/noise tokens in Arms C, D1, D2, D3 are masked from cross-entropy loss. - In-range validation exact-match (EM) and out-of-range probe accuracy are scored strictly on the final answer tokens (post-SEP / post-scratch). **Interpretation Matrix (Locked):** | Code | Observable Condition | Interpretation (Locked) | Next Step | |---|---|---|---| -| **S1** | Arm B shows $O1$ (sharp val EM $\ge 0.9$), while A, C, D remain $O\text{-PARTIAL}$ or $O4$ | **Decomposition-Gated:** Algorithmic grokking requires explicit trace supervision; unsupervised compute positions are insufficient to discover divisibility. | Sieve-rank probe on scratch outputs; inspect generated traces for correct divisibility. | +| **S1** | Arm B shows $O1$ (sharp val EM $\ge 0.9$), while A, C, D1-3 remain $O\text{-PARTIAL}$ or $O4$ | **Decomposition-Gated:** Algorithmic grokking requires explicit trace supervision; unsupervised compute positions are insufficient to discover divisibility. | Sieve-rank probe on scratch outputs; inspect generated traces for correct divisibility. | | **S2** | Arm C shows $O1$ or significant probe generalization ($P5(k)$ / $P6$ on $[1001, 2000]$) | **Pfau Hypothesis Confirmed:** Unsupervised filler tokens provide enough hidden attention routing to compute parallel divisibility tests without trace supervision. Major positive result. | Sieve-rank estimation; test filler length ablation ($M \in \{4, 8, 16, 32\}$). | -| **S3** | Arm D $\approx$ Arm C $\gg$ Arm A | **Depth/Position Invariance:** Sequence expansion enables attention routing independent of token identity. | Study attention maps across intermediate layers. | +| **S3a** | Arm D1 $\gg$ Arm D2 $\approx$ Arm D3 | **Token-Embedding Bias:** Performance in D1 was driven by learning specific token embedding biases, not pure depth expansion. | Probe embedding representations of $[a\text{-}p]$. | +| **S3b** | Arm D1 $\approx$ Arm D2 $\approx$ Arm D3 $\approx$ Arm C $\gg$ Arm A | **Pure Depth / Routing Invariance:** Mere expansion of sequence length / attention routing positions enables computation, independent of token identity or embedding trainability. | Study attention maps across intermediate layers. | +| **S3c** | Arm C $\gg$ Arm D1, D2, D3 | **Static Token Specificity:** Shared static pause embedding provides a coherent synchronization anchor that random noise destroys. | Analyze value vectors at pause positions. | | **S4** | Arm B in-range $\text{val EM} \approx 1.0$, but out-of-range probe $[1001, 2000]$ is $P4$ ($<10\%$) | **Bounded Trace Interpolation:** Scratchpad supervision fits training domain heuristics but does not induce generalizable out-of-distribution reasoning. | Test `is_prime` scratchpad to isolate divisibility vs candidate search. | -| **S5** | All arms A, B, C, D remain $O\text{-PARTIAL}$ / $O4$ ($P4$ out-of-range) | **Strongest Algorithmic Barrier:** Neither internal recurrence (RNN) nor externalized token recurrence (transformer scratch/filler) enables SGD to escape the memorization basin on next-prime. | Transition directly to Phase 2 (`is_prime` 4-arm battery). | +| **S5** | All arms A, B, C, D1-3 remain $O\text{-PARTIAL}$ / $O4$ ($P4$ out-of-range) | **Strongest Algorithmic Barrier:** Neither internal recurrence (RNN) nor externalized token recurrence (transformer scratch/filler/noise) enables SGD to escape the memorization basin on next-prime. | Transition directly to Phase 2 (`is_prime` 4-arm battery). | --- -- cgit v1.2.3