summaryrefslogtreecommitdiff
path: root/design
diff options
context:
space:
mode:
Diffstat (limited to 'design')
-rw-r--r--design/preregistration.md44
1 files changed, 44 insertions, 0 deletions
diff --git a/design/preregistration.md b/design/preregistration.md
index 89e4fee..7efa8ad 100644
--- a/design/preregistration.md
+++ b/design/preregistration.md
@@ -217,3 +217,47 @@ with H4). Equal or worse → the halt gate was not the problem.
E3–E5 run after the E1/E2 gate, at jayrup's call. Range scaling ([2, 1000]) is its own
addendum when it becomes the active phase.
+
+---
+
+## Addendum 5 (2026-08-16, pre-launch — Phase 3 diagnostics batch)
+
+Trigger: external model feedback on follow-ups; adopted the integer-token and is-prime
+decomposition experiments. Locked before any of these runs. Implementation: `task_mode`
+(next_prime | is_prime), `train_frac`, `lr_decay` flags — 41 tests green.
+
+### D1 — Is-prime diagnostic (`task_mode=is_prime`, digits input, seed 0; wd 1.0 both models + wd 0.1 rnn/transformer)
+Binary classification n → {0,1}; output = digit token "1"/"0" + EOS. EM = classification accuracy.
+Probe = classification of [101, 200]. P-code ordering adapted for classification:
+P4 → **P1** → P3 → P2. (A pure {2,3,5,7} sieve scores ~96% on the probe because most of
+[101, 200] is trivially classifiable — that accuracy IS the sieve signature here, not a
+"surprising success". P1 fires on flagged-INPUT concentration: errors on {121, 143, 169, 187}
+classified prime. Verified against a ground-truth sieve stub: exactly those 4 errors → P1.)
+Locked decomposition readings:
+- **is_prime groks (O1) while next_prime never did** → the search/increment loop is the wall,
+ not the divisibility test.
+- **is_prime reproduces next_prime's codes (no O1 anywhere)** → the divisibility operation
+ itself is unlearnable under these dynamics — the strongest negative result available.
+- Any other pattern → report codes + measurements, no further claim.
+
+### D2 — Integer-token diagnostic (`vocab_mode=integers`, next_prime, seed 0; wd 1.0 both models + wd 0.1 rnn)
+Readings (locked):
+- val EM lifts ≥ +10 points over the digits-mode control at the same wd → place-value
+ parsing was a real tax on learning.
+- within ±10 points → parsing was not the bottleneck; the algorithmic content is the wall.
+
+### E3 implementation note
+Cosine schedule inside the step loop: lr_t = lr_min + 0.5·(lr − lr_min)·(1 + cos(π·step/max)),
+lr_min = 0.1·lr → 1e-3 → 1e-4. Interpretation per Addendum 4 E3, unchanged.
+
+### E4 wd choice
+Fixed-K (halting=False) at wd 0.1 (the RNN saturates train there — E2) and wd 1.0
+(control parity). Interpretation per Addendum 4 E4, unchanged.
+
+### E5 note
+`train_frac` ∈ {0.4, 0.5} subsamples the TRAIN split only (val stays the locked 30);
+deterministic stream (seed+1000); banker's rounding documented. Runs in the next batch;
+interpretation per Addendum 4 E5, unchanged.
+
+Batch = 11 runs: e4 (2), e3 (2), ints (3), isp (4). 2-way parallel, one thread per child,
+same eval cadence, metrics, and early stop as all previous phases.