phase3: task_mode (is_prime), train_frac, lr_decay flags + tests (41 green); Addendum 5 locked; 11-job batch

7 files changed, 176 insertions, 13 deletions

diff --git a/design/preregistration.md b/design/preregistration.md
index 89e4fee..7efa8ad 100644
--- a/design/preregistration.md
+++ b/design/preregistration.md
@@ -217,3 +217,47 @@ with H4). Equal or worse → the halt gate was not the problem.
 
 E3–E5 run after the E1/E2 gate, at jayrup's call. Range scaling ([2, 1000]) is its own
 addendum when it becomes the active phase.
+
+---
+
+## Addendum 5 (2026-08-16, pre-launch — Phase 3 diagnostics batch)
+
+Trigger: external model feedback on follow-ups; adopted the integer-token and is-prime
+decomposition experiments. Locked before any of these runs. Implementation: `task_mode`
+(next_prime | is_prime), `train_frac`, `lr_decay` flags — 41 tests green.
+
+### D1 — Is-prime diagnostic (`task_mode=is_prime`, digits input, seed 0; wd 1.0 both models + wd 0.1 rnn/transformer)
+Binary classification n → {0,1}; output = digit token "1"/"0" + EOS. EM = classification accuracy.
+Probe = classification of [101, 200]. P-code ordering adapted for classification:
+P4 → **P1** → P3 → P2. (A pure {2,3,5,7} sieve scores ~96% on the probe because most of
+[101, 200] is trivially classifiable — that accuracy IS the sieve signature here, not a
+"surprising success". P1 fires on flagged-INPUT concentration: errors on {121, 143, 169, 187}
+classified prime. Verified against a ground-truth sieve stub: exactly those 4 errors → P1.)
+Locked decomposition readings:
+- **is_prime groks (O1) while next_prime never did** → the search/increment loop is the wall,
+  not the divisibility test.
+- **is_prime reproduces next_prime's codes (no O1 anywhere)** → the divisibility operation
+  itself is unlearnable under these dynamics — the strongest negative result available.
+- Any other pattern → report codes + measurements, no further claim.
+
+### D2 — Integer-token diagnostic (`vocab_mode=integers`, next_prime, seed 0; wd 1.0 both models + wd 0.1 rnn)
+Readings (locked):
+- val EM lifts ≥ +10 points over the digits-mode control at the same wd → place-value
+  parsing was a real tax on learning.
+- within ±10 points → parsing was not the bottleneck; the algorithmic content is the wall.
+
+### E3 implementation note
+Cosine schedule inside the step loop: lr_t = lr_min + 0.5·(lr − lr_min)·(1 + cos(π·step/max)),
+lr_min = 0.1·lr → 1e-3 → 1e-4. Interpretation per Addendum 4 E3, unchanged.
+
+### E4 wd choice
+Fixed-K (halting=False) at wd 0.1 (the RNN saturates train there — E2) and wd 1.0
+(control parity). Interpretation per Addendum 4 E4, unchanged.
+
+### E5 note
+`train_frac` ∈ {0.4, 0.5} subsamples the TRAIN split only (val stays the locked 30);
+deterministic stream (seed+1000); banker's rounding documented. Runs in the next batch;
+interpretation per Addendum 4 E5, unchanged.
+
+Batch = 11 runs: e4 (2), e3 (2), ints (3), isp (4). 2-way parallel, one thread per child,
+same eval cadence, metrics, and early stop as all previous phases.
diff --git a/jobs/phase3.csv b/jobs/phase3.csv
new file mode 100644
index 0000000..51e4841
--- /dev/null
+++ b/jobs/phase3.csv
@@ -0,0 +1,12 @@
+job,model,seed,flags
+e4-nohalt01,rnn,0,--halting False --weight_decay 0.1
+e4-nohalt10,rnn,0,--halting False --weight_decay 1.0
+e3-cos,rnn,0,--lr_decay True
+e3-cos,transformer,0,--lr_decay True
+int,rnn,0,--vocab_mode integers
+int,transformer,0,--vocab_mode integers
+int01,rnn,0,--vocab_mode integers --weight_decay 0.1
+isp,rnn,0,--task_mode is_prime
+isp,transformer,0,--task_mode is_prime
+isp01,rnn,0,--task_mode is_prime --weight_decay 0.1
+isp01,transformer,0,--task_mode is_prime --weight_decay 0.1
diff --git a/src/config.py b/src/config.py
index 0b161e7..60984db 100644
--- a/src/config.py
+++ b/src/config.py
@@ -8,9 +8,11 @@ from dataclasses import dataclass, fields
 class Config:
     # data
     vocab_mode: str = "digits"       # "digits" | "integers"
+    task_mode: str = "next_prime"    # "next_prime" | "is_prime"
     range_start: int = 2
     range_end: int = 100             # inclusive
     holdout_frac: float = 0.30
+    train_frac: float = 1.0          # 1.0 = use full train split; 0.4/0.5 for E5
     seed: int = 0
     # model
     model: str = "rnn"               # "rnn" | "transformer"
@@ -25,6 +27,7 @@ class Config:
     n_heads: int = 4
     # training
     lr: float = 1e-3
+    lr_decay: bool = False           # cosine 1e-3 -> 1e-4 over the run (E3)
     weight_decay: float = 1.0
     max_train_steps: int = 200_000
     eval_every: int = 200
diff --git a/src/data.py b/src/data.py
index e6ea57b..fab4c21 100644
--- a/src/data.py
+++ b/src/data.py
@@ -44,21 +44,44 @@ def decode_tokens(ts, cfg: Config) -> int:
     return int("".join(map(str, digits))) if digits else -1
 
 
+def is_prime_n(n: int) -> bool:
+    """Exact primality for n >= 2."""
+    if n < 2:
+        return False
+    d = 2
+    while d * d <= n:
+        if n % d == 0:
+            return False
+        d += 1
+    return True
+
+
 def get_splits(cfg: Config) -> tuple[list[int], list[int]]:
-    """(train, val) input lists, seeded shuffle, no overlap."""
+    """(train, val) input lists, seeded shuffle, no overlap. train_frac subsamples the
+    TRAIN split only (E5); the val split is untouched (its size is locked by prereg)."""
     rng = random.Random(cfg.seed)
     inputs = list(range(cfg.range_start, cfg.range_end + 1))
     rng.shuffle(inputs)
     n_val = max(1, round(len(inputs) * cfg.holdout_frac))
-    return sorted(inputs[n_val:]), sorted(inputs[:n_val])
+    train, val = sorted(inputs[n_val:]), sorted(inputs[:n_val])
+    if cfg.train_frac < 1.0:
+        n_tr = max(1, round(len(train) * cfg.train_frac))
+        # deterministic subsample: seeded shuffle, take first n_tr
+        sub = random.Random(cfg.seed + 1000)   # distinct stream from split shuffle
+        sub.shuffle(train)
+        train = sorted(train[:n_tr])
+    return train, val
 
 
 def build_examples(inputs: list[int], cfg: Config) -> list[tuple[list[int], list[int]]]:
-    """[(input_tokens, target_tokens+EOS), ...]"""
+    """[(input_tokens, target_tokens+EOS), ...]. task_mode selects the target function."""
     primes = sieve_primes(cfg.range_end + 100)
     out = []
     for n in inputs:
-        p = next_prime(n, primes)
+        if cfg.task_mode == "is_prime":
+            p = 1 if is_prime_n(n) else 0
+        else:
+            p = next_prime(n, primes)
         out.append((encode_int(n, cfg), encode_int(p, cfg) + [cfg.eos_id]))
     return out
 
diff --git a/src/eval.py b/src/eval.py
index 50ec40e..c964d38 100644
--- a/src/eval.py
+++ b/src/eval.py
@@ -14,7 +14,7 @@ import numpy as np
 import torch
 
 from src.config import Config
-from src.data import build_examples, decode_tokens, encode_int, get_splits, next_prime, sieve_primes
+from src.data import build_examples, decode_tokens, encode_int, get_splits, is_prime_n, next_prime, sieve_primes
 from src.model_api import build_model, greedy_decode
 from src.train import evaluate
 
@@ -31,15 +31,21 @@ def probe_report(model, cfg: Config, lo: int = 101, hi: int = 200) -> dict:
     errors = []
     easy_total = 0
     easy_wrong = 0
+    is_prime_task = cfg.task_mode == "is_prime"
     for n in range(lo, hi + 1):
         x = torch.tensor(encode_int(n, cfg), dtype=torch.long).unsqueeze(0)
         gen = greedy_decode(model, x, cfg)[0].tolist()
         pred = decode_tokens(gen, cfg)
-        target = next_prime(n, primes)
-        is_easy = (n % 2 == 0) or (n % 5 == 0)   # trivial composites (skip-evens / skip-5s)
+        if is_prime_task:
+            target = 1 if is_prime_n(n) else 0
+            ok = (pred == 1) == (target == 1)      # any non-"1" output reads as "composite"
+        else:
+            target = next_prime(n, primes)
+            ok = pred == target
+        is_easy = (n % 2 == 0) or (n % 5 == 0)     # trivial composites (skip-evens / skip-5s)
         if is_easy:
             easy_total += 1
-        if pred == target:
+        if ok:
             correct += 1
         else:
             errors.append({"n": n, "target": target, "pred": pred})
@@ -47,16 +53,22 @@ def probe_report(model, cfg: Config, lo: int = 101, hi: int = 200) -> dict:
                 easy_wrong += 1
     total = hi - lo + 1
     acc = correct / total
-    flagged = [e for e in errors if e["pred"] in FLAGGED_SIEVE_PREDS]
-    distinct_flagged = len({e["pred"] for e in flagged})
+    if is_prime_task:
+        flagged = [e for e in errors if e["n"] in FLAGGED_SIEVE_PREDS]   # input IS the classified number
+        distinct_flagged = len({e["n"] for e in flagged})
+    else:
+        flagged = [e for e in errors if e["pred"] in FLAGGED_SIEVE_PREDS]
+        distinct_flagged = len({e["pred"] for e in flagged})
     flagged_frac = len(flagged) / len(errors) if errors else 0.0
-    # classification per prereg + Addendum 3 operationalization; P4 checked first
+    # classification per prereg + Addendum 3/5 operationalization
     if easy_total and easy_wrong / easy_total > 0.5:
         code = "P4"      # fails trivial evens/5-multiples -> pure memorization
+    elif is_prime_task and len(errors) >= 3 and distinct_flagged >= 3 and flagged_frac >= 0.8:
+        code = "P1"      # is-prime: errors concentrated on no-small-factor composites -> learned sieve
     elif acc >= 0.85:
-        code = "P3"      # surprising success beyond expectation
+        code = "P3"      # surprising success beyond expectation (next_prime semantics)
     elif len(errors) >= 3 and distinct_flagged >= 3 and flagged_frac >= 0.8:
-        code = "P1"      # errors = sieves predicting no-small-factor composites -> learned {2,3,5,7} sieve
+        code = "P1"      # next_prime: errors = sieves predicting no-small-factor composites
     else:
         code = "P2"      # scattered errors -> memorization / non-transferable heuristics
     return {
diff --git a/src/train.py b/src/train.py
index aa0fb4b..406defd 100644
--- a/src/train.py
+++ b/src/train.py
@@ -166,6 +166,11 @@ def main() -> None:
             halt = out["halt_steps"]
             penalty = halt.float().mean() * lam if halt is not None and lam > 0 else 0.0
             loss = loss_tokens + penalty
+            if cfg.lr_decay:
+                # cosine 1e-3 -> 1e-4 over the full run (E3, locked in Addendum 4)
+                frac = min(1.0, step / cfg.max_train_steps)
+                lr_t = cfg.lr * 0.1 + 0.5 * (cfg.lr - cfg.lr * 0.1) * (1 + math.cos(math.pi * frac))
+                opt.param_groups[0]["lr"] = lr_t
             opt.zero_grad()
             loss.backward()
             opt.step()
diff --git a/tests/test_phase3_flags.py b/tests/test_phase3_flags.py
new file mode 100644
index 0000000..4ba8be1
--- /dev/null
+++ b/tests/test_phase3_flags.py
@@ -0,0 +1,64 @@
+"""Tests for the Phase-3 flags: task_mode=is_prime, train_frac, lr_decay."""
+import torch
+
+from src.config import Config, parse_args
+from src.data import build_examples, decode_tokens, get_splits
+from src.eval import probe_report
+from src.model_api import build_model
+
+from tests.test_eval_classification import StubModel
+
+
+def _isprime_sieve(n):
+    """Primality via {2,3,5,7} with the d<c guard — EXACT on [2,100] (every composite <=100
+    has a prime factor <=10), wrong only on no-small-factor composites like 121."""
+    if n < 2:
+        return 0
+    return int(all(n % d != 0 for d in (2, 3, 5, 7) if d < n))
+
+
+def test_is_prime_examples():
+    cfg = Config(task_mode="is_prime")
+    ex = build_examples([2, 3, 4, 97, 100], cfg)
+    got = [decode_tokens(y, cfg) for _, y in ex]
+    assert got == [1, 1, 0, 1, 0], got
+
+
+def test_is_prime_sieve_stub_perfect_in_range():
+    """The {2,3,5,7} sieve is exact on [2,100] -> evaluate() must see EM 1.0."""
+    from src.train import evaluate
+    cfg = Config(task_mode="is_prime")
+    train_in, val_in = get_splits(cfg)
+    m = StubModel(_isprime_sieve)
+    for split in (train_in, val_in):
+        tok, em, _ = evaluate(m, build_examples(split, cfg), cfg)
+        assert em == 1.0 and tok == 1.0
+
+
+def test_is_prime_probe_classified_p1():
+    """On [101,200] the sieve errs exactly on {121,143,169,187} (classified prime) -> P1."""
+    cfg = Config(task_mode="is_prime")
+    r = probe_report(StubModel(_isprime_sieve), cfg)
+    assert sorted(e["n"] for e in r["errors"]) == [121, 143, 169, 187], r["errors"]
+    assert r["code"] == "P1", r["code"]
+
+
+def test_train_frac_subsamples_train_only():
+    cfg = Config(train_frac=0.5)
+    tr, va = get_splits(cfg)
+    assert len(va) == 30                      # val untouched (locked size)
+    assert len(tr) == 34                      # round(69*0.5) = 34 (banker's rounding)
+    assert not set(tr) & set(va)
+    # deterministic across calls
+    tr2, va2 = get_splits(Config(train_frac=0.5))
+    assert tr == tr2 and va == va2
+    # subsample is a subset of the full train split
+    tr_full, va_full = get_splits(Config(train_frac=1.0))
+    assert set(tr) <= set(tr_full)
+
+
+def test_new_flags_parse():
+    a = parse_args(["rnn", "0", "--task_mode", "is_prime", "--train_frac", "0.4", "--lr_decay", "True"])
+    assert a.task_mode == "is_prime" and a.train_frac == 0.4 and a.lr_decay is True
+    b = parse_args(["transformer", "0", "--lr_decay", "False"])
+    assert b.lr_decay is False and b.task_mode == "next_prime" and b.train_frac == 1.0