1 files changed, 33 insertions, 25 deletions
diff --git a/NOTES-PHASE3.md b/NOTES-PHASE3.md
index b7b1631..5eea353 100644
--- a/NOTES-PHASE3.md
+++ b/NOTES-PHASE3.md
@@ -132,28 +132,36 @@ as a path to a grokking result on its own.
 place-value parsing was a real tax on learning. / within ±10 points → parsing was not the
 bottleneck; the algorithmic content is the wall."
 
-**Commentary.** **UNRESOLVED — evaluation crashed on all three integer-token runs**, so the
-locked ±10-point val-EM comparison cannot be scored. The three runs (`int` rnn, `int`
-transformer, `int01` rnn) all completed 200k training steps with `train_em 1.0`, then died in
-the evaluator with `IndexError: index out of range in self` inside `torch.nn.functional.embedding`
-(`rnn._encode` / `transformer.forward` → `self.embed`), reached from `probe_report` → `greedy_decode`.
-The integer-vocab token IDs exceed the embedding's vocab size of 103 (eos=102, pad=103). No
-`results.json` was written, so there is no clean best/last EM and no probe.
-
-Training-time val EM (from the logs) is the only signal, and it is **far below** the digits
-control rather than above it: best_val_em was 0.133 (`int` rnn), 0.100 (`int` transformer),
-0.067 (`int01` rnn), with val EM oscillating ~0.000–0.067 throughout. The digits-mode controls at
-the same wd scored 0.367 (rnn 1.0), 0.867 (transformer 1.0), 0.700 (rnn 0.1). So the integer
-tokens are ~23–80 points *below* the digits control — the opposite of the "parsing tax" lift.
-If this provisional training-time reading holds after the fix, D2 points at **parsing was not the
-bottleneck** (arguably integer tokens even hurt, e.g. by collapsing the halting gate to the floor,
-halt mean 2.00 in `int01`), but this is a bug report, not a scientific negative.
-
-**Caveats.** The crash is a real evaluator bug (integer vocab not wired through the probe/eval
-embedding), not a model failure; the training-time val_em numbers above are not the locked metric.
-Until the vocab bug is fixed and the runs re-evaluated, D2 contributes no clean measurement.
-
-**Implications for E6.** Fix the integer-vocab eval bug before any [2,1000] run that might want
-`vocab_mode=integers`; note that at range [2,1000] the inputs are 1–4 digits, so digits mode (with
-a widened embedding/position budget) is the natural default anyway. Do not carry the parsing-tax
-hypothesis into E6 as if confirmed — the provisional signal points the other way.
+**Commentary.** First evaluation crashed (out-of-vocab probe inputs: atomic tokens ≥ 104 exceed
+the vocab — out-of-range probing is unrepresentable by construction for integers mode). Fixed:
+probe is now N/A for integers mode (regression-tested); D2 is scored on in-range val EM, the
+only metric its locked clause uses.
+
+| job | model | wd | val EM best | val EM last | O | P | H | digits control (best/last, same wd) |
+|-----|-------|----|-------------|-------------|---|---|---|--------------------------------------|
+| int | rnn | 1.0 | 0.133 | 0.000 | O2 | N/A | H4 | 0.367 / 0.167 |
+| int | transformer | 1.0 | 0.100 | 0.067 | O2 | N/A | — | 0.867 / 0.700 |
+| int01 | rnn | 0.1 | 0.067 | 0.000 | O2 | N/A | H1 | 0.700 / 0.567 |
+
+All three cells are **20–77 points BELOW** the digits-mode control. Neither locked branch
+fires literally (the "within ±10 points" branch was written expecting a flat outcome; the
+measured outcome is a collapse). The measured fact, reported without a locked claim:
+**atomic integer tokens destroy the task** — every run is O2 (train memorizes, val ≈ 0),
+even the transformer that reaches 86.7% in digits mode.
+
+**Interpretive commentary (post-hoc, separated).** The digit representation was not a tax —
+it was the scaffolding. Compositionality is what let any generalization exist at all: in
+digits mode the model shares structure across "42", "43", "4", "3" (tens digit, units digit,
+position); in integers mode every number is a unique, once-seen token with nothing shared,
+so there is nothing to generalize — pure memorization, and even that barely holds under wd.
+This inverts the reviewer's hypothesis: removing the parsing overhead removed the substrate
+the in-range heuristic was built on.
+
+**Caveats.** Seed 0, three cells only. "Probe N/A" is a representational fact of the encoding,
+not a measurement failure. The integer-token task also changed the output distribution shape
+(2-token outputs) — the collapse is attributable to the input representation change, but that
+change bundles several things (no positional structure, unique tokens, shorter sequences).
+
+**Implications for E6.** Digits mode is not just the default — it is load-bearing. The [2,1000]
+phase should use digits (inputs are 1–4 digits there anyway) and treat integer tokens as a
+known-broken encoding, not a diagnostic toggle worth re-running.