Address Gemini+Opus repo reviews: W_U decomposition, Spearman+p-values, synthetic-pair CIs + corruption caveat, accurate fact-check, softened burden-of-proof, LayerNorm note, reviews caveat

1 files changed, 39 insertions, 10 deletions

diff --git a/results.md b/results.md
index 27b4005..4c1b604 100644
--- a/results.md
+++ b/results.md
@@ -10,18 +10,39 @@ identity, and the check returns cosine similarity **1.0000**.
 
 Pearson r between token frequency and J-lens norm, per layer. n = 65 tokens
 (full char vocab). The correlation survives the faithful implementation at
-every layer.
+every layer. p-values: all layers p ~ 10^-9 to 10^-13. Spearman rank
+correlation is even stronger: faithful -0.69 to -0.85; proxy -0.63 to -0.81
+(not a leverage artifact of extreme common tokens).
 
 ```
-  Layer   proxy r   faithful r
-  L0      -0.661    -0.643
-  L1      -0.673    -0.668
-  L2      -0.653    -0.672
-  L3      -0.648    -0.685
-  L4      -0.562    -0.637
-  L5      -0.665    -0.606
+  Layer   proxy r   faithful r   Spearman(faith)   partial r(faith, freq | ||W_U||)
+  L0      -0.661    -0.643        -0.844            -0.307
+  L1      -0.673    -0.668        -0.851            -0.319
+  L2      -0.653    -0.672        -0.825            -0.338
+  L3      -0.648    -0.685        -0.769            -0.364
+  L4      -0.562    -0.637        -0.716            -0.240
+  L5      -0.665    -0.606        -0.808            +0.056
 ```
 
+### W_U decomposition (where does the correlation come from?)
+
+The faithful lens vector for token k is `W_U[k] * J_l`. The unembedding row
+norms themselves anti-correlate with frequency:
+
+```
+  r(||W_U[k]||, freq)        = -0.606
+  r(||W_U[k]||, log10 freq)  = -0.693
+  Spearman(||W_U[k]||, freq) = -0.808
+```
+
+`r(faithful_norm, ||W_U[k]||)` per layer: +0.70 to +1.00 (L5 = +1.00 exactly,
+since J = identity at the last layer). After regressing out the W_U row-norm
+component, a layer-dependent anti-correlation survives in layers 0-4 (partial
+r ≈ -0.24 to -0.36) and vanishes at the last layer (+0.06). Interpretation:
+the frequency signal lives partly in the static unembedding geometry (which is
+baked into Anthropic's lens by definition) and partly in layer-dependent
+dynamics; the mechanism of the latter is not yet pinned down.
+
 Top tokens by faithful norm are consistently rare characters (`?`, `z`, `q`,
 `$`); bottom are common ones (space, `e`, `t`, `i`).
 
@@ -40,8 +61,16 @@ over layers 0-5:
 ```
 
 Reading: at equal frequency, the structured token scores ~1.4-1.5x higher.
-The frequency anti-correlation holds, but the lens also carries genuine
-conditional-predictability signal.
+Middle-layer ratio across seeds: 1.47 +/- 0.09 (SD), bootstrap 95% CI
+[1.37, 1.53]. The frequency anti-correlation holds, but the lens also carries
+genuine conditional-predictability signal.
+
+CAVEAT (from adversarial review): '#' was inserted at uniform random character
+positions, which slices inside words ~95% of the time (th#e, ki#ng); '@'
+always sits at a clean word boundary after "the ". Predictability is therefore
+not perfectly isolated from n-gram corruption. A clean-boundary control (noise
+token after random word boundaries) is planned; the numbers above should be
+read with that caveat until it lands.
 
 ## 3. Loss-reweighting causal test (`src/loss_reweight.py`)