1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
|
# Results
All numbers below are from the faithful J-lens (`src/jlens_v3.py`), which
computes exactly the paper's quantity: rows of `W_U * J_l` where
`J_l = E[ d h_final / d h_l ]` (average residual-to-residual Jacobian, read out
through the unembedding). Verification: at the last layer, J must be the
identity, and the check returns cosine similarity **1.0000**.
## 1. Both-ways comparison: old proxy vs faithful lens (trained 10.65M char model)
Pearson r between token frequency and J-lens norm, per layer. n = 65 tokens
(full char vocab). The correlation survives the faithful implementation at
every layer. p-values: all layers p ~ 10^-9 to 10^-13. Spearman rank
correlation is even stronger: faithful -0.69 to -0.85; proxy -0.63 to -0.81
(not a leverage artifact of extreme common tokens).
```
Layer proxy r faithful r Spearman(faith) partial r(faith, freq | ||W_U||)
L0 -0.661 -0.643 -0.844 -0.307
L1 -0.673 -0.668 -0.851 -0.319
L2 -0.653 -0.672 -0.825 -0.338
L3 -0.648 -0.685 -0.769 -0.364
L4 -0.562 -0.637 -0.716 -0.240
L5 -0.665 -0.606 -0.808 +0.056
```
### W_U decomposition (where does the correlation come from?)
The faithful lens vector for token k is `W_U[k] * J_l`. The unembedding row
norms themselves anti-correlate with frequency:
```
r(||W_U[k]||, freq) = -0.606
r(||W_U[k]||, log10 freq) = -0.693
Spearman(||W_U[k]||, freq) = -0.808
```
`r(faithful_norm, ||W_U[k]||)` per layer: +0.70 to +1.00 (L5 = +1.00 exactly,
since J = identity at the last layer). After regressing out the W_U row-norm
component, a layer-dependent anti-correlation survives in layers 0-4 (partial
r ≈ -0.24 to -0.36) and vanishes at the last layer (+0.06). Interpretation:
the frequency signal lives partly in the static unembedding geometry (which is
baked into Anthropic's lens by definition) and partly in layer-dependent
dynamics; the mechanism of the latter is not yet pinned down.
Top tokens by faithful norm are consistently rare characters (`?`, `z`, `q`,
`$`); bottom are common ones (space, `e`, `t`, `i`).
## 2. Frequency-matched synthetic pair (`src/synthetic_pair.py`)
Two new characters at identical 0.1% unigram frequency in Shakespeare:
`@` appears only after the trigger "the " (predictable in context);
`#` appears at uniform random positions. Faithful J-lens norm per seed, range
over layers 0-5:
```
seed @ norm (predictable) # norm (noise) ratio freq corr r
0 0.0232 - 0.0246 0.0152 - 0.0154 1.51-1.60 -0.59..-0.68
1 0.0224 - 0.0237 0.0148 - 0.0151 1.50-1.60 -0.59..-0.66
2 0.0215 - 0.0233 0.0154 - 0.0163 1.35-1.51 -0.62..-0.66
```
Reading: at equal frequency, the structured token scores ~1.4-1.5x higher.
Middle-layer ratio across seeds: 1.47 +/- 0.09 (SD), bootstrap 95% CI
[1.37, 1.53]. The frequency anti-correlation holds, but the lens also carries
genuine conditional-predictability signal.
CAVEAT (from adversarial review): '#' was inserted at uniform random character
positions, which slices inside words ~95% of the time (th#e, ki#ng); '@'
always sits at a clean word boundary after "the ". Predictability is therefore
not perfectly isolated from n-gram corruption. A clean-boundary control (noise
token after random word boundaries) is planned; the numbers above should be
read with that caveat until it lands.
## 3. Loss-reweighting causal test (`src/loss_reweight.py`)
Three models per seed, identical init + minibatch order: 'q' targets weighted
x2 in the loss, plain control, and a same-total-loss control upweighting random
non-'q' targets. Question: does raising effective frequency causally reduce
'q's faithful J-lens norm? PENDING — run completes within hours of this file
being written; the summary table is printed by
`python3 src/loss_reweight.py --step summary --layers 2,3,4`.
## 4. Historical / do-not-copy
- Original proxy finding (r = -0.65, `jlens_v2`): superseded by the faithful
implementation; kept only for the both-ways comparison.
- GPT-2 correlation (r = -0.18, `gpt2_jlens.py`): UNDER-POWERED (96 token
positions, n=100 sampled tokens) and computed a different quantity
(norm-per-batch vs norm-of-mean). Directionally consistent but not
publishable evidence on its own.
|
