diff options
Diffstat (limited to 'results.md')
| -rw-r--r-- | results.md | 24 |
1 files changed, 0 insertions, 24 deletions
@@ -89,30 +89,6 @@ experiment + evaluation data) finds ZERO occurrences of frequency/unigram/ token-count terms. Their README confirms the fit corpus is "a generic web-text corpus" — no frequency balancing. -## 1c. Independent corroboration (willkn, GreaterWrong, GPT-2-medium) - -An independent engineering analysis of the paper (willkn, GreaterWrong, 24 Jul -2026; notebooks: github.com/willkn/jlens_re; full text: docs/willkn-jlens- -greaterwrong-analysis.md), on GPT-2-medium (355M, d=1024, L=24, V=50,257), -found: -- the raw fitted Jacobian "misweights structural tokens (grammar, punctuation) - over semantic content" — dominant spectral channels carry ~10x the gain of - the residual pathway. Structural tokens ARE the high-frequency tokens; this - is the same distortion we measure as frequency anti-correlation, at 30x our - model scale and via a different metric (next-token faithfulness of the - readout). -- the Jacobian is essentially full-rank: 562-858 dimensions capture 90% of - spectral variance, "leaving little room for compression" — corroborates our - dimensional-starvation / full-rank result at scale. -- a single-parameter shrinkage regularizer (J + lambda*I) monotonically - recovers next-token faithfulness (beats logit lens at layer 12: 0.294 vs - 0.275). Anthropic's released fitting code applies no such regularization. -- measured noise law rel_F(J_n) ~ c_l/sqrt(n), c_l growing ~4x from layer 20 - to layer 4; per-backward cost ~5ms + 9ms/layer on L4 at dim_batch=8. - -Untested (natural next experiment): does shrinkage also remove our frequency -correlation? - ## 2. Frequency-matched synthetic pair (`src/synthetic_pair.py`) Two new characters at identical 0.1% unigram frequency in Shakespeare: |