diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 6 |
1 files changed, 6 insertions, 0 deletions
@@ -20,6 +20,11 @@ Workspace in Language Models"* (2026, 4. A causal loss-reweighting test (2x loss weight on 'q' targets vs two controls) tests whether effective frequency causally demotes a token's J-lens norm. See `results.md` for the latest numbers. +5. The geometric half generalizes: GPT-2's unembedding row norms (V = 50,257, + wte == tied lm_head) anti-correlate with token log-frequency + (r ≈ -0.45/-0.49, gpt2/gpt2-medium; monotone across frequency deciles) — + the learned W_U geometry is not a 65-char vocabulary artifact + (`src/wu_row_norm_check.py`, CPU-only). See `docs/blog-jlens-frequency.md` for the write-up and `results.md` for the numbers. The three independent adversarial reviews that shaped the project @@ -44,6 +49,7 @@ src/jlens_v3.py FAITHFUL J-lens: rows of W_U * J_l (canonical) src/synthetic_pair.py frequency-matched synthetic pair experiment src/loss_reweight.py causal loss-reweighting experiment src/gpt2_jlens.py GPT-2 scale test (under-powered; see results.md) +src/wu_row_norm_check.py at-scale W_U row-norm check (CPU-only; results.md §5) tests/ unit tests (see scripts/test.sh) scripts/test.sh canonical test command docs/blog-jlens-frequency.md write-up (Feynman-style) |