From 7ca6d8bcfd0db4feb89456100ccf766d02d5f258 Mon Sep 17 00:00:00 2001 From: Void Agent Date: Sun, 2 Aug 2026 16:46:43 +0100 Subject: Polish per review: results.md GPT-2 -0.18 demoted to do-not-cite footnote; blog section 10 adds W_U-at-init scale prediction (one-line GPT-2 unembedding check) --- docs/blog-jlens-frequency.md | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'docs') diff --git a/docs/blog-jlens-frequency.md b/docs/blog-jlens-frequency.md index dec1edf..8ab2e46 100644 --- a/docs/blog-jlens-frequency.md +++ b/docs/blog-jlens-frequency.md @@ -385,6 +385,12 @@ regime where Anthropic's claims live) with proper statistical power, and to run the occupancy test their capacity claim is actually about. That's the next post. +The learned-geometry finding makes one piece of that cheap and decisive: if +the W_U row-norm anti-correlation is a general property of softmax-output +models trained on Zipfian data, it should appear in GPT-2's unembedding matrix +directly — no Jacobian computation at all. One line of code at scale, and it +either generalizes or it doesn't. + ## 11. How to reproduce everything All code, data-prep scripts, experiment scripts, tests, and this analysis live -- cgit v1.2.3