summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorVoid Agent <void@jayrup.hermes>2026-08-02 16:46:43 +0100
committerVoid Agent <void@jayrup.hermes>2026-08-02 16:46:43 +0100
commit7ca6d8bcfd0db4feb89456100ccf766d02d5f258 (patch)
tree839907d1d00b98e87e465f48ad5cd5e45b7820a8 /docs
parentb1c232ebc00eaa115a1009d9a9f477721777ffe8 (diff)
Polish per review: results.md GPT-2 -0.18 demoted to do-not-cite footnote; blog section 10 adds W_U-at-init scale prediction (one-line GPT-2 unembedding check)
Diffstat (limited to 'docs')
-rw-r--r--docs/blog-jlens-frequency.md6
1 files changed, 6 insertions, 0 deletions
diff --git a/docs/blog-jlens-frequency.md b/docs/blog-jlens-frequency.md
index dec1edf..8ab2e46 100644
--- a/docs/blog-jlens-frequency.md
+++ b/docs/blog-jlens-frequency.md
@@ -385,6 +385,12 @@ regime where Anthropic's claims live) with proper statistical power, and to run
the occupancy test their capacity claim is actually about. That's the next
post.
+The learned-geometry finding makes one piece of that cheap and decisive: if
+the W_U row-norm anti-correlation is a general property of softmax-output
+models trained on Zipfian data, it should appear in GPT-2's unembedding matrix
+directly — no Jacobian computation at all. One line of code at scale, and it
+either generalizes or it doesn't.
+
## 11. How to reproduce everything
All code, data-prep scripts, experiment scripts, tests, and this analysis live