Add willkn GreaterWrong corroboration: independent GPT-2-medium finding that raw J-lens misweights structural (high-frequency) tokens + full-rank Jacobian; shrinkage J+lambda I as untested next experiment

1 files changed, 14 insertions, 0 deletions

diff --git a/docs/blog-jlens-frequency.md b/docs/blog-jlens-frequency.md
index e73a2fe..7e01de0 100644
--- a/docs/blog-jlens-frequency.md
+++ b/docs/blog-jlens-frequency.md
@@ -198,6 +198,20 @@ either, and its one acknowledgment of high-frequency-token trouble was in a
 separate method they chose not to use. Any "privileged subspace"
 interpretation needs a frequency control first.
 
+We are also not alone in noticing the raw lens is distorted by token
+statistics. An independent research-engineer analysis of the same paper
+(willkn, "Anthropic's J-Lens: A Research Engineer's Analysis", GreaterWrong,
+24 Jul 2026), working on GPT-2-medium (355M — thirty times our model), found
+that the raw fitted Jacobian "misweights structural tokens (grammar,
+punctuation) over semantic content" — its dominant spectral channels carry
+~10x the gain of the residual pathway. Structural tokens are the high-frequency
+tokens. They also found the Jacobian essentially full-rank (562-858 dimensions
+for 90% of spectral variance), matching our toy-scale rank result. Their fix is
+a single-parameter shrinkage regularizer (J + λI) that restores next-token
+faithfulness — and Anthropic's released fitting code applies no such
+regularization. Whether shrinkage also removes the frequency correlation is an
+experiment we have not run yet; it is a natural next step.
+
 ## 7. But not *only* frequency
 
 Now the twist. Correlation is not causation, so we ran a cleaner test. We made