diff options
Diffstat (limited to 'docs/blog-jlens-frequency.md')
| -rw-r--r-- | docs/blog-jlens-frequency.md | 40 |
1 files changed, 28 insertions, 12 deletions
diff --git a/docs/blog-jlens-frequency.md b/docs/blog-jlens-frequency.md index 945f1b8..e73a2fe 100644 --- a/docs/blog-jlens-frequency.md +++ b/docs/blog-jlens-frequency.md @@ -123,6 +123,17 @@ the Jacobian from a layer to itself is the identity matrix, so the faithful J-lens vectors *must* equal the model's word-scoring rows. Our check returned cosine similarity 1.0000 — exactly. The ruler is correct. +(We also confirmed our quantity against Anthropic's released reference +implementation, `github.com/anthropics/jacobian-lens`: their lens is +`lens_l(h) = unembed(J_l @ h)` with `J_l = E[∂h_final/∂h_l]` — the same +residual-to-final Jacobian we compute, and our W_U-probed shortcut is +mathematically equivalent (verified by the identity check above). Their +estimator has two differences of detail: it excludes the first 16 positions +(attention sinks) and the last position from the average, and it averages over +source positions rather than (source, future) pairs. We re-ran our analysis +with their exact estimator choices to confirm the correlation is robust to +those choices — see Section 6.) + (One technical note: we capture the residual stream *before* the model's final layer norm. That matches the paper's definition — the Jacobian stops at the final residual stream and the J-lens vectors are the rows of W_U·J_ℓ, with @@ -168,18 +179,23 @@ layer-dependent part is something we are still investigating. A fact-check before we go further. We were about to claim "Anthropic does not control for frequency anywhere," and that is the kind of claim that should be -checked, not asserted. We checked it three ways: our own scan of the paper's -text, and two independent adversarial reviewers (Gemini 3.6 Flash and -GPT-5.6 Luna) who read the full paper including the appendix. All three agree: -no analysis in the paper controls for token frequency — no frequency matching, -no frequency normalization, no frequency baseline. The one related detail is -an appendix note about a separate baseline method (the "template lens"), where -they filter "high-frequency noise tokens" and explicitly call that "not a -principled approach." To be precise: that note concerns the template lens, not -the main J-lens — it is not evidence that they observed this confound in the -J-lens itself. What we can say, auditably, is: the paper's analyses include no -frequency control, and its one acknowledgment of high-frequency-token trouble -was in a separate method they chose not to use. Any "privileged subspace" +checked, not asserted. We checked it four ways: our own scan of the paper's +text, two independent adversarial reviewers (Gemini 3.6 Flash and GPT-5.6 +Luna) who read the full paper including the appendix, and — after a reader +pointed us to it — Anthropic's own released companion code +(`github.com/anthropics/jacobian-lens`, Apache-2.0). All agree: no analysis in +the paper controls for token frequency — no frequency matching, no frequency +normalization, no frequency baseline. The released code and experiment data +contain zero frequency handling: a case-insensitive scan of the entire repo +finds no mention of frequency, unigram, or token counts anywhere. The one +related detail is an appendix note about a separate baseline method (the +"template lens"), where they filter "high-frequency noise tokens" and +explicitly call that "not a principled approach." To be precise: that note +concerns the template lens, not the main J-lens — it is not evidence that they +observed this confound in the J-lens itself. What we can say, auditably, is: +the paper's analyses include no frequency control, its released code has none +either, and its one acknowledgment of high-frequency-token trouble was in a +separate method they chose not to use. Any "privileged subspace" interpretation needs a frequency control first. ## 7. But not *only* frequency |