diff options
Diffstat (limited to 'docs/blog-jlens-frequency.md')
| -rw-r--r-- | docs/blog-jlens-frequency.md | 34 |
1 files changed, 21 insertions, 13 deletions
diff --git a/docs/blog-jlens-frequency.md b/docs/blog-jlens-frequency.md index d45b183..945f1b8 100644 --- a/docs/blog-jlens-frequency.md +++ b/docs/blog-jlens-frequency.md @@ -69,6 +69,11 @@ lens is ranking words by the size of this gradient, the ranking is partly pre-written by the frequency distribution before the model even learns anything. +(A note on that intuition: it applies directly to our first, simpler +implementation, which differentiated through the softmax. With the faithful +lens the mechanism is different — it turns out to live partly in the geometry +of the word-scoring matrix itself. Section 6 has the full decomposition.) + In other words: **a "privileged workspace" might just be a frequency effect wearing a fancy hat.** @@ -167,11 +172,14 @@ checked, not asserted. We checked it three ways: our own scan of the paper's text, and two independent adversarial reviewers (Gemini 3.6 Flash and GPT-5.6 Luna) who read the full paper including the appendix. All three agree: no analysis in the paper controls for token frequency — no frequency matching, -no frequency normalization, no frequency baseline. The only place the word -shows up is an appendix note about a separate baseline method (the "template -lens"), where they filter "high-frequency noise tokens" — and they explicitly -call that "not a principled approach," then never apply it to the main -J-lens. They saw the effect. They didn't fix it. Any "privileged subspace" +no frequency normalization, no frequency baseline. The one related detail is +an appendix note about a separate baseline method (the "template lens"), where +they filter "high-frequency noise tokens" and explicitly call that "not a +principled approach." To be precise: that note concerns the template lens, not +the main J-lens — it is not evidence that they observed this confound in the +J-lens itself. What we can say, auditably, is: the paper's analyses include no +frequency control, and its one acknowledgment of high-frequency-token trouble +was in a separate method they chose not to use. Any "privileged subspace" interpretation needs a frequency control first. ## 7. But not *only* frequency @@ -198,10 +206,10 @@ training runs showed: The predictable token scores **~1.4-1.5x higher** than the noise token at identical frequency, in every layer of every seed. Middle-layer ratio across the three seeds: 1.47 ± 0.09, bootstrap 95% CI [1.37, 1.53] — entirely above -1. So the lens is not a pure frequency meter. It genuinely responds to -conditional predictability — which, honestly, is what "verbalizable" should -mean. The J-lens measures *both*: a frequency prior that is never subtracted -out, and a real structure signal on top of it. +1. So the lens is not a pure frequency meter: at equal frequency, the two +tokens differ in norm. Whether that difference is specifically *conditional +predictability* (what "verbalizable" should mean) depends on a control that is +still running — see the caveat below. One caveat, found by a reviewer: the noise token '#' was inserted at random character positions, which slices *inside* words ~95% of the time (th#e, @@ -258,10 +266,10 @@ post. ## 11. How to reproduce everything All code, data-prep scripts, experiment scripts, tests, and this analysis live -in the repository: [link to cgit]. Summary of results in `results.md`. -Reproduction steps in the README. The only requirements are a Linux machine -with Docker, a CUDA GPU (any modern card; we used a 4GB Quadro K2200), and the -`pytorch/pytorch:2.4.1-cuda11.8` image. +in the repository (URL to be added once the public git instance is live). +Summary of results in `results.md`. Reproduction steps in the README. The only +requirements are a Linux machine with Docker, a CUDA GPU (any modern card; we +used a 4GB Quadro K2200), and the `pytorch/pytorch:2.4.1-cuda11.8` image. Run the test suite: ``` |