summaryrefslogtreecommitdiff
path: root/docs/blog-jlens-frequency.md
diff options
context:
space:
mode:
authorVoid Agent <void@jayrup.hermes>2026-08-02 14:13:10 +0100
committerVoid Agent <void@jayrup.hermes>2026-08-02 14:13:10 +0100
commit0164f724cce14ae517090c69f9c1e329d5c0d98c (patch)
tree6f38431f9b654e1a0b4dabf17798b09d9b270b6f /docs/blog-jlens-frequency.md
parent071b97c6afd43629a9bdb8e196ab2a3cbe86854c (diff)
Blog: fact-check section — 3 independent checks confirm Anthropic has no frequency control (incl. template-lens high-frequency note); preempt 'they have controls' strawman
Diffstat (limited to 'docs/blog-jlens-frequency.md')
-rw-r--r--docs/blog-jlens-frequency.md20
1 files changed, 18 insertions, 2 deletions
diff --git a/docs/blog-jlens-frequency.md b/docs/blog-jlens-frequency.md
index d0f7ffa..7879535 100644
--- a/docs/blog-jlens-frequency.md
+++ b/docs/blog-jlens-frequency.md
@@ -136,8 +136,19 @@ faithful lens, correlated with token frequency like this:
The correlation survived the faithful implementation — slightly *stronger*, if
anything. The rare characters ('?', 'z', 'q', '$') sit at the top of the
J-space ranking; the common ones (space, 'e', 't', 'i') sit at the bottom. On
-the paper's own quantity, the J-lens ranking is frequency-confounded. Anthropic
-does not control for this anywhere in their analysis.
+the paper's own quantity, the J-lens ranking is frequency-confounded.
+
+A fact-check before we go further. We were about to claim "Anthropic does not
+control for frequency anywhere," and that is the kind of claim that should be
+checked, not asserted. We checked it three ways: our own scan of the paper's
+text, and two independent adversarial reviewers (Gemini 3.6 Flash and
+GPT-5.6 Luna) who read the full paper including the appendix. All three agree:
+the word "frequency" appears nowhere in the paper, no analysis matches tokens
+on frequency, and no baseline controls for it. The closest thing is an
+appendix note about a separate baseline method (the "template lens") where
+they filter "high-frequency noise tokens" — and they explicitly call that
+"not a principled approach," then never apply it to the main J-lens. They saw
+the effect. They didn't fix it. We think the burden of proof is on them.
## 7. But not *only* frequency
@@ -186,6 +197,11 @@ Let us be very careful here, because it would be easy to overclaim.
Anthropic's actual capacity claim (which is about *occupancy* — how often
J-lens directions are used per position — not about the rank of the word
vectors).
+- We are **not** saying they have no controls at all. Their occupancy
+ analysis compares against random-direction baselines, and their probes
+ subtract mean concept directions. Those are real experimental controls —
+ but none of them is a token-frequency control, which is the specific thing
+ our claim is about.
- We are **not** saying the lens is useless. The synthetic-pair result shows it
carries real structure signal.
- We are **not** saying "it's just linear algebra." Our toy models don't show