summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorVoid Agent <void@jayrup.hermes>2026-08-02 16:15:00 +0100
committerVoid Agent <void@jayrup.hermes>2026-08-02 16:15:00 +0100
commitc201b8331c87294911eb3d5d07627cae8429cb5d (patch)
tree0e16dd3d56df9642bcbae265c852b2e0a648f2ba /docs
parent503a192dd39bd27c3853d6850140a526808b223a (diff)
Clean-boundary synthetic-pair control: ratio 1.311 (CI [1.258,1.402]) vs 1.471 — corruption inflated by ~12%, structure signal survives at 0% word-slicing; blog section 7 + results.md resolved
Diffstat (limited to 'docs')
-rw-r--r--docs/blog-jlens-frequency.md29
1 files changed, 21 insertions, 8 deletions
diff --git a/docs/blog-jlens-frequency.md b/docs/blog-jlens-frequency.md
index 3d68759..23529de 100644
--- a/docs/blog-jlens-frequency.md
+++ b/docs/blog-jlens-frequency.md
@@ -267,18 +267,31 @@ The predictable token scores **~1.4-1.5x higher** than the noise token at
identical frequency, in every layer of every seed. Middle-layer ratio across
the three seeds: 1.47 ± 0.09, bootstrap 95% CI [1.37, 1.53] — entirely above
1. So the lens is not a pure frequency meter: at equal frequency, the two
-tokens differ in norm. Whether that difference is specifically *conditional
-predictability* (what "verbalizable" should mean) depends on a control that is
-still running — see the caveat below.
+tokens differ in norm.
One caveat, found by a reviewer: the noise token '#' was inserted at random
character positions, which slices through the middle of a word 58% of the time
(th#e, ki#ng — letter on both sides), while '@' always sits at a clean word
-boundary after "the ". Predictability is therefore not perfectly isolated from
-n-gram corruption. We are running a clean-boundary control (noise token
-inserted after random word boundaries — 0% word-slicing, still unpredictable)
-to rule it out; the numbers above should be read with that caveat until the
-control lands.
+boundary after "the ". That confounds predictability with n-gram corruption —
+so we ran the control that isolates them: '#' inserted at random *word
+boundaries* (0% word-slicing, still unpredictable), same 0.0998% frequency,
+three fresh seeds.
+
+The control is done, and it is the honest kind of result — partly confirming,
+partly correcting:
+
+```
+ placement of '#' middle-layer ratio @/# bootstrap 95% CI
+ random (58% slicing) 1.47 ± 0.09 [1.37, 1.53]
+ clean boundary (0%) 1.31 ± 0.08 [1.26, 1.40]
+```
+
+The corruption confound was real: it inflated the estimate by about 12%. But
+it was not the whole story. At identical frequency, with clean boundaries and
+nothing sliced, the predictable token still scores ~1.3x higher than the
+unpredictable one, and the CI stays entirely above 1 in every seed. The
+conditional-predictability signal — the thing "verbalizable" should mean —
+survives the control, modestly smaller than our first estimate.
## 8. The causal test: what actually happened