diff options
| author | Void Agent <void@jayrup.hermes> | 2026-08-02 16:15:00 +0100 |
|---|---|---|
| committer | Void Agent <void@jayrup.hermes> | 2026-08-02 16:15:00 +0100 |
| commit | c201b8331c87294911eb3d5d07627cae8429cb5d (patch) | |
| tree | 0e16dd3d56df9642bcbae265c852b2e0a648f2ba /docs | |
| parent | 503a192dd39bd27c3853d6850140a526808b223a (diff) | |
Clean-boundary synthetic-pair control: ratio 1.311 (CI [1.258,1.402]) vs 1.471 — corruption inflated by ~12%, structure signal survives at 0% word-slicing; blog section 7 + results.md resolved
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/blog-jlens-frequency.md | 29 |
1 files changed, 21 insertions, 8 deletions
diff --git a/docs/blog-jlens-frequency.md b/docs/blog-jlens-frequency.md index 3d68759..23529de 100644 --- a/docs/blog-jlens-frequency.md +++ b/docs/blog-jlens-frequency.md @@ -267,18 +267,31 @@ The predictable token scores **~1.4-1.5x higher** than the noise token at identical frequency, in every layer of every seed. Middle-layer ratio across the three seeds: 1.47 ± 0.09, bootstrap 95% CI [1.37, 1.53] — entirely above 1. So the lens is not a pure frequency meter: at equal frequency, the two -tokens differ in norm. Whether that difference is specifically *conditional -predictability* (what "verbalizable" should mean) depends on a control that is -still running — see the caveat below. +tokens differ in norm. One caveat, found by a reviewer: the noise token '#' was inserted at random character positions, which slices through the middle of a word 58% of the time (th#e, ki#ng — letter on both sides), while '@' always sits at a clean word -boundary after "the ". Predictability is therefore not perfectly isolated from -n-gram corruption. We are running a clean-boundary control (noise token -inserted after random word boundaries — 0% word-slicing, still unpredictable) -to rule it out; the numbers above should be read with that caveat until the -control lands. +boundary after "the ". That confounds predictability with n-gram corruption — +so we ran the control that isolates them: '#' inserted at random *word +boundaries* (0% word-slicing, still unpredictable), same 0.0998% frequency, +three fresh seeds. + +The control is done, and it is the honest kind of result — partly confirming, +partly correcting: + +``` + placement of '#' middle-layer ratio @/# bootstrap 95% CI + random (58% slicing) 1.47 ± 0.09 [1.37, 1.53] + clean boundary (0%) 1.31 ± 0.08 [1.26, 1.40] +``` + +The corruption confound was real: it inflated the estimate by about 12%. But +it was not the whole story. At identical frequency, with clean boundaries and +nothing sliced, the predictable token still scores ~1.3x higher than the +unpredictable one, and the CI stays entirely above 1 in every seed. The +conditional-predictability signal — the thing "verbalizable" should mean — +survives the control, modestly smaller than our first estimate. ## 8. The causal test: what actually happened |
