diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/blog-jlens-frequency.md | 13 |
1 files changed, 7 insertions, 6 deletions
diff --git a/docs/blog-jlens-frequency.md b/docs/blog-jlens-frequency.md index ceb97d5..4b2b429 100644 --- a/docs/blog-jlens-frequency.md +++ b/docs/blog-jlens-frequency.md @@ -242,12 +242,13 @@ predictability* (what "verbalizable" should mean) depends on a control that is still running — see the caveat below. One caveat, found by a reviewer: the noise token '#' was inserted at random -character positions, which slices *inside* words ~95% of the time (th#e, -ki#ng), while '@' always sits at a clean word boundary after "the ". That -means predictability is not perfectly isolated from n-gram corruption. We are -running a clean-boundary control (noise token inserted after random word -boundaries — still unpredictable, no word-slicing) to rule it out; the numbers -above should be read with that caveat until the control lands. +character positions, which slices through the middle of a word 58% of the time +(th#e, ki#ng — letter on both sides), while '@' always sits at a clean word +boundary after "the ". Predictability is therefore not perfectly isolated from +n-gram corruption. We are running a clean-boundary control (noise token +inserted after random word boundaries — 0% word-slicing, still unpredictable) +to rule it out; the numbers above should be read with that caveat until the +control lands. ## 8. The causal test: what actually happened |