summaryrefslogtreecommitdiff
path: root/results.md
diff options
context:
space:
mode:
authorVoid Agent <void@jayrup.hermes>2026-08-02 16:15:00 +0100
committerVoid Agent <void@jayrup.hermes>2026-08-02 16:15:00 +0100
commitc201b8331c87294911eb3d5d07627cae8429cb5d (patch)
tree0e16dd3d56df9642bcbae265c852b2e0a648f2ba /results.md
parent503a192dd39bd27c3853d6850140a526808b223a (diff)
Clean-boundary synthetic-pair control: ratio 1.311 (CI [1.258,1.402]) vs 1.471 — corruption inflated by ~12%, structure signal survives at 0% word-slicing; blog section 7 + results.md resolved
Diffstat (limited to 'results.md')
-rw-r--r--results.md23
1 files changed, 16 insertions, 7 deletions
diff --git a/results.md b/results.md
index 4225e39..c9e0c23 100644
--- a/results.md
+++ b/results.md
@@ -126,13 +126,22 @@ Middle-layer ratio across seeds: 1.47 +/- 0.09 (SD), bootstrap 95% CI
[1.37, 1.53]. The frequency anti-correlation holds, but the lens also carries
genuine conditional-predictability signal.
-CAVEAT (from adversarial review): '#' was inserted at uniform random character
-positions, which slices through the middle of a word 58% of the time (letter
-on both sides: th#e, ki#ng); '@' always sits at a clean word boundary after
-"the ". Predictability is therefore not perfectly isolated from n-gram
-corruption. A clean-boundary control (noise token after random word
-boundaries, 0% word-slicing) is running; the numbers above should be read with
-that caveat until it lands.
+CAVEAT + CONTROL (resolved): '#' was originally inserted at uniform random
+character positions, which slices through the middle of a word 58% of the time
+(letter on both sides: th#e, ki#ng); '@' always sits at a clean word boundary
+after "the ". A clean-boundary control (noise token after random word
+boundaries, 0% word-slicing, same frequency) was run with three fresh seeds:
+
+```
+ placement of '#' middle-layer ratio @/# bootstrap 95% CI
+ random (58% slicing) 1.471 +/- 0.090 [1.368, 1.529]
+ clean boundary (0%) 1.311 +/- 0.079 [1.258, 1.402]
+```
+
+Reading: the corruption confound was real (inflated the ratio by ~12%) but not
+the whole story — the structure signal survives at clean boundaries, CI
+entirely above 1 in every seed. Original run: outputs/synth_pair/seed{s}/;
+control run: outputs/synth_pair_clean/seed{s}/.
## 3. Loss-reweighting causal test (`src/loss_reweight.py`)