summaryrefslogtreecommitdiff
path: root/results.md
diff options
context:
space:
mode:
authorVoid Agent <void@jayrup.hermes>2026-08-02 15:37:01 +0100
committerVoid Agent <void@jayrup.hermes>2026-08-02 15:37:01 +0100
commite1d881c5651022e2eea5e1ad8e9bf5f6ae688fac (patch)
tree1eb0b2a203d45e0b62984f4623301a35ef636317 /results.md
parent3bd33fbf685c6b16747d6ec4c1026d4fa6d04966 (diff)
Robustness confirmed: freq correlation identical under Anthropic's exact estimator (skip16+source_mean), max delta 0.008 all layers
Diffstat (limited to 'results.md')
-rw-r--r--results.md18
1 files changed, 16 insertions, 2 deletions
diff --git a/results.md b/results.md
index 88834c0..4225e39 100644
--- a/results.md
+++ b/results.md
@@ -61,8 +61,22 @@ Their estimator (`jlens/fitting.py`) differs from our default in two details:
mean over (source, future) pairs.
We added `--skip_first N --source_mean` to `src/jlens_v3.py` to mirror their
-estimator exactly. The robustness re-run (same model, their estimator choices)
-is queued behind the loss-reweighting GPU job; results will be appended here.
+estimator exactly and re-ran the same model (outputs/jlens_v3_anthropic/).
+Result: the frequency correlation is essentially IDENTICAL under their
+estimator at every layer (max delta 0.008):
+
+```
+ Layer default r anthropic r delta
+ L0 -0.643 -0.651 -0.008
+ L1 -0.668 -0.669 -0.000
+ L2 -0.672 -0.673 -0.001
+ L3 -0.685 -0.687 -0.002
+ L4 -0.637 -0.635 +0.002
+ L5 -0.606 -0.606 +0.000
+```
+
+The correlation is robust to the estimator's averaging choices (position mask
+and normalization). Last-layer identity check passed again (cos-sim 1.0000).
Frequency scan: case-insensitive grep of their entire repo (README, code,
experiment + evaluation data) finds ZERO occurrences of frequency/unigram/