diff options
Diffstat (limited to 'results.md')
| -rw-r--r-- | results.md | 18 |
1 files changed, 16 insertions, 2 deletions
@@ -61,8 +61,22 @@ Their estimator (`jlens/fitting.py`) differs from our default in two details: mean over (source, future) pairs. We added `--skip_first N --source_mean` to `src/jlens_v3.py` to mirror their -estimator exactly. The robustness re-run (same model, their estimator choices) -is queued behind the loss-reweighting GPU job; results will be appended here. +estimator exactly and re-ran the same model (outputs/jlens_v3_anthropic/). +Result: the frequency correlation is essentially IDENTICAL under their +estimator at every layer (max delta 0.008): + +``` + Layer default r anthropic r delta + L0 -0.643 -0.651 -0.008 + L1 -0.668 -0.669 -0.000 + L2 -0.672 -0.673 -0.001 + L3 -0.685 -0.687 -0.002 + L4 -0.637 -0.635 +0.002 + L5 -0.606 -0.606 +0.000 +``` + +The correlation is robust to the estimator's averaging choices (position mask +and normalization). Last-layer identity check passed again (cos-sim 1.0000). Frequency scan: case-insensitive grep of their entire repo (README, code, experiment + evaluation data) finds ZERO occurrences of frequency/unigram/ |