diff options
Diffstat (limited to 'docs/blog-jlens-frequency.md')
| -rw-r--r-- | docs/blog-jlens-frequency.md | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/docs/blog-jlens-frequency.md b/docs/blog-jlens-frequency.md index 4b2b429..34ce5fa 100644 --- a/docs/blog-jlens-frequency.md +++ b/docs/blog-jlens-frequency.md @@ -131,8 +131,9 @@ mathematically equivalent (verified by the identity check above). Their estimator has two differences of detail: it excludes the first 16 positions (attention sinks) and the last position from the average, and it averages over source positions rather than (source, future) pairs. We re-ran our analysis -with their exact estimator choices to confirm the correlation is robust to -those choices — see Section 6.) +with their exact estimator choices: the frequency correlation is essentially +identical at every layer (max delta 0.008, see results.md section 1b), so the +result is robust to those choices.) (One technical note: we capture the residual stream *before* the model's final layer norm. That matches the paper's definition — the Jacobian stops at the |