summaryrefslogtreecommitdiff
path: root/results.md
diff options
context:
space:
mode:
authorVoid Agent <void@jayrup.hermes>2026-08-02 16:43:21 +0100
committerVoid Agent <void@jayrup.hermes>2026-08-02 16:43:21 +0100
commitb1c232ebc00eaa115a1009d9a9f477721777ffe8 (patch)
treebc01fe0b9480f56efcb099c45eaf8d451f30a98e /results.md
parentc201b8331c87294911eb3d5d07627cae8429cb5d (diff)
Blog revision per reviewer feedback: de-emphasize reviewer name-drops, promote LN technical note, willkn as Related-work, table footnote, section 8 power reframe (~13 seeds/arm needed); add W_U-at-init finding (learned geometry, r~0 at init vs -0.61 trained)
Diffstat (limited to 'results.md')
-rw-r--r--results.md6
1 files changed, 6 insertions, 0 deletions
diff --git a/results.md b/results.md
index c9e0c23..692ce03 100644
--- a/results.md
+++ b/results.md
@@ -43,6 +43,12 @@ the frequency signal lives partly in the static unembedding geometry (which is
baked into Anthropic's lens by definition) and partly in layer-dependent
dynamics; the mechanism of the latter is not yet pinned down.
+W_U geometry is LEARNED, not present at init: fresh models (3 seeds, same
+config) show r(||W_U[k]||, freq) = +0.002, -0.189, -0.166 (trained: -0.606),
+and row-norm spread grows ~4x during training ([0.36, 0.44] -> [0.86, 1.52]).
+The model learns to push rare-token unembedding rows outward and nothing
+corrects it.
+
Top tokens by faithful norm are consistently rare characters (`?`, `z`, `q`,
`$`); bottom are common ones (space, `e`, `t`, `i`).