From b1c232ebc00eaa115a1009d9a9f477721777ffe8 Mon Sep 17 00:00:00 2001 From: Void Agent Date: Sun, 2 Aug 2026 16:43:21 +0100 Subject: Blog revision per reviewer feedback: de-emphasize reviewer name-drops, promote LN technical note, willkn as Related-work, table footnote, section 8 power reframe (~13 seeds/arm needed); add W_U-at-init finding (learned geometry, r~0 at init vs -0.61 trained) --- results.md | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'results.md') diff --git a/results.md b/results.md index c9e0c23..692ce03 100644 --- a/results.md +++ b/results.md @@ -43,6 +43,12 @@ the frequency signal lives partly in the static unembedding geometry (which is baked into Anthropic's lens by definition) and partly in layer-dependent dynamics; the mechanism of the latter is not yet pinned down. +W_U geometry is LEARNED, not present at init: fresh models (3 seeds, same +config) show r(||W_U[k]||, freq) = +0.002, -0.189, -0.166 (trained: -0.606), +and row-norm spread grows ~4x during training ([0.36, 0.44] -> [0.86, 1.52]). +The model learns to push rare-token unembedding rows outward and nothing +corrects it. + Top tokens by faithful norm are consistently rare characters (`?`, `z`, `q`, `$`); bottom are common ones (space, `e`, `t`, `i`). -- cgit v1.2.3