summaryrefslogtreecommitdiff
path: root/results.md
diff options
context:
space:
mode:
authorVoid Agent <void@jayrup.hermes>2026-08-02 15:23:54 +0100
committerVoid Agent <void@jayrup.hermes>2026-08-02 15:23:54 +0100
commit5461d3beca4f7bc8aa2feb7d1320056b18e69656 (patch)
tree88cdb23f30a778ec7c24897456cf302edd3fe7de /results.md
parent781a07b0c52c62657a8030eec8bbab76d26fa90f (diff)
Loss-reweighting results: q/control 0.944 (CI crosses 1), q/ctrl_random 0.985 — small non-significant causal demotion; freq corr invariant (r~-0.66) across all 9 models; blog section 8 + results.md section 3 filled honestly
Diffstat (limited to 'results.md')
-rw-r--r--results.md30
1 files changed, 27 insertions, 3 deletions
diff --git a/results.md b/results.md
index 48c1c48..afb130e 100644
--- a/results.md
+++ b/results.md
@@ -124,9 +124,33 @@ read with that caveat until it lands.
Three models per seed, identical init + minibatch order: 'q' targets weighted
x2 in the loss, plain control, and a same-total-loss control upweighting random
non-'q' targets. Question: does raising effective frequency causally reduce
-'q's faithful J-lens norm? PENDING — run completes within hours of this file
-being written; the summary table is printed by
-`python3 src/loss_reweight.py --step summary --layers 2,3,4`.
+'q's faithful J-lens norm? Faithful lens norm of 'q' (mean over layers 2-4):
+
+```
+ seed | q(2x) control ctrl_random | q/control q/ctrl_random
+ 0 | 0.0163 0.0174 0.0152 | 0.934 1.069
+ 1 | 0.0150 0.0171 0.0161 | 0.881 0.932
+ 2 | 0.0161 0.0158 0.0169 | 1.019 0.952
+```
+
+Cross-seed: q/control mean = 0.944 (bootstrap 95% CI [0.881, 1.019]),
+q/ctrl_random mean = 0.985 (CI [0.932, 1.069]).
+
+Reading:
+1. Expected-direction signal, small: 2x loss pressure lowers 'q' norm in 2/3
+ seeds (~5.6% mean vs control), but CI crosses 1.0 and the effect vanishes
+ vs the random-upweight control (0.985). Not a robust causal demotion at
+ this power.
+2. Frequency correlation is invariant across all 9 models: mean r = -0.66
+ (min -0.685, max -0.626) in every mode/seed/layer — training with 'q'
+ upweighted does not change the correlation structure (geometric, per the
+ W_U decomposition).
+3. Contrast: doubling ACTUAL corpus occurrences of 'q' (original ablation)
+ dropped its norm 67%; doubling its LOSS weight drops it ~6%. Data-frequency
+ is the strong causal lever; gradient reweighting under AdamW is weak
+ (adaptive per-param scaling absorbs part of the signal).
+4. Absolute norms differ across experiments (base L3 q = 0.0109 vs these
+ 0.015-0.018) — only within-experiment comparisons are meaningful.
## 4. Historical / do-not-copy