jspace-nanogpt.git/results.md, branch main
J-space on nanoGPT: Jacobian Lens replication & critique. J-lens norms anti-correlate with token frequency (r ~ -0.61 to -0.69); frequency-matched synthetic pair + causal loss-reweighting tests.
http://git.jayrup.me/c/jspace-nanogpt.git/atom?h=main
2026-08-02T15:46:43Z
Polish per review: results.md GPT-2 -0.18 demoted to do-not-cite footnote; blog section 10 adds W_U-at-init scale prediction (one-line GPT-2 unembedding check)
2026-08-02T15:46:43Z
Void Agent
void@jayrup.hermes
2026-08-02T15:46:43Z
urn:sha1:7ca6d8bcfd0db4feb89456100ccf766d02d5f258
Blog revision per reviewer feedback: de-emphasize reviewer name-drops, promote LN technical note, willkn as Related-work, table footnote, section 8 power reframe (~13 seeds/arm needed); add W_U-at-init finding (learned geometry, r~0 at init vs -0.61 trained)
2026-08-02T15:43:21Z
Void Agent
void@jayrup.hermes
2026-08-02T15:43:21Z
urn:sha1:b1c232ebc00eaa115a1009d9a9f477721777ffe8
Clean-boundary synthetic-pair control: ratio 1.311 (CI [1.258,1.402]) vs 1.471 — corruption inflated by ~12%, structure signal survives at 0% word-slicing; blog section 7 + results.md resolved
2026-08-02T15:15:00Z
Void Agent
void@jayrup.hermes
2026-08-02T15:15:00Z
urn:sha1:c201b8331c87294911eb3d5d07627cae8429cb5d
Robustness confirmed: freq correlation identical under Anthropic's exact estimator (skip16+source_mean), max delta 0.008 all layers
2026-08-02T14:37:01Z
Void Agent
void@jayrup.hermes
2026-08-02T14:37:01Z
urn:sha1:e1d881c5651022e2eea5e1ad8e9bf5f6ae688fac
Fix in-word metric: strict (both-sides, 58% random / 0% clean) vs touches (95%); blog/results caveat corrected to the strict discriminator
2026-08-02T14:31:53Z
Void Agent
void@jayrup.hermes
2026-08-02T14:31:53Z
urn:sha1:3bd33fbf685c6b16747d6ec4c1026d4fa6d04966
Loss-reweighting results: q/control 0.944 (CI crosses 1), q/ctrl_random 0.985 — small non-significant causal demotion; freq corr invariant (r~-0.66) across all 9 models; blog section 8 + results.md section 3 filled honestly
2026-08-02T14:23:54Z
Void Agent
void@jayrup.hermes
2026-08-02T14:23:54Z
urn:sha1:5461d3beca4f7bc8aa2feb7d1320056b18e69656
Add willkn GreaterWrong corroboration: independent GPT-2-medium finding that raw J-lens misweights structural (high-frequency) tokens + full-rank Jacobian; shrinkage J+lambda I as untested next experiment
2026-08-02T13:54:17Z
Void Agent
void@jayrup.hermes
2026-08-02T13:54:17Z
urn:sha1:781a07b0c52c62657a8030eec8bbab76d26fa90f
Verify against Anthropic's official jacobian-lens repo: estimator match, zero frequency in their code+data, add --skip_first/--source_mean mirror mode
2026-08-02T13:45:51Z
Void Agent
void@jayrup.hermes
2026-08-02T13:45:51Z
urn:sha1:1b9b346dc88a3c043af2fa6ef1a1c7ca04a7311b
Address Gemini+Opus repo reviews: W_U decomposition, Spearman+p-values, synthetic-pair CIs + corruption caveat, accurate fact-check, softened burden-of-proof, LayerNorm note, reviews caveat
2026-08-02T13:19:39Z
Void Agent
void@jayrup.hermes
2026-08-02T13:19:39Z
urn:sha1:42660815eef000bcb662287a97cbb3012b6c90b8
Docs: Feynman-style blog draft, MIT license, requirements, results.md, README rewrite with repro steps; reviews -> docs/reviews
2026-08-02T12:52:40Z
Void Agent
void@jayrup.hermes
2026-08-02T12:52:40Z
urn:sha1:071b97c6afd43629a9bdb8e196ab2a3cbe86854c