jspace-nanogpt.git - J-space on nanoGPT: Jacobian Lens replication & critique. J-lens norms anti-correlate with token frequency (r ~ -0.61 to -0.69); frequency-matched synthetic pair + causal loss-reweighting tests.

Age	Commit message (Collapse)	Author
4 days	Blog (Claude's review): add 'short version' TL;DR up top for wider audience; ↵	Void Agent
	promote W_U decomposition to its own flagged subsection as the novel mechanistic contribution
4 days	Blog: fill in public repo URL (git.jayrup.me/c/jspace-nanogpt.git)	Void Agent

4 days	Robustness confirmed: freq correlation identical under Anthropic's exact ↵	Void Agent
	estimator (skip16+source_mean), max delta 0.008 all layers
4 days	Fix in-word metric: strict (both-sides, 58% random / 0% clean) vs touches ↵	Void Agent
	(95%); blog/results caveat corrected to the strict discriminator
4 days	synthetic_pair: add --clean boundary mode (noise token at word starts) + ↵	Void Agent
	configurable dirs; prep reports in-word fraction
4 days	Loss-reweighting results: q/control 0.944 (CI crosses 1), q/ctrl_random ↵	Void Agent
	0.985 — small non-significant causal demotion; freq corr invariant (r~-0.66) across all 9 models; blog section 8 + results.md section 3 filled honestly
4 days	Add willkn GreaterWrong corroboration: independent GPT-2-medium finding that ↵	Void Agent
	raw J-lens misweights structural (high-frequency) tokens + full-rank Jacobian; shrinkage J+lambda I as untested next experiment
4 days	Verify against Anthropic's official jacobian-lens repo: estimator match, ↵	Void Agent
	zero frequency in their code+data, add --skip_first/--source_mean mirror mode
4 days	test_jlens_v3: move tiny model to DEV (fix cpu/cuda mismatch on cuda runs)	Void Agent

4 days	Address Luna repo review: dynamic repo root (no hardcoded cwd), J-lens ↵	Void Agent
	correctness test (last-layer identity), auditable fact-check, mechanism narrative fix, softened conditional-predictability claim, README data-prep + artifacts note
4 days	Add src/stats_decomp.py: reproducible W_U decomposition, p-values, Spearman, ↵	Void Agent
	synthetic-pair CI (backs results.md numbers)
4 days	Document n_pairs denominator (exact count) and ctrl_random gradient-mass ↵	Void Agent
	caveat (comment-only)
4 days	Address Gemini+Opus repo reviews: W_U decomposition, Spearman+p-values, ↵	Void Agent
	synthetic-pair CIs + corruption caveat, accurate fact-check, softened burden-of-proof, LayerNorm note, reviews caveat
4 days	Blog: fact-check section — 3 independent checks confirm Anthropic has no ↵	Void Agent
	frequency control (incl. template-lens high-frequency note); preempt 'they have controls' strawman
4 days	Docs: Feynman-style blog draft, MIT license, requirements, results.md, ↵	Void Agent
	README rewrite with repro steps; reviews -> docs/reviews
4 days	Add canonical test command: scripts/test.sh (runs ↵	Void Agent
	tests/test_loss_reweight.py; skips without torch)
4 days	loss_reweight: pass Y to forward for full logits (Karpathy nanoGPT returns ↵	Void Agent
	last-position logits when targets=None); regression test
4 days	Add tests/test_loss_reweight.py: batch determinism + weighting semantics ↵	Void Agent
	(runs in container)
4 days	loss_reweight: CPU generators for randint/randperm (CUDA generators ↵	Void Agent
	unsupported on torch 2.4); cuda-path verified
4 days	Add causal experiments: synthetic frequency-matched pair (Gemini design) + ↵	Void Agent
	loss-reweighting (Codex design); jlens_v3 output_dir + last-layer device fix
6 days	jlens_v3: chunk VJP probes (16) to fit K2200 4GB; proxy chunk fix; e2e ↵	Void Agent
	verified (last-layer cos-sim=1.0)
6 days	Add faithful J-lens (jlens_v3): W_U-probed residual Jacobian per paper; ↵	Void Agent
	both-ways comparison vs log-softmax proxy; 3-model adversarial reviews
7 days	GPT-2 J-lens: use local Shakespeare + fix summary	Void Agent

7 days	GPT-2 Small J-lens: frequency + dimensionality sampling	Void Agent

7 days	Add Pythia-70m test script	Void Agent

7 days	Fix: lr→learning_rate in configure_optimizers	Void Agent

7 days	Add dimensional starvation experiment (d_model=16,32,64,128 vs 384)	Void Agent

7 days	Add controlled frequency ablation experiment (doubled-q)	Void Agent

8 days	jlens_v2: fix import path for src/ -> root layout	Void Agent

8 days	Add jlens_v2.py: simplified gradient approach for J-lens	Void Agent

8 days	jlens.py: fix hooks — don't detach activations, preserve gradient graph	Void Agent

8 days	jlens.py: fix load_model for dict-format nanoGPT checkpoints	Void Agent

8 days	Fix jlens.py import path: src/ -> project root	Void Agent

8 days	Maxwell GPU fixes: disable bf16 SDPA, force fp32	Void Agent
	- model.py: honor config.flash flag (defaults True, False disables SDPA) - config: force dtype=float32 and flash=False for K2200 (compute 5.0) - Maxwell GPUs don't support bf16; SDPA internally uses bf16 operations
8 days	Restructure: nanoGPT at root, custom code in src/	Void Agent
	- Move model.py, train.py, configurator.py to root for nanoGPT compatibility - data/ and config/ directories at root with Shakespeare dataset prep scripts - src/jlens.py updated to import model from project root - Cleaned up stale src/config/ and duplicate src/ files - Fixed .gitignore: exclude out-shakespeare-char/ instead of raw data dirs
8 days	Initial project setup: J-lens implementation for nanoGPT	Void Agent
	- Core J-lens computation (batched per-layer, per-token gradient method) - Project README with background and experiment plan - Sync script for meru Docker container deployment - Upstream nanoGPT model code copied to src/ Architecture: Computes d(log_p(token))/d(residual_stream) averaged over corpus contexts, replicating Anthropic's Jacobian Lens technique.