From 3986b9f5e7e7efc0bd10143a860e19e1b889fe60 Mon Sep 17 00:00:00 2001 From: Void Agent Date: Fri, 31 Jul 2026 16:22:45 +0100 Subject: Add faithful J-lens (jlens_v3): W_U-probed residual Jacobian per paper; both-ways comparison vs log-softmax proxy; 3-model adversarial reviews --- reviews/2026-07-31-claude-opus-4.6.md | 38 +++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 reviews/2026-07-31-claude-opus-4.6.md (limited to 'reviews/2026-07-31-claude-opus-4.6.md') diff --git a/reviews/2026-07-31-claude-opus-4.6.md b/reviews/2026-07-31-claude-opus-4.6.md new file mode 100644 index 0000000..2ff8227 --- /dev/null +++ b/reviews/2026-07-31-claude-opus-4.6.md @@ -0,0 +1,38 @@ +# Review: Claude Opus 4.6 (agy, 2026-07-31) + +Adversarial review of the J-space replication project. Full brief: /tmp/jspace_brief.md. + +## Headline + +**The implementation computes the wrong quantity.** + +Anthropic's J-lens vectors are rows of `W_U · E[∂h_final/∂h_ℓ]` — the Jacobian stops at +the final *residual stream*, before softmax. Our code (jlens_v2.py) differentiates through +`log_softmax`, which folds in a `(1 − p(k))` factor that mechanically anticorrelates norm +with frequency. The r = −0.65 may be an artefact of this difference, not a property of the +model's representations. + +## Q1 + +The 67% causal drop and the min(V,d) rank law both follow from the softmax gradient +mechanics and linear algebra respectively — neither requires a "workspace" explanation. +The causal experiment doesn't control for the softmax saturation confound. + +## Q2 + +The cheapest kill-or-save experiment: compute J-lens BOTH ways (our `∇ log p` vs +Anthropic-faithful `∇ h_final` composed with `W_U`) on the *existing* trained model. +If the correlation vanishes with the faithful method, the thesis is dead. +~30 min of compute, zero retraining. + +## Q3 + +gpt2_jlens.py also has a norm-averaging bug: it accumulates `E[‖∇‖]` (average of norms) +rather than `‖E[∇]‖` (norm of average), which are different quantities by Jensen's +inequality. + +## Q4 + +Frame as "open confounds to control," not "refutation." We haven't faithfully replicated +their method, and we haven't addressed any of their functional experiments (steering, +verbal report, reasoning ablation). -- cgit v1.2.3