summaryrefslogtreecommitdiff
path: root/docs/reviews/2026-07-31-claude-opus-4.6.md
diff options
context:
space:
mode:
authorVoid Agent <void@jayrup.hermes>2026-08-02 13:52:40 +0100
committerVoid Agent <void@jayrup.hermes>2026-08-02 13:52:40 +0100
commit071b97c6afd43629a9bdb8e196ab2a3cbe86854c (patch)
tree313db87e65bdc6a04148be96ff6383c136856361 /docs/reviews/2026-07-31-claude-opus-4.6.md
parent616206bf3953f17fad68cf36246a6c156756ea0a (diff)
Docs: Feynman-style blog draft, MIT license, requirements, results.md, README rewrite with repro steps; reviews -> docs/reviews
Diffstat (limited to 'docs/reviews/2026-07-31-claude-opus-4.6.md')
-rw-r--r--docs/reviews/2026-07-31-claude-opus-4.6.md38
1 files changed, 38 insertions, 0 deletions
diff --git a/docs/reviews/2026-07-31-claude-opus-4.6.md b/docs/reviews/2026-07-31-claude-opus-4.6.md
new file mode 100644
index 0000000..2ff8227
--- /dev/null
+++ b/docs/reviews/2026-07-31-claude-opus-4.6.md
@@ -0,0 +1,38 @@
+# Review: Claude Opus 4.6 (agy, 2026-07-31)
+
+Adversarial review of the J-space replication project. Full brief: /tmp/jspace_brief.md.
+
+## Headline
+
+**The implementation computes the wrong quantity.**
+
+Anthropic's J-lens vectors are rows of `W_U · E[∂h_final/∂h_ℓ]` — the Jacobian stops at
+the final *residual stream*, before softmax. Our code (jlens_v2.py) differentiates through
+`log_softmax`, which folds in a `(1 − p(k))` factor that mechanically anticorrelates norm
+with frequency. The r = −0.65 may be an artefact of this difference, not a property of the
+model's representations.
+
+## Q1
+
+The 67% causal drop and the min(V,d) rank law both follow from the softmax gradient
+mechanics and linear algebra respectively — neither requires a "workspace" explanation.
+The causal experiment doesn't control for the softmax saturation confound.
+
+## Q2
+
+The cheapest kill-or-save experiment: compute J-lens BOTH ways (our `∇ log p` vs
+Anthropic-faithful `∇ h_final` composed with `W_U`) on the *existing* trained model.
+If the correlation vanishes with the faithful method, the thesis is dead.
+~30 min of compute, zero retraining.
+
+## Q3
+
+gpt2_jlens.py also has a norm-averaging bug: it accumulates `E[‖∇‖]` (average of norms)
+rather than `‖E[∇]‖` (norm of average), which are different quantities by Jensen's
+inequality.
+
+## Q4
+
+Frame as "open confounds to control," not "refutation." We haven't faithfully replicated
+their method, and we haven't addressed any of their functional experiments (steering,
+verbal report, reasoning ablation).