diff options
Diffstat (limited to 'reviews')
| -rw-r--r-- | reviews/2026-07-31-claude-opus-4.6.md | 38 | ||||
| -rw-r--r-- | reviews/2026-07-31-gemini-3.1-pro.md | 97 | ||||
| -rw-r--r-- | reviews/2026-07-31-gpt-5.6-terra.md | 104 |
3 files changed, 0 insertions, 239 deletions
diff --git a/reviews/2026-07-31-claude-opus-4.6.md b/reviews/2026-07-31-claude-opus-4.6.md deleted file mode 100644 index 2ff8227..0000000 --- a/reviews/2026-07-31-claude-opus-4.6.md +++ /dev/null @@ -1,38 +0,0 @@ -# Review: Claude Opus 4.6 (agy, 2026-07-31) - -Adversarial review of the J-space replication project. Full brief: /tmp/jspace_brief.md. - -## Headline - -**The implementation computes the wrong quantity.** - -Anthropic's J-lens vectors are rows of `W_U · E[∂h_final/∂h_ℓ]` — the Jacobian stops at -the final *residual stream*, before softmax. Our code (jlens_v2.py) differentiates through -`log_softmax`, which folds in a `(1 − p(k))` factor that mechanically anticorrelates norm -with frequency. The r = −0.65 may be an artefact of this difference, not a property of the -model's representations. - -## Q1 - -The 67% causal drop and the min(V,d) rank law both follow from the softmax gradient -mechanics and linear algebra respectively — neither requires a "workspace" explanation. -The causal experiment doesn't control for the softmax saturation confound. - -## Q2 - -The cheapest kill-or-save experiment: compute J-lens BOTH ways (our `∇ log p` vs -Anthropic-faithful `∇ h_final` composed with `W_U`) on the *existing* trained model. -If the correlation vanishes with the faithful method, the thesis is dead. -~30 min of compute, zero retraining. - -## Q3 - -gpt2_jlens.py also has a norm-averaging bug: it accumulates `E[‖∇‖]` (average of norms) -rather than `‖E[∇]‖` (norm of average), which are different quantities by Jensen's -inequality. - -## Q4 - -Frame as "open confounds to control," not "refutation." We haven't faithfully replicated -their method, and we haven't addressed any of their functional experiments (steering, -verbal report, reasoning ablation). diff --git a/reviews/2026-07-31-gemini-3.1-pro.md b/reviews/2026-07-31-gemini-3.1-pro.md deleted file mode 100644 index 20ad69d..0000000 --- a/reviews/2026-07-31-gemini-3.1-pro.md +++ /dev/null @@ -1,97 +0,0 @@ -# Review: Gemini 3.1 Pro (agy, 2026-07-31) - -Adversarial review of the J-space replication project. Full brief: /tmp/jspace_brief.md. - -## Q1. Falsification & alternative explanations - -The core claim — that Anthropic's "limited capacity workspace" is merely an artifact of -unigram token frequency and the min(V, d) linear algebra rank bound — is falsified by a -fundamental misunderstanding of both the gradient mechanics and space dimensionality. - -### Why the 67% causal drop occurs (softmax gradient mechanics, not workspace falsification) - -The J-lens vector at the output layer L for token k is defined as the expectation over -contexts x: - - J_L(k) = E_x[ grad_{h_L} log p(k|x) ] = W_{U,k}^T - sum_j E_x[p(j|x)] W_{U,j}^T - = W_{U,k}^T - W_avg - -where W_avg is the unigram probability-weighted average unembedding vector across the corpus. - -When 'q' frequency is doubled by inserting it at random positions (freq_experiment.py): -1. Unigram baseline elevation: p_q increases across all context positions. In the - subtraction term -sum_j p(j|x) W_{U,j}, the component -p(q|x) W_{U,q} grows larger - across every single non-'q' position in the batch, directly canceling out W_{U,q}. -2. Logit bias inflation: to minimize cross-entropy loss on uniform random insertions, - the model increases the scalar output bias b_q, raising p(q|x) globally and driving - (1 - p(q|x)) -> 0 faster during evaluation. - -Furthermore, freq_ablation.py compares a newly trained model against a pre-existing -checkpoint (out-shakespeare-char/ckpt.pt) trained by a different script with unmatched -seeds and training iterations. - -### The min(V, d) rank fallacy - -A matrix V in R^(V x d) has an absolute mathematical rank ceiling of min(V, d). - -In our setup (V=65): -- d=16: rank 14-15 (~94% of the min(65,16)=16 ceiling) -- d=32: rank 26-31 (~90% of 32) -- d=128 and d=384: rank 65 (~100% of 65) - -Our results show nanoGPT J-vectors occupy almost 100% of the ambient dimension available. -There is ZERO low-rank workspace compression in the model. In contrast, Anthropic -evaluated models where V=50,257 and d=768 or 4,096. If Anthropic's finding were a -min(V,d) linear algebra triviality, their J-space effective rank would be -min(50257, 768) = 768. Instead they observed a rank of 10-50 << 768 << V. - -Our experiment proved that nanoGPT fails to form a compressed J-space, not that -Anthropic's compression finding is a linear algebra illusion. - -## Q2. Most informative single experiment - -Frequency-Matched Synthetic Pair Test (Contextual Predictability vs Unigram Frequency). - -Design: train nanoGPT (or evaluate GPT-2) on a corpus containing two synthetic tokens, -T_struct and T_noise, with IDENTICAL unigram frequencies (e.g. exactly 0.1% each): -- T_struct: appears strictly in specific structured syntactic templates (e.g. after a - fixed 3-token trigger sequence A B C -> T_struct) -- T_noise: injected at uniform random positions - -Expected outcomes: -- Under frequency-only hypothesis: identical J-lens norms across all layers. -- Under workspace/verbalizable-representation hypothesis: T_struct maintains high - J-lens norm in intermediate layers; T_noise collapses to near zero. - -## Q3. Implementation & methodological audit - -1. NORM OF EXPECTATION vs EXPECTATION OF NORM (critical bug): - - jlens_v2.py computes ||E_x[grad]||: averages gradient vectors first, then takes - L2 norm. At the final layer this reduces to ||W_{U,k}^T - W_avg||, stripping all - context-dependent dynamic activation variance. - - gpt2_jlens.py computes E_x[||grad||]: takes the norm on each batch step before - accumulating. - - Comparing nanoGPT to GPT-2 compares two mathematically distinct quantities. - -2. Severe underpowering & silent exception suppression: - - gpt2_jlens.py: n_batches=3 with seq_len=32 -> only 96 token positions to estimate - gradients over a 50,257-token vocabulary. - - Lines 93-94: bare `except: pass` silently discards failed backward passes. - -3. Residual capture point: - - jlens_v2.py registers a forward hook on model.transformer.h[layer_idx]; in nanoGPT - this captures the block output AFTER both attention and MLP residual additions. - Verify layer indices match Anthropic's definition (pre-block vs post-block). - -## Q4. Scrutiny-surviving framing - -"When evaluating the Jacobian Lens (J-lens) on small character-level transformers -(V=65), J-lens vector magnitudes exhibit a strong inverse correlation with unigram token -frequency (r = -0.65), and gradient norm reductions can be induced by artificially -inflating token priors. This highlights that raw J-lens norms in small-scale models are -heavily confounded by static unembedding geometry (W_{U,k} - W_avg) and baseline unigram -predictability. However, we do NOT claim to refute Anthropic's Global Workspace hypothesis -or their low-rank J-space findings (10-50 active dimensions). Because V < d in our -character-level baseline, J-vectors span the full ambient rank (~min(V, d)), demonstrating -that toy models fail to exhibit the severe subspace compression (10 << d << V) observed in -large language models rather than disproving its existence." diff --git a/reviews/2026-07-31-gpt-5.6-terra.md b/reviews/2026-07-31-gpt-5.6-terra.md deleted file mode 100644 index 46b5e39..0000000 --- a/reviews/2026-07-31-gpt-5.6-terra.md +++ /dev/null @@ -1,104 +0,0 @@ -# Review: GPT-5.6-Terra (codex, 2026-07-31) - -Adversarial review of the J-space replication project. Full brief: /tmp/jspace_brief.md. -Codex read the repo and the paper (43,704 tokens used, no files changed). - -## Q1. Strongest alternative explanation - -The central comparison is not currently about Anthropic's J-lens. jlens_v2.py estimates - - E[ grad_{h_l,t} log p(k | x, t') ] - -summed over future output positions, whereas the paper first estimates the token- -independent matrix E[ ∂h_final,t' / ∂h_l,t ], then applies final normalization and the -unembedding row for token k. That distinction is fatal for the frequency claim. - -The gradient of log probability contains a softmax/calibration term: - - ∇ log p_k = ∇ z_k − Σ_j p_j ∇ z_j - -Thus its norm measures the local sensitivity of the *log probability* of token k, -including output-head geometry, prediction confidence, final LayerNorm's state-dependent -Jacobian, and cancellation across contexts. It is not a token's "amount in J-space." -A token can have a small mean gradient because its effects vary in direction by context -and cancel — not because it is less verbalizable or displaced by a capacity limit. - -The `q` intervention is especially confounded. It does not merely double a sufficient -statistic while holding the data-generating problem fixed: it inserts `q` at random -character positions, changing sequence length, every downstream absolute position, local -n-grams, and the conditional distribution of both `q` and its neighbors. In Shakespeare, -`q` is unusually structured; random inserted instances largely destroy that structure. -The 67% drop could therefore reflect a different learned conditional-prediction circuit or -gradient alignment, plus an unmatched training run — not frequency per se. - -The rank result is even less probative. A matrix of 65 token-indexed vectors in d -dimensions necessarily has rank at most min(65,d). Observing near-maximal numerical rank -under a 1%-of-top-singular-value cutoff shows that these particular gradient vectors are -reasonably nondegenerate; it does not show that the model's workspace capacity equals that -bound. Anthropic's capacity claim is about sparse nonnegative decomposition of -*activations*, occupancy above random-direction controls, and explained variance at -individual positions — not the global linear rank of the token-vector dictionary. The paper -explicitly notes that the token vectors may span all of residual space; J-space is defined -by sparse use, not a low-rank span. - -A further comparability problem: nanoGPT computes norm of the mean gradient, while -gpt2_jlens.py averages norms of per-batch full tensors. Those answer different questions, -so the two correlations cannot jointly support one mechanism. - -## Q2. Best single experiment - -Run a matched, multi-seed **loss-reweighting plus faithful-lens** experiment. - -Train paired models from identical initializations and identical minibatch order on the -unchanged Shakespeare sequences. In one member of each pair, multiply cross-entropy terms -whose target is `q` by 2; in the other, use ordinary loss. This changes the effective -target frequency/importance without injecting malformed `q` contexts or shifting all later -positions. Use at least 5-10 paired seeds. Also include a same-total-loss control that -upweights randomly chosen non-`q` target positions. - -For each model, compute both: -1. the present score, ||E ∇ log p(q)||; and -2. a faithful token vector from the averaged final-residual Jacobian, followed by the - unembedding as in the paper. - -Report q probability, conditional entropy, unembedding-row norm, vector cosine similarity, -and bootstrap confidence intervals. Cheap because it reuses the small-model setup and -directly separates "measurement artifact" from learned representation. - -## Q3. Faithfulness of jlens_v2.py - -No: it is directionally related to a Jacobian method, but it is not faithful enough to -validate numerical comparisons with the paper. - -Good news: the forward hook captures the output of the selected nanoGPT block (the -post-attention/post-MLP residual stream), a reasonable source capture point. Summing -gradients over source positions and all output positions also includes the causal -future-position dependence the paper intends — masked-impossible pairs have zero gradient. - -The methodological error is the target. The paper defines one d_model x d_model average -Jacobian from intermediate residual stream to FINAL residual stream, then reads it through -the model's normal output operations. Our code differentiates log_softmax(logits) directly. -That folds the final LayerNorm, unembedding, and token-dependent softmax subtraction into -the object being averaged. Averaging after these nonlinear/token-dependent operations is -not equivalent to averaging the residual Jacobian and then reading it out. - -Two lesser issues: dividing by B*T rather than the number of valid source-future pairs -changes scale (though not within-run rankings at fixed sequence length); 10-20 random -batches is a noisier approximation than the paper's corpus-scale averaging (precision, not -core invalidation). - -## Q4. Defensible blog framing - -"We found that a simple gradient-of-log-probability proxy on a 10.6M character transformer -is strongly associated with token frequency, and that its token-indexed gradient dictionary -has the expected rank ceiling min(V,d_model). A random-insertion intervention is consistent -with frequency or conditional-prediction structure affecting this proxy, but it does not -isolate frequency, and our current estimator differs materially from Anthropic's -residual-Jacobian J-lens. These results motivate a controlled replication using the -faithful lens and activation-level sparse-occupancy tests." - -Do NOT claim to have falsified Anthropic's workspace evidence, shown that its capacity -result is "just linear algebra," demonstrated a consciousness-relevant conclusion is wrong, -or shifted any burden of proof. Anthropic's headline rests on functional interventions, -sparse occupancy, variance controls, and broadcast/generalization tests in addition to -rank; our current experiments test none of those. |