1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
|
# Review: Gemini 3.1 Pro (agy, 2026-07-31)
Adversarial review of the J-space replication project. Full brief: /tmp/jspace_brief.md.
## Q1. Falsification & alternative explanations
The core claim — that Anthropic's "limited capacity workspace" is merely an artifact of
unigram token frequency and the min(V, d) linear algebra rank bound — is falsified by a
fundamental misunderstanding of both the gradient mechanics and space dimensionality.
### Why the 67% causal drop occurs (softmax gradient mechanics, not workspace falsification)
The J-lens vector at the output layer L for token k is defined as the expectation over
contexts x:
J_L(k) = E_x[ grad_{h_L} log p(k|x) ] = W_{U,k}^T - sum_j E_x[p(j|x)] W_{U,j}^T
= W_{U,k}^T - W_avg
where W_avg is the unigram probability-weighted average unembedding vector across the corpus.
When 'q' frequency is doubled by inserting it at random positions (freq_experiment.py):
1. Unigram baseline elevation: p_q increases across all context positions. In the
subtraction term -sum_j p(j|x) W_{U,j}, the component -p(q|x) W_{U,q} grows larger
across every single non-'q' position in the batch, directly canceling out W_{U,q}.
2. Logit bias inflation: to minimize cross-entropy loss on uniform random insertions,
the model increases the scalar output bias b_q, raising p(q|x) globally and driving
(1 - p(q|x)) -> 0 faster during evaluation.
Furthermore, freq_ablation.py compares a newly trained model against a pre-existing
checkpoint (out-shakespeare-char/ckpt.pt) trained by a different script with unmatched
seeds and training iterations.
### The min(V, d) rank fallacy
A matrix V in R^(V x d) has an absolute mathematical rank ceiling of min(V, d).
In our setup (V=65):
- d=16: rank 14-15 (~94% of the min(65,16)=16 ceiling)
- d=32: rank 26-31 (~90% of 32)
- d=128 and d=384: rank 65 (~100% of 65)
Our results show nanoGPT J-vectors occupy almost 100% of the ambient dimension available.
There is ZERO low-rank workspace compression in the model. In contrast, Anthropic
evaluated models where V=50,257 and d=768 or 4,096. If Anthropic's finding were a
min(V,d) linear algebra triviality, their J-space effective rank would be
min(50257, 768) = 768. Instead they observed a rank of 10-50 << 768 << V.
Our experiment proved that nanoGPT fails to form a compressed J-space, not that
Anthropic's compression finding is a linear algebra illusion.
## Q2. Most informative single experiment
Frequency-Matched Synthetic Pair Test (Contextual Predictability vs Unigram Frequency).
Design: train nanoGPT (or evaluate GPT-2) on a corpus containing two synthetic tokens,
T_struct and T_noise, with IDENTICAL unigram frequencies (e.g. exactly 0.1% each):
- T_struct: appears strictly in specific structured syntactic templates (e.g. after a
fixed 3-token trigger sequence A B C -> T_struct)
- T_noise: injected at uniform random positions
Expected outcomes:
- Under frequency-only hypothesis: identical J-lens norms across all layers.
- Under workspace/verbalizable-representation hypothesis: T_struct maintains high
J-lens norm in intermediate layers; T_noise collapses to near zero.
## Q3. Implementation & methodological audit
1. NORM OF EXPECTATION vs EXPECTATION OF NORM (critical bug):
- jlens_v2.py computes ||E_x[grad]||: averages gradient vectors first, then takes
L2 norm. At the final layer this reduces to ||W_{U,k}^T - W_avg||, stripping all
context-dependent dynamic activation variance.
- gpt2_jlens.py computes E_x[||grad||]: takes the norm on each batch step before
accumulating.
- Comparing nanoGPT to GPT-2 compares two mathematically distinct quantities.
2. Severe underpowering & silent exception suppression:
- gpt2_jlens.py: n_batches=3 with seq_len=32 -> only 96 token positions to estimate
gradients over a 50,257-token vocabulary.
- Lines 93-94: bare `except: pass` silently discards failed backward passes.
3. Residual capture point:
- jlens_v2.py registers a forward hook on model.transformer.h[layer_idx]; in nanoGPT
this captures the block output AFTER both attention and MLP residual additions.
Verify layer indices match Anthropic's definition (pre-block vs post-block).
## Q4. Scrutiny-surviving framing
"When evaluating the Jacobian Lens (J-lens) on small character-level transformers
(V=65), J-lens vector magnitudes exhibit a strong inverse correlation with unigram token
frequency (r = -0.65), and gradient norm reductions can be induced by artificially
inflating token priors. This highlights that raw J-lens norms in small-scale models are
heavily confounded by static unembedding geometry (W_{U,k} - W_avg) and baseline unigram
predictability. However, we do NOT claim to refute Anthropic's Global Workspace hypothesis
or their low-rank J-space findings (10-50 active dimensions). Because V < d in our
character-level baseline, J-vectors span the full ambient rank (~min(V, d)), demonstrating
that toy models fail to exhibit the severe subspace compression (10 << d << V) observed in
large language models rather than disproving its existence."
|
