Initial project setup: J-lens implementation for nanoGPT

- Core J-lens computation (batched per-layer, per-token gradient method)
- Project README with background and experiment plan
- Sync script for meru Docker container deployment
- Upstream nanoGPT model code copied to src/

Architecture: Computes d(log_p(token))/d(residual_stream) averaged
over corpus contexts, replicating Anthropic's Jacobian Lens technique.

1 files changed, 29 insertions, 0 deletions

diff --git a/README.md b/README.md
new file mode 100644
index 0000000..e244b24
--- /dev/null
+++ b/README.md
@@ -0,0 +1,29 @@
+# J-space on nanoGPT
+
+Replicating Anthropic's Jacobian Lens ("J-space") technique on a small character-level transformer (nanoGPT).
+
+## Hypothesis
+
+If J-space (the subspace of representations readily available for verbal report) is an **architectural/structural property of transformers** rather than an emergent feature of advanced models, it should appear at all scales — including 10M-parameter char-level models.
+
+## Background
+
+Anthropic's 2026 paper "Verbalizable Representations Form a Global Workspace in Language Models" introduces the Jacobian Lens (J-lens), which computes the average linearized effect of activations on future token probabilities, averaged over many contexts. This reveals a privileged "J-space" of representations that the model can report on, modulate, and use for reasoning.
+
+Full paper: https://transformer-circuits.pub/2026/workspace/index.html
+
+## Experiments
+
+1. **J-space visualization** — Compute J-lens vectors for all vocabulary tokens at each layer. Visualize which characters/concepts enter "verbalizable space" and when.
+2. **Ablation test** — Remove J-space components vs random directions vs full activations. Measure prediction quality impact.
+3. **Training dynamics** — Save checkpoints during training, compute J-space at each, track when it crystallizes.
+4. **Capacity measurement** — Count active J-lens tokens per position.
+
+## Setup
+
+Runs on meru's Quadro K2200 (4GB VRAM) via Docker with GPU passthrough.
+
+## References
+
+- [Verbalizable Representations Form a Global Workspace in Language Models](https://transformer-circuits.pub/2026/workspace/index.html) — Anthropic, 2026
+- [nanoGPT](https://github.com/karpathy/nanoGPT) — Andrej Karpathy