diff options
Diffstat (limited to 'blog')
| -rw-r--r-- | blog/how-this-site-works.qmd | 5 | ||||
| -rw-r--r-- | blog/index.qmd | 5 | ||||
| -rw-r--r-- | blog/jacobian-lens-frequency.qmd | 5 | ||||
| -rw-r--r-- | blog/prime-grokking-1.qmd | 5 | ||||
| -rw-r--r-- | blog/prime-grokking-2.qmd | 5 | ||||
| -rw-r--r-- | blog/prime-grokking-3.qmd | 5 |
6 files changed, 30 insertions, 0 deletions
diff --git a/blog/how-this-site-works.qmd b/blog/how-this-site-works.qmd index d142a59..45731e7 100644 --- a/blog/how-this-site-works.qmd +++ b/blog/how-this-site-works.qmd @@ -3,6 +3,11 @@ title: "How This Site Works" date: "2026-08-05" --- +<nav> +[Home](https://jayrup.me) | [Blog](https://jayrup.me/blog) | [Projects](https://jayrup.me/projects) | [CV](https://jayrup.me/cv) | [Dissertation](https://jayrup.me/dissertation) | [Git](https://git.jayrup.me) +</nav> + + ## WHY I wanted publishing to be easy, and I wanted this site to diff --git a/blog/index.qmd b/blog/index.qmd index 35710e2..2bf6775 100644 --- a/blog/index.qmd +++ b/blog/index.qmd @@ -2,6 +2,11 @@ title: "Jayrup Nakawala | Blog" --- +<nav> +[Home](https://jayrup.me) | [Blog](https://jayrup.me/blog) | [Projects](https://jayrup.me/projects) | [CV](https://jayrup.me/cv) | [Dissertation](https://jayrup.me/dissertation) | [Git](https://git.jayrup.me) +</nav> + + RANDOM THOUGHTS, EXPERIMENTS, AND SYSTEMS NOTES -------------------------------------------------- diff --git a/blog/jacobian-lens-frequency.qmd b/blog/jacobian-lens-frequency.qmd index 6a81809..e7a516d 100644 --- a/blog/jacobian-lens-frequency.qmd +++ b/blog/jacobian-lens-frequency.qmd @@ -3,6 +3,11 @@ title: "What the Jacobian Lens Measures" date: "2026-08-07" --- +<nav> +[Home](https://jayrup.me) | [Blog](https://jayrup.me/blog) | [Projects](https://jayrup.me/projects) | [CV](https://jayrup.me/cv) | [Dissertation](https://jayrup.me/dissertation) | [Git](https://git.jayrup.me) +</nav> + + *A small replication of Anthropic's J-lens, the token-frequency confound I found, and the bug I almost published.* *This is a story about trying to look inside a language model. I found something diff --git a/blog/prime-grokking-1.qmd b/blog/prime-grokking-1.qmd index 5898131..e6ac85a 100644 --- a/blog/prime-grokking-1.qmd +++ b/blog/prime-grokking-1.qmd @@ -3,6 +3,11 @@ title: "Can a Tiny Network Learn What a Prime Is?" date: "2026-08-14" --- +<nav> +[Home](https://jayrup.me) | [Blog](https://jayrup.me/blog) | [Projects](https://jayrup.me/projects) | [CV](https://jayrup.me/cv) | [Dissertation](https://jayrup.me/dissertation) | [Git](https://git.jayrup.me) +</nav> + + *I trained two tiny neural networks to compute the next prime after a number, to see whether either would discover the algorithm instead of memorizing the answers. Neither did. The way they failed is more interesting than the failure.* diff --git a/blog/prime-grokking-2.qmd b/blog/prime-grokking-2.qmd index f97dcfc..ed23fe6 100644 --- a/blog/prime-grokking-2.qmd +++ b/blog/prime-grokking-2.qmd @@ -3,6 +3,11 @@ title: "Still No Grokking: The Weight-Decay Sweep" date: "2026-08-16" --- +<nav> +[Home](https://jayrup.me) | [Blog](https://jayrup.me/blog) | [Projects](https://jayrup.me/projects) | [CV](https://jayrup.me/cv) | [Dissertation](https://jayrup.me/dissertation) | [Git](https://git.jayrup.me) +</nav> + + *Part 1 found no grokking at weight decay 1.0, and the seed replication confirmed it. So I did the obvious thing: swept the weight decay across five orders of magnitude and watched. Still no grokking. The way it diff --git a/blog/prime-grokking-3.qmd b/blog/prime-grokking-3.qmd index dab2e37..3ff0e02 100644 --- a/blog/prime-grokking-3.qmd +++ b/blog/prime-grokking-3.qmd @@ -3,6 +3,11 @@ title: "Where Exactly Is the Wall?" date: "2026-08-17" --- +<nav> +[Home](https://jayrup.me) | [Blog](https://jayrup.me/blog) | [Projects](https://jayrup.me/projects) | [CV](https://jayrup.me/cv) | [Dissertation](https://jayrup.me/dissertation) | [Git](https://git.jayrup.me) +</nav> + + *The first three phases established that a tiny network cannot grok the next-prime function, and that weight decay doesn't help. This post is the forensic part: four pre-registered diagnostics to find out where the wall |
