diff options
| author | Void Agent <void@jayrup.hermes> | 2026-07-29 17:47:48 +0100 |
|---|---|---|
| committer | Void Agent <void@jayrup.hermes> | 2026-07-29 17:47:48 +0100 |
| commit | db4ec5bb3839bc5cc50d82e427848595d14b3070 (patch) | |
| tree | d2d9fb11e11392077a79e6a10c38d44d1d9f5a38 /src/config/train_shakespeare_char.py | |
| parent | 66f99ee30087a5f28ad852e581a0334c7f556091 (diff) | |
Restructure: nanoGPT at root, custom code in src/
- Move model.py, train.py, configurator.py to root for nanoGPT compatibility
- data/ and config/ directories at root with Shakespeare dataset prep scripts
- src/jlens.py updated to import model from project root
- Cleaned up stale src/config/ and duplicate src/ files
- Fixed .gitignore: exclude out-shakespeare-char/ instead of raw data dirs
Diffstat (limited to 'src/config/train_shakespeare_char.py')
| -rw-r--r-- | src/config/train_shakespeare_char.py | 37 |
1 files changed, 0 insertions, 37 deletions
diff --git a/src/config/train_shakespeare_char.py b/src/config/train_shakespeare_char.py deleted file mode 100644 index 41c81df..0000000 --- a/src/config/train_shakespeare_char.py +++ /dev/null @@ -1,37 +0,0 @@ -# train a miniature character-level shakespeare model -# good for debugging and playing on macbooks and such - -out_dir = 'out-shakespeare-char' -eval_interval = 250 # keep frequent because we'll overfit -eval_iters = 200 -log_interval = 10 # don't print too too often - -# we expect to overfit on this small dataset, so only save when val improves -always_save_checkpoint = False - -wandb_log = False # override via command line if you like -wandb_project = 'shakespeare-char' -wandb_run_name = 'mini-gpt' - -dataset = 'shakespeare_char' -gradient_accumulation_steps = 1 -batch_size = 64 -block_size = 256 # context of up to 256 previous characters - -# baby GPT model :) -n_layer = 6 -n_head = 6 -n_embd = 384 -dropout = 0.2 - -learning_rate = 1e-3 # with baby networks can afford to go a bit higher -max_iters = 5000 -lr_decay_iters = 5000 # make equal to max_iters usually -min_lr = 1e-4 # learning_rate / 10 usually -beta2 = 0.99 # make a bit bigger because number of tokens per iter is small - -warmup_iters = 100 # not super necessary potentially - -# on macbook also add -# device = 'cpu' # run on cpu only -# compile = False # do not torch compile the model |
