diff options
| author | Void Agent <void@jayrup.hermes> | 2026-07-29 17:47:48 +0100 |
|---|---|---|
| committer | Void Agent <void@jayrup.hermes> | 2026-07-29 17:47:48 +0100 |
| commit | db4ec5bb3839bc5cc50d82e427848595d14b3070 (patch) | |
| tree | d2d9fb11e11392077a79e6a10c38d44d1d9f5a38 /src/finetune_shakespeare.py | |
| parent | 66f99ee30087a5f28ad852e581a0334c7f556091 (diff) | |
Restructure: nanoGPT at root, custom code in src/
- Move model.py, train.py, configurator.py to root for nanoGPT compatibility
- data/ and config/ directories at root with Shakespeare dataset prep scripts
- src/jlens.py updated to import model from project root
- Cleaned up stale src/config/ and duplicate src/ files
- Fixed .gitignore: exclude out-shakespeare-char/ instead of raw data dirs
Diffstat (limited to 'src/finetune_shakespeare.py')
| -rw-r--r-- | src/finetune_shakespeare.py | 25 |
1 files changed, 0 insertions, 25 deletions
diff --git a/src/finetune_shakespeare.py b/src/finetune_shakespeare.py deleted file mode 100644 index 148a4c4..0000000 --- a/src/finetune_shakespeare.py +++ /dev/null @@ -1,25 +0,0 @@ -import time - -out_dir = 'out-shakespeare' -eval_interval = 5 -eval_iters = 40 -wandb_log = False # feel free to turn on -wandb_project = 'shakespeare' -wandb_run_name = 'ft-' + str(time.time()) - -dataset = 'shakespeare' -init_from = 'gpt2-xl' # this is the largest GPT-2 model - -# only save checkpoints if the validation loss improves -always_save_checkpoint = False - -# the number of examples per iter: -# 1 batch_size * 32 grad_accum * 1024 tokens = 32,768 tokens/iter -# shakespeare has 301,966 tokens, so 1 epoch ~= 9.2 iters -batch_size = 1 -gradient_accumulation_steps = 32 -max_iters = 20 - -# finetune at constant LR -learning_rate = 3e-5 -decay_lr = False |
