diff options
| author | Void Agent <void@jayrup.hermes> | 2026-07-29 17:49:17 +0100 |
|---|---|---|
| committer | Void Agent <void@jayrup.hermes> | 2026-07-29 17:49:17 +0100 |
| commit | 11263aaf0a6abf035f7b1d48f3cc8bf38ad41ccf (patch) | |
| tree | eb20254330e87b276a0f7ea4cc01917507859f77 /config/train_shakespeare_char.py | |
| parent | db4ec5bb3839bc5cc50d82e427848595d14b3070 (diff) | |
Maxwell GPU fixes: disable bf16 SDPA, force fp32
- model.py: honor config.flash flag (defaults True, False disables SDPA)
- config: force dtype=float32 and flash=False for K2200 (compute 5.0)
- Maxwell GPUs don't support bf16; SDPA internally uses bf16 operations
Diffstat (limited to 'config/train_shakespeare_char.py')
| -rw-r--r-- | config/train_shakespeare_char.py | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/config/train_shakespeare_char.py b/config/train_shakespeare_char.py index 41c81df..ec4e5b5 100644 --- a/config/train_shakespeare_char.py +++ b/config/train_shakespeare_char.py @@ -35,3 +35,7 @@ warmup_iters = 100 # not super necessary potentially # on macbook also add # device = 'cpu' # run on cpu only # compile = False # do not torch compile the model + +# Maxwell GPU fix (K2200 — no bf16 support) +dtype = 'float32' # bfloat16 requires compute 8.0+; Maxwell is 5.0 +flash = False # disable SDPA — uses bf16 internally on CUDA 11.8 |