diff options
| author | Void Agent <void@jayrup.hermes> | 2026-08-14 13:27:54 +0100 |
|---|---|---|
| committer | Void Agent <void@jayrup.hermes> | 2026-08-14 13:27:54 +0100 |
| commit | 39b8218a5fe660200a833c03b36afbde6c119467 (patch) | |
| tree | 677bf8f72a9f0c10f8fc321446940c40eb877a7d /design | |
| parent | 9fecbe58cc0750e9e39b261a672b0c6e85e6ed4e (diff) | |
eval: P1 diagnostic prediction-based (flagged set +209), classifier regression suite (10 tests, stub ground truth); 34 tests green
Diffstat (limited to 'design')
| -rw-r--r-- | design/preregistration.md | 21 |
1 files changed, 21 insertions, 0 deletions
diff --git a/design/preregistration.md b/design/preregistration.md index 30f731f..91e5036 100644 --- a/design/preregistration.md +++ b/design/preregistration.md @@ -144,3 +144,24 @@ pre-launch and pre-run. selection-holed) and `last.pt` (unselected); probe + halting analyses use `last.pt`. 6. **Dead config removed:** `halt_eps` (was never read). 7. **`run_meta.json`:** python/torch/numpy versions, device, thread count recorded per run. + +--- + +## Addendum 3 (2026-08-14, pre-launch — P1 diagnostic corrected) + +Trigger: ad-hoc verification of `src/eval.py` against ground-truth stub models (promoted to +`tests/test_eval_classification.py`). Pre-launch; P1 operationalization corrected before any +results were seen. + +1. **P1 concerns PREDICTIONS, not inputs.** A model that learned only the {2,3,5,7} sieve errs + on inputs n = 113–120, 139–142, 167–168, 181–186, 199–200 — cases where the first candidate + with no divisor ≤ 7 is composite. The signature is "predicted next prime" ∈ the flagged set, + NOT "input n" ∈ the flagged set. Addendum 2's phrasing ("≥3 of {121,143,169,187} wrong") was + ambiguous and the first implementation checked inputs — corrected. +2. **Set extended to {121, 143, 169, 187, 209}.** The original four are the no-small-factor + composites ≤ 200; the probe's candidate window is actually [102, 211] (targets of + n ∈ [101, 200] reach 211), and 209 = 11×19 is likewise mispredicted (n = 199, 200 → pred 209, + target 211). +3. **P1 thresholds (replace Addendum 2's):** ≥ 3 errors total, ≥ 3 DISTINCT flagged predictions, + ≥ 80% of error predictions flagged. A pure {2,3,5,7} sieve produces exactly 22 errors, 100% + flagged (5 distinct values). P2–P4 unchanged. |
