summaryrefslogtreecommitdiff
path: root/design
diff options
context:
space:
mode:
authorVoid Agent <void@jayrup.hermes>2026-08-14 13:27:54 +0100
committerVoid Agent <void@jayrup.hermes>2026-08-14 13:27:54 +0100
commit39b8218a5fe660200a833c03b36afbde6c119467 (patch)
tree677bf8f72a9f0c10f8fc321446940c40eb877a7d /design
parent9fecbe58cc0750e9e39b261a672b0c6e85e6ed4e (diff)
eval: P1 diagnostic prediction-based (flagged set +209), classifier regression suite (10 tests, stub ground truth); 34 tests green
Diffstat (limited to 'design')
-rw-r--r--design/preregistration.md21
1 files changed, 21 insertions, 0 deletions
diff --git a/design/preregistration.md b/design/preregistration.md
index 30f731f..91e5036 100644
--- a/design/preregistration.md
+++ b/design/preregistration.md
@@ -144,3 +144,24 @@ pre-launch and pre-run.
selection-holed) and `last.pt` (unselected); probe + halting analyses use `last.pt`.
6. **Dead config removed:** `halt_eps` (was never read).
7. **`run_meta.json`:** python/torch/numpy versions, device, thread count recorded per run.
+
+---
+
+## Addendum 3 (2026-08-14, pre-launch — P1 diagnostic corrected)
+
+Trigger: ad-hoc verification of `src/eval.py` against ground-truth stub models (promoted to
+`tests/test_eval_classification.py`). Pre-launch; P1 operationalization corrected before any
+results were seen.
+
+1. **P1 concerns PREDICTIONS, not inputs.** A model that learned only the {2,3,5,7} sieve errs
+ on inputs n = 113–120, 139–142, 167–168, 181–186, 199–200 — cases where the first candidate
+ with no divisor ≤ 7 is composite. The signature is "predicted next prime" ∈ the flagged set,
+ NOT "input n" ∈ the flagged set. Addendum 2's phrasing ("≥3 of {121,143,169,187} wrong") was
+ ambiguous and the first implementation checked inputs — corrected.
+2. **Set extended to {121, 143, 169, 187, 209}.** The original four are the no-small-factor
+ composites ≤ 200; the probe's candidate window is actually [102, 211] (targets of
+ n ∈ [101, 200] reach 211), and 209 = 11×19 is likewise mispredicted (n = 199, 200 → pred 209,
+ target 211).
+3. **P1 thresholds (replace Addendum 2's):** ≥ 3 errors total, ≥ 3 DISTINCT flagged predictions,
+ ≥ 80% of error predictions flagged. A pure {2,3,5,7} sieve produces exactly 22 errors, 100%
+ flagged (5 distinct values). P2–P4 unchanged.