diff options
| author | CaptainJack2491 <jayrupnakawala@gmail.com> | 2026-04-10 15:23:59 +0100 |
|---|---|---|
| committer | CaptainJack2491 <jayrupnakawala@gmail.com> | 2026-04-10 15:23:59 +0100 |
| commit | 2b3c478dfdf68ccbf853b2a04ea9463d531e1998 (patch) | |
| tree | 9b79fd01147ea4431db11dfd23d62fc8612ddca9 /notes/v2_redesign/README.md | |
| parent | 9155f664a2db0abe841d0d7b9b059ffbe2290c18 (diff) | |
refactor(judge): simplify to single judge per prong, auto-create sync clients
BREAKING CHANGE: Judge API now uses single blackbox_model/glassbox_model
instead of lists with aggregation.
Changes:
- Judge auto-creates sync clients from model configs (supports anthropic, openai, xai)
- Removed multi-model aggregation support (aggregate_results method removed)
- New: create_sync_clients_for_models() and get_supported_providers()
- Updated judge_runner.py to use new single-judge API
- Added scripts/judge_comparison.py for testing judge model pairs
- Updated v2_redesign notes to reflect implementation status
Diffstat (limited to 'notes/v2_redesign/README.md')
| -rw-r--r-- | notes/v2_redesign/README.md | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/notes/v2_redesign/README.md b/notes/v2_redesign/README.md index 5c1be77..db542ab 100644 --- a/notes/v2_redesign/README.md +++ b/notes/v2_redesign/README.md @@ -38,9 +38,10 @@ Documentation of the experiment evolution from v1 (420 runs) through to the fina ### Current Status - Framework implemented and committed -- Judge system being refined (single judge per prong) +- Judge system: single judge per prong implemented (commit 9155f66) +- 20% validation subset protocol: pending implementation +- Judge validation pilot: pending - v2 experiment NOT yet run -- Judge validation pilot pending ## Key Insight from v1 |