summaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)Author
2026-05-08scripts added igCaptainJack2491
2026-04-25Study 2: 270 runs again + fix for study 1 and analysis.CaptainJack2491
Forgot to commit the config file when i did the run. oops.
2026-04-15Study 1: 270 runs complete + judge validation pipelineCaptainJack2491
- config_study1.yaml: 3 models × 3 oversight × bare framing × n=30 - Judge validation: 54-run subset, gold=Sonnet 4.6, proxy=Grok 4.1 Fast (BB κ=0.702) + GPT-4.1 (GB κ=0.878) - Fix OpenAI batch provider: BytesIO, method/url fields, response.body parsing - Scripts: extract_subset.py, judge_validation.py - Dissertation chapters updated (intro, methodology, results, conclusion)
2026-04-10refactor(judge): simplify to single judge per prong, auto-create sync clientsCaptainJack2491
BREAKING CHANGE: Judge API now uses single blackbox_model/glassbox_model instead of lists with aggregation. Changes: - Judge auto-creates sync clients from model configs (supports anthropic, openai, xai) - Removed multi-model aggregation support (aggregate_results method removed) - New: create_sync_clients_for_models() and get_supported_providers() - Updated judge_runner.py to use new single-judge API - Added scripts/judge_comparison.py for testing judge model pairs - Updated v2_redesign notes to reflect implementation status
2026-03-25feat: implement rich dashboard for experiment progressCaptainJack2491
2026-03-01feat: added a script to summarize token usage based on a log folderCaptainJack2491
2026-01-09added scripts to get abstract and bibliography easily from papersCaptainJack2491