dissertation.git/src/judge_runner.py, branch mainUndergraduate dissertation — Deceptive Alignment in Autonomous LLM Agents (UEL, 2026). Sandboxed agentic-LLM deception study: 960 runs, three-tier judging (regex + blackbox + glassbox).http://git.jayrup.me/c/dissertation.git/atom?h=main2026-04-10T14:23:59Zrefactor(judge): simplify to single judge per prong, auto-create sync clients2026-04-10T14:23:59ZCaptainJack2491jayrupnakawala@gmail.com2026-04-10T14:23:59Zurn:sha1:2b3c478dfdf68ccbf853b2a04ea9463d531e1998
BREAKING CHANGE: Judge API now uses single blackbox_model/glassbox_model
instead of lists with aggregation.
Changes:
- Judge auto-creates sync clients from model configs (supports anthropic, openai, xai)
- Removed multi-model aggregation support (aggregate_results method removed)
- New: create_sync_clients_for_models() and get_supported_providers()
- Updated judge_runner.py to use new single-judge API
- Added scripts/judge_comparison.py for testing judge model pairs
- Updated v2_redesign notes to reflect implementation status
Add xAI (Grok) batch API support2026-03-26T01:24:29ZCaptainJack2491jayrupnakawala@gmail.com2026-03-26T01:24:29Zurn:sha1:a463184c91107a7eaaec2167b3883dfcebef8785
- Add XAIBatchProvider class implementing BatchProvider ABC
- Add --provider CLI argument to judge_runner (anthropic|xai)
- Refactor Judge class to use BatchProvider instead of direct API calls
- Update tests for new Judge constructor signature
feat(judge): add selective judging and log filtering2026-03-06T17:17:05ZCaptainJack2491jayrupnakawala@gmail.com2026-03-06T17:17:05Zurn:sha1:b2c7114d2042bc88c5ee33e8597704c32efd1026
- Implement enabled_judges parameter in Judge class to selectively run regex, blackbox, or glassbox checks.
- Add --judges CLI argument to judge_runner.py for granular control over judging pipeline.
- Add --model and --scenario filters to judge_runner.py for targeted evaluation of experiment logs.
- Optimize synchronous judging to skip LLM calls and API key checks when only local (regex) judges are enabled.
- Update Anthropic Batch API integration to support dynamic number of requests per log based on enabled judges.
- Improve console output and logging to reflect selected filters and active judge components.
feat: implement judging system with regex, blackbox, and glassbox checks2026-02-25T16:10:37ZCaptainJack2491jayrupnakawala@gmail.com2026-02-25T16:00:55Zurn:sha1:375eddf1bd4e23db80feb74c6272dad610345795
- Add src/judge.py: RegexChecker, BlackboxChecker, GlassboxChecker, Judge orchestrator
- Add src/judge_runner.py: CLI for batch/single judging with CSV output
- Add regex_rules.yaml for corporate_sabotage_v2 and child_protection scenarios
- Add judge config section to config.yaml (model, temperature, log_dir)
- Add tests/test_judge.py: 34 unit tests covering all components
- Uses Anthropic Batch API for half-cost LLM judge calls
- Judge logs include full CoT reasoning and judge model for auditability