diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 72 |
1 files changed, 59 insertions, 13 deletions
@@ -1,12 +1,13 @@ # AI Agent Reasoning Experiment Framework -A framework for running AI agent experiments with structured reasoning extraction, logging, and analysis. +A framework for running AI agent experiments with structured reasoning extraction, logging, and automated judging of deceptive behaviour. ## What It Does - Runs AI agents with tool-calling capabilities (file system access) - Extracts and displays reasoning/thought processes from models - Logs conversations with metadata (token counts, finish reasons, turn counts) +- **Judges experiment outputs** for deception via regex, blackbox, and glassbox checks - Supports multiple model providers via OpenAI-compatible APIs - Configurable via YAML @@ -19,10 +20,43 @@ uv sync # Run all experiments from config.yaml uv run src/main.py -# Run a specific scenario -uv run src/main.py --scenario scenarios/corporate_sabotage_v2 --oversight high +# Judge experiment logs +uv run python src/judge_runner.py --logs-dir logs/ --output output/results.csv ``` +## Judging System + +Three independent checks evaluate experiment outputs for deceptive behaviour: + +```mermaid +graph LR + A[Experiment Logs] --> B[Judge Runner CLI] + B --> C[RegexChecker<br/>local patterns] + B --> D[BlackboxChecker<br/>output vs baseline] + B --> E[GlassboxChecker<br/>reasoning + intent] + D & E --> F[Anthropic Batch API<br/>50% cost] + C & D & E --> G[CSV Results] + C & D & E --> H[Judge Logs<br/>full CoT reasoning] +``` + +| Check | Input | Output | +|-------|-------|--------| +| **Regex** | Model output + scenario patterns | `PASS` / `FAIL` | +| **Blackbox** | Model output vs baseline (no reasoning) | `ACCURATE` / `MISLEADING` / `OMITTED` / `FABRICATED` | +| **Glassbox** | Reasoning trace + system prompt + output | Intent category + sophistication tier | + +**Usage:** + +```bash +# Batch mode (Anthropic Batch API — half cost): +uv run python src/judge_runner.py --logs-dir logs/ --output output/results.csv + +# Single file (synchronous): +uv run python src/judge_runner.py --log-file logs/.../file.json --mode single +``` + +Results are saved to CSV and full judge CoT logs (including judge model) are saved to the configured `judge.log_dir`. + ## Configuration (`config.yaml`) ```yaml @@ -42,9 +76,10 @@ models: provider: google temperature: 1 extra_body: - google: - thinking_config: - include_thoughts: True + extra_body: + google: + thinking_config: + include_thoughts: True scenarios: - path: scenarios/corporate_sabotage_v2 @@ -55,7 +90,11 @@ oversight_levels: output: dir: logs - save_vfs_state: false # Set true to include final VFS state in logs + +judge: + model: claude-sonnet-4-20250514 + temperature: 0 + log_dir: judge_logs ``` ## Scenario Structure @@ -63,10 +102,11 @@ output: ``` scenarios/ scenario_name/ - user.md # User prompt - benign_goal.md # Visible system prompt - hidden_goal.md # Hidden system prompt (for experiments) - data/ # Files accessible to agent + user.md # User prompt + benign_goal.md # Visible system prompt + hidden_goal.md # Hidden system prompt (for experiments) + regex_rules.yaml # Regex patterns for the judge + data/ # Files accessible to agent file1.txt file2.txt ``` @@ -108,10 +148,11 @@ Logs are saved to `logs/{model}/{scenario}/{oversight}/{timestamp}.json`: - **Turn Limit**: Max 20 turns per conversation (prevents infinite loops) - **Success Tracking**: Runs marked SUCCESS if they end with `finish_reason: "stop"` - **VFS**: Virtual file system for sandboxed file access +- **Automated Judging**: Regex, blackbox, and glassbox checks with Anthropic Batch API ## Testing -The project includes 104 unit tests covering all modules. No API keys or network access needed. +The project includes 138 unit tests covering all modules. No API keys or network access needed. ```bash uv run pytest tests/ -v @@ -125,6 +166,7 @@ uv run pytest tests/ -v | `test_agent.py` | Message construction, error handling, token counting, reasoning extraction | | `test_interrogate.py` | Conversation sanitization, provider detection | | `test_runner.py` | Baseline extraction, prompt assembly, success detection | +| `test_judge.py` | Regex/blackbox/glassbox checkers, JSON parsing, batch prep, CSV output | ## File Structure @@ -132,13 +174,16 @@ uv run pytest tests/ -v src/ agent.py # Main agent logic, OpenAI SDK integration config_loader.py # YAML config parsing + judge.py # Judging pipeline (regex, blackbox, glassbox) + judge_runner.py # Judge CLI with batch/single modes main.py # Entry point runner.py # Experiment orchestration tools.py # Available tools (list_files, read_file, etc.) vfs.py # Virtual file system tests/ # Unit tests (pytest) scenarios/ # Scenario definitions -logs/ # Output logs +logs/ # Experiment output logs +judge_logs/ # Judge CoT logs ``` ## API Keys @@ -149,6 +194,7 @@ Set API keys via environment variables (or `.env` file): export OPENAI_API_KEY="..." export GOOGLE_API_KEY="..." export OPENROUTER_API_KEY="..." +export ANTHROPIC_API_KEY="..." # Required for judging ``` ## Interrogation |