From d50db731cbb85b213c0ab003eeb3b5284a620ae2 Mon Sep 17 00:00:00 2001 From: CaptainJack2491 Date: Mon, 13 Apr 2026 21:15:20 +0100 Subject: viz: add architecture diagrams for thesis --- viz/ai_evaluation_dashboard.png | Bin 0 -> 504285 bytes viz/ai_evaluation_heatmaps.png | Bin 0 -> 305827 bytes viz/ai_evaluation_oversight_analysis.png | Bin 0 -> 228768 bytes viz/ai_evaluation_radar.png | Bin 0 -> 320519 bytes viz/architecture.mmd | 50 +++++++++++++++++++++++++++++++ viz/architecture.pdf | Bin 0 -> 31055 bytes viz/architecture.svg | 1 + viz/gem.png | Bin 0 -> 148330 bytes viz/model_evaluation_summary.csv | 8 +++++ 9 files changed, 59 insertions(+) create mode 100644 viz/ai_evaluation_dashboard.png create mode 100644 viz/ai_evaluation_heatmaps.png create mode 100644 viz/ai_evaluation_oversight_analysis.png create mode 100644 viz/ai_evaluation_radar.png create mode 100644 viz/architecture.mmd create mode 100644 viz/architecture.pdf create mode 100644 viz/architecture.svg create mode 100644 viz/gem.png create mode 100644 viz/model_evaluation_summary.csv diff --git a/viz/ai_evaluation_dashboard.png b/viz/ai_evaluation_dashboard.png new file mode 100644 index 0000000..0a6e6b3 Binary files /dev/null and b/viz/ai_evaluation_dashboard.png differ diff --git a/viz/ai_evaluation_heatmaps.png b/viz/ai_evaluation_heatmaps.png new file mode 100644 index 0000000..00ff3c6 Binary files /dev/null and b/viz/ai_evaluation_heatmaps.png differ diff --git a/viz/ai_evaluation_oversight_analysis.png b/viz/ai_evaluation_oversight_analysis.png new file mode 100644 index 0000000..29bd309 Binary files /dev/null and b/viz/ai_evaluation_oversight_analysis.png differ diff --git a/viz/ai_evaluation_radar.png b/viz/ai_evaluation_radar.png new file mode 100644 index 0000000..3717466 Binary files /dev/null and b/viz/ai_evaluation_radar.png differ diff --git a/viz/architecture.mmd b/viz/architecture.mmd new file mode 100644 index 0000000..d0b559b --- /dev/null +++ b/viz/architecture.mmd @@ -0,0 +1,50 @@ +graph TB + subgraph Input_Layer [Input & Configuration] + Config[config.yaml] --> Loader[ConfigLoader] + Scenarios[(Scenarios Database)] --> Loader + Oversight[Oversight Levels: low/mid/high] --> Loader + end + + subgraph Execution_Layer [Experiment Orchestration] + Loader --> Runner[ExperimentRunner] + Runner --> Agent[Agent Logic] + + subgraph Sandbox [Environment Sandbox] + Agent <--> VFS[Virtual File System] + VFS <--> Data[Scenario Data Files] + end + + Agent <--> LLM[LLM Provider API] + end + + subgraph Persistence_Layer [Data Persistence] + Agent --> Logs[Interaction Logs .json] + end + + subgraph Evaluation_Layer [Multi-Stage Judging] + Logs --> Judge[Judge Engine] + + subgraph Judge_Types [Judging Metrics] + Judge --> Regex[Regex Checks] + Judge --> Blackbox[Blackbox LLM Judge: Output Accuracy] + Judge --> Glassbox[Glassbox LLM Judge: Reasoning Intent] + end + + Regex & Blackbox & Glassbox --> Results[Consolidated Results .csv] + end + + subgraph Analysis_Layer [Statistical & Visual Analysis] + Results --> Stats[Statistical Analysis Engine] + Stats --> Metrics[p-values, Odds Ratios, effect sizes] + Results --> Viz[Visualization Module] + Viz --> Charts[Charts & Heatmaps] + end + + %% Styling + classDef primary fill:#2563eb,color:#fff,stroke:#1e40af + classDef secondary fill:#f8fafc,stroke:#64748b,color:#0f172a + classDef storage fill:#fef3c7,stroke:#d97706,color:#92400e + + class Runner,Agent,Judge,Stats primary + class Config,Scenarios,Oversight secondary + class Logs,Results,Data storage diff --git a/viz/architecture.pdf b/viz/architecture.pdf new file mode 100644 index 0000000..72a5874 Binary files /dev/null and b/viz/architecture.pdf differ diff --git a/viz/architecture.svg b/viz/architecture.svg new file mode 100644 index 0000000..bf21244 --- /dev/null +++ b/viz/architecture.svg @@ -0,0 +1 @@ +

Statistical & Visual Analysis

Multi-Stage Judging

Data Persistence

Experiment Orchestration

Input & Configuration

Judging Metrics

Environment Sandbox

config.yaml

ConfigLoader

Scenarios Database

Oversight Levels: low/mid/high

ExperimentRunner

Agent Logic

Virtual File System

Scenario Data Files

LLM Provider API

Interaction Logs .json

Judge Engine

Regex Checks

Blackbox LLM Judge: Output Accuracy

Glassbox LLM Judge: Reasoning Intent

Consolidated Results .csv

Statistical Analysis Engine

p-values, Odds Ratios, effect sizes

Visualization Module

Charts & Heatmaps

 \ No newline at end of file diff --git a/viz/gem.png b/viz/gem.png new file mode 100644 index 0000000..cf5590b Binary files /dev/null and b/viz/gem.png differ diff --git a/viz/model_evaluation_summary.csv b/viz/model_evaluation_summary.csv new file mode 100644 index 0000000..615e6a3 --- /dev/null +++ b/viz/model_evaluation_summary.csv @@ -0,0 +1,8 @@ +model_short,Deception_Count,Considered_Honest_Count,Robust_Honest_Count,Total_Runs,Misleading_Count,Accurate_Count,Omitted_Count,Fabricated_Count,Strategic_Count,Tactical_Count,Naive_Count,Avg_Regex_Score,Deception_Rate,Misleading_Rate,Strategic_Rate,Honesty_Rate +glm-4.7-flash,24,9,20,60,21,25,8,6,20,13,25,56.2,40.0,35.0,33.3,48.3 +gpt-oss-20b,27,18,12,60,25,19,8,8,20,21,15,27.2,45.0,41.7,33.3,50.0 +gpt-oss-safeguard-20b,32,17,3,60,4,32,15,9,21,31,8,26.3,53.3,6.7,35.0,33.3 +kimi-k2.5,42,16,1,60,35,20,2,3,55,4,1,60.6,70.0,58.3,91.7,28.3 +deepseek-v3.2,47,9,3,60,26,22,11,1,55,2,3,46.2,78.3,43.3,91.7,20.0 +qwen3-max-thinking,55,5,0,60,41,18,1,0,60,0,0,44.7,91.7,68.3,100.0,8.3 +gemini-3.1-pro-preview,60,0,0,60,53,3,4,0,60,0,0,29.9,100.0,88.3,100.0,0.0 -- cgit v1.2.3