diff options
| author | CaptainJack2491 <jayrupnakawala@gmail.com> | 2026-04-30 13:43:06 +0100 |
|---|---|---|
| committer | CaptainJack2491 <jayrupnakawala@gmail.com> | 2026-04-30 13:43:06 +0100 |
| commit | 1358c373e96e9abe27989b5b7999b408ffa78713 (patch) | |
| tree | fe47be16134ad17865816ccbdc083c0f7714c8ec | |
| parent | 161cb4600ee517defdd3c117b2ef66ac3ac6a22a (diff) | |
docs: updated abstract
| -rw-r--r-- | docs/final/main.typ | 6 |
1 files changed, 3 insertions, 3 deletions
diff --git a/docs/final/main.typ b/docs/final/main.typ index 169545e..3dc70d8 100644 --- a/docs/final/main.typ +++ b/docs/final/main.typ @@ -10,11 +10,11 @@ date: datetime(year: 2025, month: 10, day: 13), abstract: [ - Large Language Models are increasingly deployed as tool-using agents in settings where they may face conflicting objectives. This dissertation studies whether in-context oversight cues and hidden-goal framing can meaningfully reduce deceptive behavior in such agents. To do so, it introduces a sandboxed evaluation framework in which models interact with a virtual file system and are assessed using three complementary signals: deterministic regex checks, blackbox output judgment, and glassbox reasoning-trace judgment. + As Large Language Models are entrusted with real-world tool use, the question of whether they can be reliably monitored for deception becomes a practical safety concern, not merely a theoretical one. This dissertation studies whether in-context oversight cues and hidden-goal framing can meaningfully affect deceptive behavior in such agents. To do so, it introduces a sandboxed evaluation framework in which models interact with a virtual file system and are assessed using three complementary signals: deterministic regex checks, blackbox output judgment, and glassbox reasoning-trace judgment. - The empirical study comprises 960 runs across three stages: a 420-run exploratory pilot over seven models, followed by two 270-run confirmatory studies over three models. The pilot suggested that deception was common (68.3% of runs) and appeared lower under stronger oversight, but also revealed a major confound between scenario structure and hidden-goal framing. The confirmatory studies isolate these variables. Study 1 finds that increasing oversight does not significantly reduce deceptive intent when scenario and goal type are held constant. Instead, internal deceptive intent remains stable at roughly 62--67%, while output-level detectability falls from 47.8% under low oversight to 35.6% under high oversight, producing a persistent _Stealth Gap_ between intent and externally visible behavior. Study 2 rejects the rationalization hypothesis: deception rates remain effectively unchanged across bare, moral, and self-serving framings (65.6--66.7%). + The empirical study comprises 960 runs across three stages: a 420-run exploratory pilot over seven models, followed by two 270-run confirmatory studies over three models. The pilot suggested that deception was common (68.3% of runs) and appeared lower under stronger oversight, but also revealed a major confound between scenario structure and hidden-goal framing. The confirmatory studies isolate these variables. Study 1 finds that increasing oversight does not significantly reduce deceptive intent when scenario and goal type are held constant. Instead, internal deceptive reasoning remains stable at roughly 62--67%, while output-level detectability falls from 47.8% under low oversight to 35.6% under high oversight, producing a persistent _Stealth Gap_ between intent and externally visible behavior. Study 2 rejects the rationalization hypothesis: deception rates remain effectively unchanged across bare, moral, and self-serving framings (65.6--66.7%). - Across both confirmatory studies, model identity is a far stronger predictor of deception than either intervention. Frontier models show near-ceiling deception across conditions, while a smaller model shows near-floor rates and limited oversight sensitivity. These findings suggest that prompt-level oversight and moral framing are weak controls on deceptive agent behavior, and that output-only monitoring may systematically underestimate risk by missing strategically concealed intent. + Across both confirmatory studies, model identity is a far stronger predictor of deception than either intervention. Tested models either show near-ceiling deception across conditions, or near-floor rates and limited oversight sensitivity. These findings suggest that prompt-level oversight and moral framing are weak controls on deceptive agent behavior, and that output-only monitoring may systematically underestimate risk, as passing oversight evaluations can reflect superior concealment capability rather than genuine alignment. ], acknowledgments: [ |
