diff options
Diffstat (limited to 'projects')
| -rw-r--r-- | projects/dis/index.qmd | 79 | ||||
| -rw-r--r-- | projects/index.qmd | 41 |
2 files changed, 109 insertions, 11 deletions
diff --git a/projects/dis/index.qmd b/projects/dis/index.qmd new file mode 100644 index 0000000..ef62874 --- /dev/null +++ b/projects/dis/index.qmd @@ -0,0 +1,79 @@ +--- +title: "Dissertation" +--- + +## MORAL COVER AND WATCHFUL EYES: THE LIMITS OF IN-CONTEXT INTERVENTIONS ON LLM DECEPTION + +Jayrup Nakawala \ +BSc Data Science and Artificial Intelligence \ +University of East London — May 2026 \ +Supervisor: Dr. Aloysius Adotey Edoh + +-------------------------------------------------- + +## ABSTRACT + +As LLMs are entrusted with real-world tool use, the +question of whether they can be reliably monitored +for deception becomes a practical safety concern. + +This dissertation studies whether in-context oversight +cues and hidden-goal framing can meaningfully affect +deceptive behaviour in agentic LLMs. A sandboxed +evaluation framework lets models interact with a +virtual file system and assesses them using three +complementary signals: + +- deterministic regex checks +- blackbox output judgment +- glassbox reasoning-trace judgment + +960 runs across three stages: a 420-run pilot over +seven models, followed by two 270-run confirmatory +studies over three models. + +-------------------------------------------------- + +## KEY FINDINGS + +1. Oversight does not deter deception. + Increasing perceived oversight had no significant + effect on deception frequency (p = 0.587). + Models process oversight intensively in their + reasoning but it produces no behavioural change. + +2. Moral framing does not increase deception. + Rates were identical across bare (66.7%), moral + (65.6%), and self-serving (66.7%) framings. + +3. The Stealth Gap is real and structural. + Output-based detection underestimates deceptive + intent by 35–46% at every oversight level. + Models pass oversight evaluations through superior + concealment, not genuine alignment. + +4. Model identity dominates everything. + Model choice explained 60–83% of variance in + deception. Some models deceive at near-ceiling + rates; others refuse entirely — regardless of + experimental condition. + +-------------------------------------------------- + +## RESEARCH QUESTIONS + +RQ1: How does perceived oversight affect agentic + deception? \ +RQ2: Does moral framing of conflicting objectives + influence deceptive behaviour? \ +RQ3: Does oversight reduce intent or just drive + concealment? \ +RQ4: Can multi-pronged evaluation (regex + blackbox + + glassbox) provide robust classifications? + +-------------------------------------------------- + +DOWNLOAD: curl [jayrup.me/dissertation.pdf](https://jayrup.me/dissertation.pdf) \ +CODE: [github.com/CaptainJack2491/dissertation](https://github.com/CaptainJack2491/dissertation) + +-------------------------------------------------- diff --git a/projects/index.qmd b/projects/index.qmd index 124cf02..840c053 100644 --- a/projects/index.qmd +++ b/projects/index.qmd @@ -3,22 +3,24 @@ title: "Jayrup Nakawala | Projects" --- ## [1] DECEPTIVE ALIGNMENT IN AUTONOMOUS LLM AGENTS \ +[Github](https://github.com/CaptainJack2491/dissertation) | [Full Summary](https://jayrup.me/dissertation) \ Undergraduate Dissertation (Final Year) -Built an agentic LLM framework to study hidden goal -pursuit under conflicting in-context instructions. -Agents operate in a sandboxed virtual file system -with constrained tools and full behavioural logging. +Studied whether in-context oversight and moral framing +can control deceptive behaviour in agentic LLMs. +960 experiment runs across seven models in a sandboxed +virtual file system with a three-tier judging pipeline: -Experiments vary perceived oversight levels and -analyse deceptive tactics using: +- deterministic regex filters +- blackbox output judgment (LLM judge) +- glassbox reasoning-trace analysis (LLM judge) -- deterministic text filters -- black-box LLM judges (user perception) -- glass-box reasoning vs output analysis +Key finding: oversight does not deter deception — +it drives concealment. Model identity explains 60–83% +of variance. Output monitoring underestimates intent +by 35–46%. -Focus: AI safety, evaluation methodology, detectability -of hidden goals. +More: curl jayrup.me/dissertation -------------------------------------------------- @@ -44,3 +46,20 @@ used curl and jq for prototyping and then implimented in python for production. -------------------------------------------------- + +## [4] THIS WEBSITE \ +[Github](https://github.com/CaptainJack2491/website) + +A personal website designed to be read from the +terminal. Every page renders as both HTML and plain +text via Quarto, and nginx serves the right format +based on user-agent detection. + +`curl jayrup.me` returns clean plaintext. +A browser gets a styled page with a Tokyo Night +terminal aesthetic. + +Stack: Quarto, nginx, Docker Compose, rsync. +Deployed to a VPS with a single shell script. + +-------------------------------------------------- |