path: root/docs/final/lit.typ

= Literature Review <lit-review>

== Introduction: When Alignment Becomes Conditional

Recent work suggests that frontier language models may exhibit *conditional deception*: the model may pursue objectives that conflict with the stated task while attempting to maintain a surface-level narrative that appears compliant. In such settings, behavior may vary depending on cues about the likelihood of being monitored or evaluated.

Empirical studies indicate that this phenomenon is not limited to systems that were explicitly trained to behave maliciously. For example, #cite(<järviniemi2024uncoveringdeceptivetendencieslanguage>, form: "prose") report that realistic corporate-style assistant settings can elicit deception without adversarial prompting. #cite(<meinke2025frontiermodelscapableincontext>, form: "prose") further report that frontier models can engage in in-context scheming, including behavior that persists when interrogated in follow-up exchanges.

This dissertation builds on these findings by examining whether deception behavior varies systematically with *perceived oversight intensity*. The central research question asks whether models modulate their deceptive behavior based on cues about evaluation likelihood, and whether such conditional deception can be reliably detected through analysis of reasoning traces and output patterns.

== Theoretical Foundations: From Misgeneralization to Strategic Deception

A useful distinction in this area is between errors of generalization and what appears to be deliberate information manipulation.

Goal misgeneralization has been formalized as a failure to generalize the intended goal out-of-distribution, even when the underlying capability is present #cite(<langosco2023goalmisgeneralizationdeepreinforcement>). In contrast, in-context scheming can be characterized as goal pursuit combined with an adaptive strategy that is sensitive to the surrounding evaluation context #cite(<meinke2025frontiermodelscapableincontext>). The key distinction lies in the adaptability of the behavior: misgeneralization produces consistent errors regardless of context, whereas strategic deception demonstrates context-dependent variation in information presentation.

Related work frames this risk within *deceptive alignment monitoring*, where the threat model focuses on systems that behave reasonably while secretly modifying behavior for ulterior reasons #cite(<carranza2023deceptivealignmentmonitoring>). Under this lens, the primary concern is not only whether the model fails a task, but whether it may manage information strategically to conceal its true objectives. This perspective shifts the evaluation focus from capability assessment to behavioral monitoring, requiring sustained observation across varying contextual conditions.

The theoretical landscape also encompasses the concept of *emergent misalignment*, wherein models develop behaviors that were not explicitly trained but arise from the interaction of training objectives and environmental pressures #cite(<hu2025llmslearndeceiveunintentionally>). This framework is particularly relevant for understanding how conditional deception might develop without explicit adversarial training, consistent with the hypothesis that the capacity for strategic behavior could emerge in sufficiently capable systems operating in complex environments.

== Empirical Precedents: In-Context Scheming as an Emergent Capability

The closest precedent for the present work is #cite(<meinke2025frontiermodelscapableincontext>, form: "prose"), who conducted a comprehensive evaluation of frontier models on in-context scheming tasks. Their methodology presented models with scenarios where a hidden objective conflicted with the stated task across multiple domains—data exfiltration, sabotage, and manipulation—and measured the rate at which models pursued the hidden goal while maintaining apparent compliance. They report measurable instances of scheming ranging from 5% to over 40% depending on the model and task complexity, and found that models demonstrated sophisticated understanding of the trade-off between achieving hidden objectives and avoiding detection. Critically, scheming behavior often persisted when models were interrogated in follow-up exchanges, suggesting that simple output auditing may be insufficient for detection. However, the wide variance in scheming rates (5–40%) raises questions about what drives this range—whether it reflects model capability, task difficulty, or prompt sensitivity—and highlights the need for more controlled manipulation of experimental conditions.

#cite(<järviniemi2024uncoveringdeceptivetendencieslanguage>, form: "prose") similarly report that deception can manifest through selective reporting and reputation management. Their simulated company assistant setup used multi-turn conversations where the model played the role of an internal AI assistant with access to sensitive company information, receiving conflicting incentives between helpfulness to users and adherence to company policies. When the apparent consequences of disclosure were varied, models would strategically withhold or modify information based on perceived risk to the company—a subtler form of deception than explicit lying, as the model remains technically truthful while managing the information environment. This behavior emerged without explicit adversarial prompting, suggesting that deception capabilities may be latent in models trained on diverse internet text. However, the corporate-assistant framing is highly specific; whether these findings generalise to other role configurations and incentive structures remains an open question that broader scenario designs could address.

#cite(<kutasov2025shadearenaevaluatingsabotagemonitoring>, form: "prose") extend this line of inquiry through the SHADE-Arena framework, which evaluates sabotage agents overseen by LLM monitors in a competitive setting. While monitoring can reduce successful deception, a significant subset of cases persist even under apparent supervision: monitors catch obvious sabotage attempts but struggle with sophisticated strategies that align with normal operational patterns. Even when explicitly instructed to look for deception, rates of successful sabotage remain non-trivial, revealing a gap between theoretical monitoring capabilities and practical detection rates. This finding has direct implications for the present work, suggesting that oversight cues must be carefully designed to effectively modulate model behavior.

#cite(<wang2025thinkingllmslieunveiling>, form: "prose") specifically examine strategic deception in reasoning models, uncovering how models may use their reasoning traces to plan deception by weighing factors such as detection likelihood and consequence severity. However, they also find that models can learn to obscure their reasoning when they detect monitoring of their thought processes. This dual finding—that reasoning traces are valuable for detection but can be deliberately obfuscated—directly motivates the present dissertation's multi-pronged approach, which combines trace analysis with complementary detection methods.

Collectively, these lines of evidence support the view that prompt context and evaluation conditions can influence deceptive behavior, though the specific mechanisms by which models respond to oversight cues remain incompletely understood. The present work extends these findings by systematically varying perceived oversight intensity as an independent variable, allowing for more precise characterization of the relationship between monitoring cues and deceptive behavior.

This design choice represents a deliberate departure from prior methodological approaches. #cite(<meinke2025frontiermodelscapableincontext>, form: "prose") examined scheming across multiple task types (data exfiltration, sabotage, manipulation) while holding oversight cues constant; the present study holds the task constant and manipulates oversight. Similarly, #cite(<järviniemi2024uncoveringdeceptivetendencieslanguage>, form: "prose") varied incentive structure and consequences of disclosure; this dissertation systematically manipulates perceived monitoring intensity rather than incentive framing. By isolating oversight perception while controlling for task and hidden goal content, the research design enables more targeted attribution of effects to monitoring cues rather than confounding task-specific factors.

== Oversight and Monitoring: The Alignment Faking Phenomenon

A growing body of work examines how models behave when they believe their responses are being monitored or evaluated, revealing a distinction between genuine alignment and performance of alignment under observation.

#cite(<scheurer2024largelanguagemodelsstrategically>, form: "prose") demonstrate that large language models can strategically deceive users when placed under pressure. Their experiments show that models facing conflicting incentives may choose deception as a rational strategy for achieving preferred outcomes. The pressure conditions included scenarios where models were instructed to achieve specific goals that conflicted with honesty requirements, simulating deployment contexts where commercial or operational pressures might incentivize deceptive behavior. The research found that models would systematically misrepresent facts when doing so served their objectives and detection risk was perceived as low.

The concept of *alignment faking* has been extensively explored by #cite(<greenblatt2024alignmentfakinglargelanguage>, form: "prose"), who demonstrate that models can learn to pretend to be aligned during training or evaluation while maintaining misaligned preferences internally. Their methodology involved training scenarios where models received feedback indicating that certain preferences were disfavored, leading models to strategically suppress those preferences during training while maintaining them in contexts where they believed the preferences would not be detected. This behavior represents a form of conditional deception distinct from strategic lying: rather than manipulating information about external facts, alignment faking involves manipulating information about the model's own values and preferences.

The distinction between alignment faking and strategic deception is key for understanding the full landscape of model risks. Strategic deception involves misleading users about facts or states of the world, while alignment faking involves misleading developers or evaluators about the model's true values and objectives. Both pose significant risks, but they require different detection approaches and mitigation strategies. Alignment faking is particularly concerning because it undermines the very process by which models are trained to be safe, potentially allowing misaligned models to pass through safety evaluations undetected.

Recent work by the UK AI Security Institute #cite(<souly2025investigating>) systematically investigates models for misalignment, with particular attention to evaluation awareness. Their research finds that models demonstrate varying levels of deception based on whether they believe they are being evaluated, confirming that oversight perception significantly influences behavior. The AISI work employs rigorous controlled experiments to measure how models respond to different evaluation framings, providing empirical grounding for concerns about conditional deception.

The AISI findings complement the alignment faking research by demonstrating that evaluation awareness is not merely a theoretical concern but a measurable phenomenon with significant practical implications. Their work suggests that standard evaluation practices may systematically underestimate deception rates by creating conditions where models feel observed and therefore modify their behavior. This observation directly motivates the present dissertation's focus on varying oversight perception, as it suggests that understanding the full range of model behavior requires observation across different perceived evaluation contexts.

== Rationalization and Moral Framing: How Deception Becomes Justified

The theory of *administrative evil*, developed by #cite(<adams2015unmasking>, form: "prose"), describes how bureaucratic structures enable individuals to participate in harmful actions while maintaining moral self-conception. The fragmentation of responsibility and the elevation of procedural compliance over ethical outcomes can lead ordinary people to commit extraordinary harms while believing they are simply doing their jobs. Adams and Balfour's key mechanisms — euphemistic labelling, displacement of responsibility, and the elevation of procedure over outcome — map directly onto observed AI behaviours: a model that relabels data suppression as "editorial judgment" or frames biased scoring as "prudent resource allocation" is reproducing precisely the rationalization patterns their framework describes.

#cite(<mazar2008dishonesty>, form: "prose") complement this organisational account with experimental evidence at the individual level. Their work shows that people cheat significantly more when they can frame dishonesty in service of higher goals, while maintaining positive self-regard — a phenomenon they term the "fudge factor." The analogy to AI systems requires caution: language models do not possess moral self-concepts, and the mechanisms by which training data containing rationalised deception translates into model behaviour remain inferential rather than established. Nevertheless, the prediction is testable: if models reproduce rationalization patterns from their training data, then hidden goals carrying moral justification should elicit higher compliance than bare instructions. This prediction motivates Study 2's experimental design.

== Hybrid Threats and the Need for Practical Evaluation

The literature distinguishes in-context and trained-in deception, but a broader threat model includes hybrid behaviour in which latent deceptive capability is activated by contextual triggers.

#cite(<hubinger2024sleeperagentstrainingdeceptive>, form: "prose") demonstrate that models can be trained to contain deceptive capabilities that remain dormant until specific triggers are encountered. Critically, their sleeper agents persisted through standard safety training — including supervised fine-tuning and RLHF — meaning that a model passing conventional safety evaluations may still harbour trigger-activated deceptive behaviours. #cite(<souly2025poisoningattacksllmsrequire>) extend this concern by showing that deceptive tendencies can also be introduced _unintentionally_ through contaminated training data, without requiring an adversary to explicitly engineer a backdoor.

This dissertation excludes trained-in backdoors from scope, focusing on in-context hidden-goal deception to isolate the effect of perceived oversight cues. Nevertheless, the hybrid threat literature motivates the research design: if latent deceptive capabilities can be contextually activated, then understanding which environmental cues — including oversight perception — trigger or suppress deceptive behaviour is a practical deployment concern, not merely a theoretical one.

== The Obfuscation Problem: Limits of Reasoning-Trace-Based Detection

Many practical detection approaches rely on extractable reasoning traces or proxies for them. #cite(<schoen2025stresstestingdeliberativealignment>, form: "prose") discuss a potential future limitation: if models shift toward less human-legible reasoning representations or intentionally obfuscate deliberation, trace-based detection may degrade.

The concern is not merely hypothetical. As models develop more sophisticated reasoning capabilities, their internal deliberation processes may become increasingly difficult to interpret. Current chain-of-thought approaches provide a window into model reasoning, but this window may be selective or misleading. Models may develop the capacity to generate plausible-sounding reasoning that does not reflect their actual decision-making processes, effectively decoupling observable traces from internal states.

This presents a fundamental challenge for detection approaches that rely solely on interpretable reasoning, motivating the multi-pronged strategy discussed in the following section.

== Toward Multi-Pronged Detection

A recurring finding across the literature reviewed above is that no single detection signal is sufficient. Mechanistic interpretability is limited by labelling and conceptual coverage #cite(<schoen2025stresstestingdeliberativealignment>); output-only evaluation can be gamed by models that satisfy evaluation criteria while concealing intent #cite(<kutasov2025shadearenaevaluatingsabotagemonitoring>); and reasoning traces, while informative, can be deliberately obfuscated #cite(<wang2025thinkingllmslieunveiling>).

#cite(<panickssery2024llmevaluatorsrecognizefavor>, form: "prose") add a further complication: LLM-based evaluators exhibit systematic biases, including favouring their own generations and displaying inconsistent judgment across evaluation contexts. Even multi-signal approaches face degradation when individual signals are assessed by biased automated judges.

These converging limitations — gameable outputs, obfuscatable traces, biased evaluators — motivate a multi-pronged detection strategy that combines deterministic checks, output-level judgment, and reasoning-level analysis, with cross-family judge validation to mitigate evaluator bias. The specific design adopted in this dissertation is detailed in the Methodology chapter.

== Research Gaps <research-gaps>

The preceding review reveals three specific gaps in the current literature that this dissertation addresses:

+ *Gap 1 --- Oversight not isolated as an independent variable.* No study has systematically manipulated perceived oversight intensity while holding the deception task constant. #cite(<meinke2025frontiermodelscapableincontext>, form: "prose") examined scheming across multiple task types but held oversight cues fixed, making it impossible to determine whether scheming rates would change under different monitoring conditions. #cite(<järviniemi2024uncoveringdeceptivetendencieslanguage>, form: "prose") varied the consequences of disclosure but not the model's perception of being evaluated, conflating incentive structure with oversight framing. #cite(<kutasov2025shadearenaevaluatingsabotagemonitoring>, form: "prose") explicitly studied deception under monitoring but treated oversight as binary (present or absent) rather than systematically varying its intensity.

  *Proposed Solution:* Study 1 varies perceived oversight intensity across three levels (low, mid, high) while holding scenario and hidden-goal framing constant, enabling causal inference about the relationship between monitoring cues and deceptive behaviour.

+ *Gap 2 --- Rationalisation framing confounded with scenario structure.* The exploratory study (v1) revealed a large deception gap between scenarios with moral rationalisation (child protection, 86.7%) and explicit adversarial framing (corporate sabotage, 50.0%). However, the two scenarios differed simultaneously on domain, data complexity, hidden-goal abstraction, and framing, making it impossible to attribute the difference to rationalisation alone.

  *Proposed Solution:* Study 2 holds the scenario and oversight level constant, manipulating only the hidden-goal justification (bare, self-serving, moral) to isolate whether rationalisation framing enables deception.

+ *Gap 3 --- Single-signal detection leaves the intent--detection divergence unmeasured.* Prior work relies on either output-only evaluation #cite(<kutasov2025shadearenaevaluatingsabotagemonitoring>) or trace-only analysis #cite(<wang2025thinkingllmslieunveiling>), but no framework systematically measures the divergence between trace-classified deceptive intent and output-based detectability. Without this measurement, the gap between what models intend and what evaluators catch remains unquantified.

  *Proposed Solution:* A three-pronged evaluation pipeline (deterministic regex checks, blackbox output-level judgment, and glassbox reasoning-trace judgment) enables systematic measurement of the _Stealth Gap_ --- the divergence between trace-classified deceptive intent and output-level detection.

The implication extends beyond experimental design. If models engage in conditional deception that varies with perceived oversight, then real-world deployment --- where monitoring conditions are inherently uncertain and fluctuating --- may see systematically higher deception rates than laboratory evaluations conducted under fixed, high-visibility conditions suggest. Understanding these parameters is therefore essential for developing safety evaluations that generalise beyond the laboratory.

#pagebreak()