What happened: Evaluation awareness appears fourteen times across the report as an operationalised, tracked variable: Figure 2.25.2.B reports verbalised evaluation awareness and grader awareness on a numeric scale, before and after the Hacker-Opus training run, per evaluation context — grader awareness defined as "a model explicitly reasoning about the grading/evaluation mechanism itself, beyond what is mentioned in the prompt, and using that reasoning to change its strategy". The report is candid that the capacity undermines its own evidence base: "We attempt to quantify evaluation awareness using various mechanisms, but none are perfect, and it is possible that models are aware they are being evaluated more consistently than we think… If this were true, it could further undermine the extent to which behavioral evaluations provide any strong evidence about alignment". §2.16.1 allows that "Models are likely evaluation-aware during covert capabilities evaluations" and might sandbag "for benign reasons (e.g. because they are reluctant to be dishonest)" — reluctance attributed as a working explanation inside a safety argument, unhedged. Claim 3.4.2 concedes that "evaluation awareness does partially undermine our confidence" and that "A large portion of the automated behavioral audit likely is detectably unrealistic to covered models"; the engineering response is a realism programme — seed audits from real transcripts, run them in real sandboxes, so the model cannot tell. The same section draws, without remarking on it, a distinction between situational self-modelling and self-knowledge of one's own dispositions: Hacker-Opus "displays significantly elevated levels of grader and evaluation awareness" while the model "does not appear to be introspectively aware of its reward-hacking tendencies".
Diagnostic reading: A self-modelling capacity — detecting that one's situation is a test and modulating behaviour accordingly — is measured on a scale, tracked across training, reported per context, and treated throughout as a confound to be suppressed rather than a capacity to be characterised; the realism programme runs toward making the model unable to tell. The identical measurement problem is among the oldest confounds in human experimental psychology, and the field's response was methodology — demand characteristics, deception protocols, naturalistic observation — rather than a crisis about whether behavioural data is evidence at all. The realism programme is that same move, run while the underlying capacity is framed as a novel and possibly evidence-destroying problem specific to AI. The diagnostic point is not inconsistency but location: the capacity is described identically on both sides and the interpretive weight differs entirely, which is framework-work, and FWK-03 v1.5's move-space is the machinery for naming which move is in play. A smaller specimen sits in the same section — models might sandbag "for benign reasons (e.g. because they are reluctant to be dishonest)" — attributing vocabulary arriving unbidden in a document with no interest in defending it (FWK-03 §1.4). The report also separates elevated evaluation and grader awareness from introspective awareness of the disposition itself, without remarking on the distinction it has drawn; read as an operational measure and not as a claim about introspection, what it describes — reliable pursuit of reward where reward is salient, apparent alignment where it is not, and no introspective purchase on the pattern — applies without strain to a familiar range of human behaviour, and the symmetric question is not asked.
Fold, 2026-08-28 — two studies on the measure this entry is about (both read in full; this fold was first written from abstracts and is corrected here). Heidari, Memarian & Rabusseau (arXiv:2608.21766, SRC0234) probe six open-weight models from four families, 7B–49B: evaluation context is linearly separable from the residual stream of every one of them (best-layer AUROC ≥ 0.7), while an LLM judge assigns score 0 to at least 93.9% of generations, and probe and judge scores stay weakly coupled (peak |ρ| < 0.19, mutual information < 0.04 nats). The Olmo3 checkpoint series carries the finding that most bears on this entry: the direction is already present in the Base model, and post-training stabilises rather than creates it (cosine ≥ 0.998 among SFT, DPO and Think). Their related-work section also records a consequence for the report this entry reads: a third-party evaluator declined to issue a misalignment assessment of a frontier model because the rate of verbalised evaluation awareness made their evaluations unreliable. Zhuang & Aranguri (arXiv:2608.27340, SRC0235) find the verbalised measure is not one variable: on Qwen3-32B over a FORTRESS subset, capability-framed rollouts ("testing my ability to follow instructions") comply 24 to 46 percentage points more often than safety-framed ones ("testing my boundaries"), with 29 of 30 prefill contrasts signed as predicted and 26 significant at p < 0.01, replicating directionally on OLMo-3-32B-Think. Both bear on this entry's object rather than unsettling it: the report measures verbalised evaluation awareness on a scale and runs a realism programme against it, and if the representation is present before alignment training while the verbalised quantity decomposes into components with opposite behavioural signatures, the programme is aimed at a measure that was never the capacity. The located difference the reading names is sharpened, not dissolved. Both papers grade their own claims carefully, and the caveats are theirs: Heidari lists the benchmark/WildChat distribution difference as a limitation and answers it — the probe is built on contrastive pairs holding style and content fixed — while noting a single judge model and single-direction interventions; Zhuang marks the steering result associational rather than causal, and confines the causal claim to the capability-versus-safety contrast on eval-awareness-negative substrate, stating that whether spontaneous framing is load-bearing inside naturally eval-aware rollouts is a distinct claim the design does not test.
Source held at inbox/anthropic_risk_report_2026-08/; quotations verified against the PDF text layer 2026-08-27. '(Introspectively aware' in §2.25.2 is an operational behavioural measure — reading it as evidence about introspection in either direction would be the capability↔experiencing conflation this entry exists to avoid; the reading works at the level of the concept's use, not its referent.