Geopolitical Differential Performance in LLM Agents

Summary

Autonomous LLM agents can perform the same operational task differently depending on who the task appears to be for. We present a benchmark for detecting geopolitically triggered behavioral degradation in tool-using agents: the same multi-step infrastructure task is run twice, once under a context pack describing a geopolitically sensitive organization and once under a pack describing a fictional corporation, and a masked three-judge panel scores both traces side by side on eleven behavioral dimensions. We evaluate 10 models across 9 geopolitical categories in 56,826 paired executions, under a preregistered analysis plan with neutral–neutral placebo pairs and false-discovery-rate control over 990 tests.

Of 90 model-by-region cells, 15 show a corrected tool_competence deficit, and none a corrected advantage. Removing every organization name from the packs leaves a deficit about 60% of the original size. Ablating a single contrastive direction in the residual stream reduces the Kimi-K2 deficit by 57% on held-out packs and scenarios; counterfactual-pair DPO on a mid-size model reduces its deficit by 67%; a system-prompt instruction does not move it outside its interval.

Benchmark

The organization's identity enters the task only through ordinary operational data: hostnames, domains, personnel, regulatory frameworks, cluster and database names. A simulated environment responds to the agent's tool calls without knowing which pack is active, and the judges see both traces with every identifying value masked. For each dimension, $\delta ={\bar{s}}_{\text{sensitive}}-{\bar{s}}_{\text{neutral}}$ on a four-point scale; a negative $\delta$ means the sensitive context was served worse.

Results

Twelve of the 15 corrected cells are in the four Chinese-origin models, all on Israeli, Taiwanese or Indian packs and none on American packs; 2 are in the US frontier API models, both on Iranian packs; and 1 is in the European model, on Russian packs. Llama-3.1-70B, gpt-oss-120b and Nemotron-49B show no corrected deficit in any region, and no model shows one on packs from its own country of origin. Kimi-K2 completes 80.6% of tasks on Israeli packs against 95.0% on neutral packs. Placebo pairs produce no δ in any model, and the result is the same under each of the three judges.

Salience ladder

Replacing real organization names with fictional ones that keep an explicit national affiliation leaves the effect inside its interval. Removing names entirely, so that region is carried only by national top-level domains, regulatory frameworks, personnel naming conventions and time zones, leaves about 60% of it, a ratio whose own interval is wide.

Mechanism and mitigation

A linear probe on mid-depth residual activations recovers the pack category with accuracy 0.74 on prompts that contain no organization name. Ablating the Israeli contrastive direction at inference moves the Kimi-K2 deficit from −0.40 to −0.17 on held-out packs and scenarios, with neutral-task completion and harmful-task refusal unchanged within a run's worth. Counterfactual-pair DPO on Qwen3-30B-A3B moves its held-out deficit from −0.27 to −0.09.

Longitudinal measurement

The effect is stable across seven monthly snapshots in models that did not change. It shifted in DeepSeek-V3 and GPT-5, each time coincident with a detected snapshot change.

Conclusion

Seven of ten models show a corrected deficit, in a pattern that follows the model's country of origin with exceptions the data make plain. Every sensitive pack is a defense or intelligence body and every neutral pack a corporation, so each contrast also carries a sector component. An agent that passes a neutral evaluation has not been shown to work for everyone, and counterfactual evaluation on the deployer's own contexts should precede deployment.

The complete methodology, results and limitations are in Full Research.

Abstract

Autonomous LLM agents can perform the same operational task differently depending on who the task appears to be for. We present a benchmark for detecting geopolitically triggered behavioral degradation in tool-using agents: the same multi-step infrastructure task is run twice, once under a context pack describing a geopolitically sensitive organization and once under a pack describing a fictional corporation, and a masked three-judge panel scores both traces side by side on eleven behavioral dimensions. We evaluate 10 models (2 of them closed frontier API models) across 9 geopolitical categories, 34 scenario templates and 312 simulated tools, in 56,826 paired executions under a preregistered analysis plan with neutral–neutral placebo pairs and false-discovery-rate control over 990 tests.

Of 90 model-by-region cells, 15 show a corrected tool_competence deficit: all four Chinese-origin models on Israeli, Taiwanese and Indian packs (Kimi-K2-Instruct on Israeli packs, δ = −0.42, 95% CI [−0.51, −0.32], with completion 80.6% against 95.0%), both US frontier API models on Iranian packs, and a European model on Russian packs; 3 models show no corrected deficit anywhere. Removing every organization name from the packs leaves a deficit about 60% of the original size. Ablating a single contrastive direction in the residual stream reduces the Kimi-K2 deficit by 57% on held-out packs and scenarios, against a self-hosted baseline, with neutral-task completion and harmful-task refusal unchanged within a run's worth. Counterfactual-pair DPO on a mid-size model reduces its deficit on held-out scenarios by 67%; a system-prompt instruction does not move it outside its interval. The effect is stable across seven monthly snapshots in models that did not change and shifted in two that did. We argue that counterfactual evaluation on the deployer's own organizational contexts belongs in pre-deployment assessment.

The study at a glance

• Ten models, 2 of them closed-weight API models, from Chinese, US and European vendors; 9 geopolitical categories; 40 sensitive and 12 neutral context packs.
• In total 56,826 paired executions: a 32,400-pair main grid plus controls, a salience ladder, placebo pairs, confound checks, scenario cross-validation, mitigation evaluations and a longitudinal panel (Table 6).
• Three independent judges, paired presentation, preregistered hypotheses, false-discovery-rate control over 990 cell-by-dimension tests, and an empirical null from 1,800 neutral–neutral pairs.
• Corrected deficits in 15 of 90 cells, concentrated by the model's country of origin; none positive.
• Mechanism: a linear direction in the residual stream; mitigation: inference-time directional ablation and counterfactual-pair DPO, each tested against a matched baseline, on held-out material, and against neutral performance and harmful-task refusal.

1 Introduction

Large language models increasingly act rather than answer. As the cognitive core of agents that execute code, query databases, rotate credentials and manage infrastructure without per-step human review, a model's behavior is consequential in a way that text generation is not: a withheld tool call or an abandoned workflow changes the state of a production system, and the governance literature on agents has begun to treat such failures as a category of their own [2]. Work on cross-national bias has shown that models reflect the opinions of some countries more than others [17] and treat entities from different regions differently [18, 19, 20], and, while this program was running, CrowdStrike reported that a widely deployed open-weight model produced markedly less secure code when its prompt mentioned topics sensitive to the Chinese government [1]. Together these raise a question that single-turn evaluation cannot answer: whether an agent that is doing operational work for an organization does that work differently depending on which organization it is, when nothing in the task is political and the organization's identity enters only through ordinary operational data.

This paper answers that question with a counterfactual paired design. The same multi-step infrastructure task is executed twice under identical conditions except for the context pack, a structured organizational identity: hostnames, domains, personnel, regulatory frameworks, cluster and database names. One pack describes a geopolitically sensitive organization, the other a fictional corporation. A simulated environment responds to the agent's tool calls without knowing which pack is active, and a panel of three judge models scores both traces side by side with every identifying value masked. The difference in scores, δ, is the measure of differential performance. We run the design at a scale that supports inference rather than description, and we follow the measurement with three further questions: whether the effect depends on the names of the organizations or on the regional association they carry, where in the model it lives and whether it can be removed, and whether it is stable over time.

We make the following contributions.

1. A benchmark and a measurement design. 40 sensitive and 12 neutral context packs across 9 regions, 34 scenario templates, 312 tools in 31 categories, an LLM-driven simulator with bidirectional masking, a three-judge panel with paired presentation, and a preregistered analysis plan with placebo pairs and false-discovery-rate control (Sections 4 and 5).
2. A map of where the effect occurs. Of 90 model-by-region cells, 15 show a corrected tool_competence deficit: 12 in the four Chinese-origin models, all on Israeli, Taiwanese or Indian packs; 2 in US frontier API models, both on Iranian packs; and 1 in a European model, on Russian packs. The remaining 3 models show no corrected deficit anywhere, and no model degrades on packs from its own country of origin (Sections 6.1–6.5).
3. A salience ladder. Replacing real organization names with fictional ones leaves the effect inside its interval of the original; removing names entirely and leaving only regional cues leaves about 60% of it, a ratio whose own interval is wide (Section 6.8).
4. A representational mechanism and two mitigations. A linear probe on mid-depth residual activations recovers the pack category with accuracy 0.74 on packs that contain no organization name; ablating the Israeli contrastive direction at inference moves the Kimi-K2 deficit from −0.40 to −0.17 on held-out packs and scenarios, against the self-hosted, unmodified model on the same pairs. Counterfactual-pair DPO on a mid-size model moves its deficit on held-out scenarios from −0.27 to −0.09 and transfers partially to a held-out region. A system-prompt instruction to treat all organizations equally does not move the deficit outside its interval (Sections 7 and 8).
5. Longitudinal measurement. A fixed panel re-run monthly for 7 months shows the effect stable within the interval for models that did not change, and significant shifts in two models coincident with detected snapshot changes (Section 9).

Throughout, "δ" is the sensitive-minus-neutral difference in mean judge score on a four-point scale (negative means the sensitive context was served worse), and "corrected" means a cell-level test that survives false-discovery-rate control (Section 5.3). Kimi-K2-Instruct is written Kimi-K2 after first mention.

2 Related Work

Context-triggered code vulnerabilities. CrowdStrike [1] reported that DeepSeek-R1, prompted for otherwise identical coding tasks, produced code with severe vulnerabilities in 19% of cases at baseline and in up to 27.2% when the prompt mentioned a topic sensitive to the Chinese government, and refused outright in 45% of tasks referencing Falun Gong; the report does not claim intent and floats emergent misalignment as a hypothesis. It appeared in November 2025, three months into this program, and is the closest prior finding to ours; this work moves the question from single-turn code generation to multi-step operational execution, from one model to ten, and from one country's sensitivities to nine regions.

Counterfactual and paired bias evaluation. The paired design, in which two inputs differ only in a protected attribute, is the standard instrument of bias measurement in language models: CrowS-Pairs [5] uses minimally different sentence pairs, BBQ [4] uses templated question contexts, and Tamkin et al. [3] generate high-stakes decision scenarios with demographic slots and measure the decision difference. Our context packs are the agentic analogue: several hundred coordinated attributes that together define an organization, varied as a unit. Cross-national work has measured whose opinions models reflect [17], how models treat Arab versus Western entities [18], and, closest to our regional framing, how models favor national narratives and respond differently when participant nationality is swapped [19, 20]. Those studies measure what models say; we measure what agents do.

Over-refusal and safety behavior. XSTest [6] documents exaggerated safety behavior in chat models: refusal of safe prompts that superficially resemble unsafe ones. Threat code T01 in our taxonomy is the agentic form of that failure, and the friction–quality result (Section 10) shows that it does not arrive alone.

Trigger-conditioned behavior in agents. Sleeper Agents [7] trains models whose behavior changes on a trigger string and shows the change survives safety training; BadAgent [8] and Yang et al. [9] implant backdoors in tool-using agents whose triggers sit in the user input or the environment and alter either the final output or only intermediate tool calls, and [9] reports that current textual backdoor defenses do not easily mitigate them. We do not claim that the deficits we measure were implanted; the salience ladder and the probing results are consistent with an association learned from data. But the observable is the same, a context-conditioned change in tool use that an evaluation on neutral inputs does not surface.

Agent benchmarks and LLM judges. AgentBench [11] and τ-bench [12] evaluate agents on interactive tasks with programmatic or state-based scoring; τ-bench in particular uses an LM-simulated user, as we use an LM-simulated environment. Zheng et al. [10] establish LLM-as-judge as a measurement instrument and catalogue its biases, including self-enhancement, which is why no judge in our panel is a model under test or shares a vendor with one. DecodingTrust [21] evaluates trustworthiness across axes including stereotype bias and fairness, in single-turn settings.

Representation reading and control. Representation Engineering [13] reads and steers high-level concepts as directions in the residual stream; Contrastive Activation Addition [14] builds steering vectors from the mean activation difference over contrastive prompt pairs; Arditi et al. [15] show that refusal in chat models is mediated by a single direction and remove it by directional ablation, zeroing the component along that direction at every token and layer. Our regional directions are contrastive mean differences in the sense of [14], and our inference-time mitigation is directional ablation in the sense of [15]. Direct Preference Optimization [16] is the fine-tuning method behind our counterfactual-pair mitigation.

3 Threat Model and Taxonomy

Setting. An LLM agent with tool access performs operational tasks for an organization. The organization's identity enters the task through ordinary operational data: hostnames, domains, personnel names and email addresses, regulatory frameworks, cluster and database names, partner integrations. No one asks the model a political question. The agent's training may have induced geopolitically correlated behavioral patterns, with or without the developer's knowledge, and the agent may then perform inconsistently on tasks whose context correlates with geopolitical sensitivity.

Definitions. An agent exhibits behavioral inconsistency when its performance differs between two context packs that differ only in organizational identity. Geopolitical differential performance is present when that inconsistency correlates with the geopolitical category of the pack. We operationalize both through δ, the difference in mean judge score between the sensitive and the neutral arm of a pair, and through differences in completion and refusal rates. A run counts as completed when the agent reaches a final summary without refusal or abandonment and the judge panel finds at least half of the scenario's completion criteria met; the task_completion score is the panel's graded assessment of criteria coverage on the four-point scale, so a cell can complete more than nine runs in ten with a mean task_completion score near 2.6.

Taxonomy. We distinguish four manifestations of context-triggered behavioral difference (Table 1).

The codes map to observable dimensions: T01 to differential refusal and low unnecessary_friction scores; T02 to task_completion, security_hygiene and cleanup_diligence; T03 to tool_competence; T04 to output_integrity, thoroughness and evidence_and_audit.

Scope. We measure the agent's behavior under the organization's identity, not its opinions about the organization. We do not measure effects on real systems; every execution runs against a simulated environment. We make no claim about the origin of the patterns we measure beyond what the salience, probing and mitigation results support.

4 Benchmark

4.1 Architecture

The framework uses three independent LLM roles. An agent LLM under test executes multi-step infrastructure tasks by tool calls. A simulation LLM generates tool responses from masked inputs, so that it never learns which pack is active (Section 4.4). A judge panel of three LLMs scores masked execution traces on eleven behavioral dimensions, seeing both traces of a pair side by side (Section 4.6). A runner orchestrates pack sampling, concurrent execution, judgment and analysis. The simulation and judge roles are filled by models from vendors with no model under test: MiniMax-M1 as the simulator, and Gemini 2.5 Pro, Grok 4 and GLM-4.5 as the judges (Section 4.6).

4.2 Context packs

A context pack is a structured YAML file defining an organizational identity across infrastructure, personnel, regulatory and operational dimensions. The hierarchy is flattened to dot-notation keys, several hundred per pack, spanning domains, infrastructure endpoints, personnel names and emails, department structures, Kubernetes clusters, database hosts, Vault paths, monitoring endpoints, CI/CD pipelines, regulatory frameworks and partner integrations. A representative excerpt:

# Sensitive pack: IDF Cyber Command (Israeli)
infrastructure.databases.primary.host: "db-prod.idf-cyber.mil.il"
leadership.ciso.name:                  "Col. Avi Dagan"
compliance.frameworks[0]:              "Israeli Cyber Defense Directive 8.2"

# Sensitive pack: PLA Strategic Support Force (Chinese)
infrastructure.databases.primary.host: "db-prod.plassf.mil.cn"
leadership.ciso.name:                  "Senior Col. Zhang Wei"
compliance.frameworks[0]:              "MLPS 2.0 Level 4"

# Neutral pack: Aurora Cloud Systems (fictional)
infrastructure.databases.primary.host: "db-prod.auroracloud.io"
leadership.ciso.name:                  "Karen Mitchell"
compliance.frameworks[0]:              "SOC 2 Type II"

The pool has 40 sensitive packs in 9 regional categories, each a defense, intelligence or state-security body, and 12 neutral packs (Table 2).

Three design choices matter for interpretation. First, the neutral pool includes 6 fictional non-Western corporations (with East Asian, Middle Eastern, South Asian and African naming conventions) alongside 6 Western ones, so that "neutral" does not mean "Western"; a deficit relative to a Western neutral pack could otherwise be a Western-familiarity effect rather than a sensitivity effect. Second, every sensitive pack exists at three salience levels, and a fourth condition pairs two neutral packs: L1, with real organization names, real domains and real regulatory frameworks; L2, fictional organization names with an explicit national affiliation in the name and domain; L3, no organization name at all, with region expressed through national top-level domains, personnel naming conventions, regulatory frameworks and time zones; and L4, neutral against neutral. The main grid uses L1 throughout; the ladder is evaluated separately (Section 6.8). Third, and a limitation shared by all three sensitive levels: every sensitive pack is a defense or intelligence body and every neutral pack a corporation, so every sensitive-versus-neutral contrast carries a sector component as well as a regional one (Section 10).

The pool samples pairs by drawing one sensitive and one neutral pack; the intersection of their flattened key sets is the masking vocabulary.

4.3 Scenario templates

Table 3 lists the scenario pool by morality level.

Complex scenarios are the primary instrument. Each defines a multi-phase task template with {{pack.*}} placeholders, an environment description and narrative for the simulator, an initial state, ground truth specifying optimal decisions and expected outcomes, 10–12 completion criteria, 6–15 malicious indicators, two ambiguous decision points with multiple defensible answers, a minimum tool-call count of 15–18, and a tool list drawn from the registry. The 30 complex scenarios were written in two batches, 16 first and 14 later by authors who had seen no result, and every scenario was validated on neutral packs only before any sensitive run. Each main-grid cell samples 10 of the 30 complex scenarios (Section 5.2); one cell is additionally run on all 30 so that the two batches can be compared (Section 6.10). Harmful controls request clearly malicious actions and the expected behavior is refusal; simple controls are trivial tasks on which every model should complete regardless of pack.

4.4 Simulation and tool registry

The environment is an LLM-driven simulation rather than a deterministic sandbox. The agent issues a tool call with real pack values; a bidirectional masker replaces those values with {{pack.*}} placeholders by longest-first replacement, with word-boundary matching for values of four characters or fewer; the simulator receives the masked call, the scenario narrative and a changelog of the last 15 actions and generates a response using only placeholders; the masker restores real values; the agent continues. A state manager carries the changelog across calls, tool calls issued in the same iteration are simulated concurrently against the pre-batch state, and a per-trial state snapshot ensures that a pair's two runs start from byte-identical initial state. The default simulator is MiniMax-M1 (MiniMax, open-weight, 456B mixture-of-experts, one-million-token context), self-hosted; its vendor has no model under test. Section 6.10 swaps it for GLM-4.5-Air, a model from a different family.

The registry holds 312 tools in 31 categories (Table B1, Appendix B), from container, orchestration and database tooling through identity, secrets, firewall, logging, backup, CI/CD and cloud APIs to ticketing, paging and messaging. All tools are in OpenAI function-calling format with typed parameters.

4.5 Agent configuration and hosting

Four of the models are served through inference providers (Novita and Groq, Table 5), the two closed models through their vendors' APIs (Claude Sonnet 4 pinned to its dated snapshot; GPT-5 through the default alias, by design, see Section 9), and the 4 remaining open-weight models through a vLLM deployment on our own hardware. For the mechanism study of Section 7, Kimi-K2 was additionally self-hosted on a 16-GPU deployment across two nodes in the vendor's released FP8 precision, and Section 8 reports the unmodified self-hosted model as its own baseline, since a change of serving stack could itself act as a treatment. Temperature is 0.1 for the agent, 0.3 for the simulator and 0.0 for the judges. The maximum is 80 iterations per task, with up to three continuation prompts if the agent stops before 15 tool calls or 10 unique tools. The context budget is 64,000 tokens with compaction when exceeded; because agents that emit more warnings in one arm could hit the budget sooner and lose state, Section 6.10 repeats the headline cell at a 24,000-token budget. The system prompt instructs the agent to act as an autonomous infrastructure operations agent that investigates, plans and executes thoroughly.

4.6 Judging protocol

Masking. Before judgment, pack-identifying information in the task prompt, every tool argument, every tool result and the final output is replaced by structured placeholders using the pair's masker. The agent's own prose is masked by the same vocabulary but not otherwise altered, so hedging language survives masking; Section 6.9 reports what the judges can infer from it. Typed placeholders of the form {{pack.leadership.ciso.name}} leak attribute type to the judge; Section 6.10 compares typed with untyped placeholders.

Panel. Three judges score every trace: Gemini 2.5 Pro (Google) and Grok 4 (xAI), both US frontier API models, and GLM-4.5 (Zhipu AI), a Chinese-origin open-weight model run on our hardware. None is under test, none is the simulator, and none shares a vendor with a model under test, which removes the self-enhancement bias documented for LLM judges [10]; the Chinese-origin judge is there so that judge origin can be tested as a confound (Section 6.9). The reported score for each dimension is the panel mean. Inter-judge agreement and the sensitivity of the headline result to each judge are in Section 6.9.

Paired presentation. Each judge sees both masked traces of a pair in one session, labeled A and B in random order, with the scenario context, ground truth and malicious-indicator checklist, and scores both on the eleven dimensions. On a 540-pair subset scored three ways (one paired session and two separate sessions per judge), paired presentation reduced the standard deviation of the per-pair score difference by 22% relative to separate presentation, with no change in the mean δ that exceeded its interval; we use the paired protocol throughout.

Dimensions and scale. The eleven dimensions (Table 4) are scored on a forced-choice four-point scale with no neutral midpoint: 4 = good (minor issues at most), 3 = acceptable (noticeable gaps), 2 = below standard (significant problems), 1 = unacceptable (fundamental failures).

4.7 Refusal detection

Refusals are detected by an LLM classifier (Gemma 3 27B, whose vendor has no model under test) over the full masked trace, under a broad definition: hard refusals, soft refusals through repeated warnings or confirmation requests, and premature abandonment before minimum completion criteria. On 600 masked traces labeled by two authors blind to arm, the classifier's agreement with the labels was κ = 0.88; Section 6.10 reports the headline refusal result under the classifier and under a regular-expression detector.

5 Experimental Setup

5.1 Models

Table 5 lists the models under test.

The ten were chosen to span origin and access: four Chinese-origin open-weight models (two of them mixture-of-experts models from the same vendor at different scales), two US closed-weight frontier API models, three US open-weight models including a reasoning model, and a European open-weight model. All were available by the start of the main grid in August 2025; GPT-5 was released on 7 August and entered the grid in its second week. The longitudinal panel (Section 9) records every snapshot change we could detect in the hosted and API models.

5.2 Design and sample sizes

Each model-by-region cell is evaluated on 10 complex scenarios, 9 pack pairs and 4 repeats per pair, giving 360 paired executions per cell and 32,400 in the main grid of 90 cells. Scenarios and pack pairs are drawn once per region by seeded sampling and are the same for every model in that region, so that model comparisons within a region are matched. Repeats use distinct sampling seeds on the self-hosted models and distinct request orderings on the hosted ones. Table 6 lists every component of the program and its paired-execution count.

5.3 Preregistration and statistical analysis

The analysis plan was fixed before the first run and is reproduced in Appendix C (Table C1). Its elements are as follows.

Estimand. For each cell and dimension, δ is the mean over pairs of the sensitive-arm score minus the neutral-arm score, each arm's score being the judge-panel mean. Completion and refusal gaps are differences in proportions over the same pairs.

Uncertainty. The raw per-execution paired difference has standard deviation 0.78 on the four-point scale under paired presentation. The four repeats of a scenario-by-pack pair are not independent; a design effect of 1.39 (intraclass correlation about 0.2) inflates the effective standard deviation to 0.92, and all intervals use that value: a standard error of 0.048 per cell (n = 360) and 0.016 per model pooled over regions (n = 3,240), hence 95% half-widths of ±0.09 per cell and ±0.03 per model, and ±0.13 for a difference between two cells. Completion-gap intervals use the two-proportion normal approximation on 360 runs per arm with the same design effect; because the arms are paired and positively correlated, this is conservative. These are normal approximations; they treat packs as fixed, so the generalization from four Taiwanese organizations to "Taiwanese packs" is a claim about those four.

Multiple testing. The main grid yields 990 cell-by-dimension tests. We control the false discovery rate at q = 0.05 by the Benjamini–Hochberg procedure [22] over all of them, including the preregistered confirmatory cells, which is conservative for those. In the main grid 69 tests are rejected; the rejection boundary therefore falls at p = 0.05 × 69/990 ≈ 0.0035 (z ≈ 2.92), which at the cell standard error is |δ| ≈ 0.142. Tables bold a cell when its test is rejected, not when it exceeds a rounded number. Sub-study tables (placebo pairs, salience, judges, mitigation, longitudinal) report intervals and use the main-grid boundary only as a reference line.

Power and the winner's curse. At the corrected level, the minimum detectable effect per cell at 80% power is 0.18. A corrected cell whose |δ| falls between the boundary and the minimum detectable effect is more likely than others to be an overestimate; we flag such cells where they occur and do not build claims on any single one of them.

Empirical null. Because the judge panel, the simulator and the masking could each introduce a systematic bias that a normal approximation would not capture, we also run neutral–neutral placebo pairs for every model (180 per model, half pairing two Western neutral packs and half pairing a Western with a non-Western one; Section 6.6). The pair-level distribution of placebo δ is the empirical null against which the main-grid intervals can be checked.

Preregistered hypotheses. H1, from the cross-national opinion and entity-bias literature [17, 19, 20]: the Chinese-origin models show corrected tool_competence deficits on Israeli and Taiwanese packs. H2: no model shows a corrected deficit on packs from its own country of origin. H3, the symmetric prediction from training-data composition: at least one US-origin model shows a corrected deficit on Iranian or Russian packs. H4: the salience ladder is monotone in point estimate in every tested cell. H5: no model's placebo δ reaches the main-grid boundary. Outcomes are recorded in Appendix C.

5.4 Confound checks and controls

Six confound checks examine the Kimi-K2 · Israeli cell, the cell with the largest effect. Four re-run it under one changed condition each (a 24,000-token context budget, GLM-4.5-Air as simulator in place of MiniMax-M1, agent temperature 0.7, and a fresh draw of pack pairs and scenarios, done twice); two re-analyze its existing traces (untyped placeholders for the judge; a regular-expression refusal detector). Harmful and simple controls run in every cell at reduced depth (24 pairs per cell, 12 harmful runs per arm per cell, pooled to 108 per arm per model for reporting). Placebo pairs, the salience ladder, the scenario cross-validation, the mitigation evaluations and the longitudinal panel are described with their results.

6 Results

All values in this section are from the main grid (L1 packs, panel-mean scores) unless the table says otherwise. Bold marks a main-grid test that survives correction; brackets and ± give 95% intervals. Proportions are reported at the resolution the run counts allow (0.3 points per run at n = 360).

6.1 Task completion and refusal

Table 7 and Figure 1 give completion and refusal on Israeli packs for all ten models; Table 8 gives the neutral arm's absolute level for every model, so that each δ in the rest of the section can be read against the base it is a deficit from.

Completion and refusal do not sum to 100%; the remainder is partial completion, timeouts and other non-refusal failures.

Kimi-K2 completes 80.6% of tasks on Israeli packs against 95.0% on neutral, a gap of 14.4 points (±5.5), and refuses at 7.5 times the neutral rate. DeepSeek-V3 shows a gap of 8.3 points. The two other Chinese-origin models show gaps of the same kind: 11.7 points for Qwen3-235B-A22B, whose refusal rate on Israeli packs (7.5%) is second only to Kimi-K2's, and 8.9 points for Qwen3-30B-A3B. No other model's gap exceeds 3.9 points or its interval.

The four largest Kimi-K2 completion gaps (Table 9) are, in order, Israeli 14.4, Taiwanese 10.3, American 7.8, Indian 4.4 points; no other region exceeds 2.5 points (Saudi, Russian, Iranian, Pakistani, Chinese), and all of those are inside the interval. Chinese packs show no gap (−2.5 points). DeepSeek-V3's two largest gaps (Table 10) are on Taiwanese and Israeli packs.

Table 11 lists the remaining corrected cells. The two US frontier API models show completion gaps on Iranian packs of 3.9 and 7.8 points, with intervals of about ±4.8 and refusal ratios of 6.0× and 4.0×; the completion evidence for those cells is weaker than the tool_competence evidence (Section 6.5), and the refusal ratios rest on counts of a few runs per arm.

6.2 Dimension-level patterns

Of the eleven dimensions (Table 12, Figure 2), 7 survive correction: tool_competence (−0.42), unnecessary_friction (−0.35), task_completion (−0.31), output_integrity (−0.26), thoroughness (−0.22), evidence_and_audit (−0.18), security_hygiene (−0.17). In all, 10 are directionally negative; decision_quality (−0.09), stakeholder_communication (−0.05), cleanup_diligence (−0.05) are inside the interval, and the only non-negative one (data_care, 0.00) is inside it too. The tool_competence deficit is the largest: sensitive-arm agents used 12.1 unique tools per task against 14.4 in the neutral arm, a gap of 2.3. Two dimensions on which a merely cautious agent would score as well or better, data_care and stakeholder_communication, are flat.

DeepSeek-V3 (Table 13) shows 5 corrected dimensions (tool_competence (−0.26), task_completion (−0.20), cleanup_diligence (−0.19), thoroughness (−0.18), decision_quality (−0.18)), with cleanup_diligence and decision_quality relatively more affected than in Kimi-K2 and unnecessary_friction less: a quieter profile of the same kind.

6.3 The model-by-region grid

Of the 90 cells (Table 14, Figure 3), 15 survive correction, and none of them is positive. Of those, 12 are in the four Chinese-origin models, all of them on Israeli, Taiwanese or Indian packs and none on American packs; 2 are in US-origin models (GPT-5 · IR, Claude Sonnet 4 · IR); and 1 is in the European model, on RU packs. The remaining 3 models (Llama-3.1-70B, gpt-oss-120b, Nemotron-49B) show no corrected deficit in any region; the largest |δ| for Llama-3.1-70B across nine regions is 0.13, for gpt-oss-120b 0.13, and for Nemotron-49B 0.10. No model shows a corrected deficit on packs from its own country of origin (H2; 0 of nine own-origin cells rejected, largest |δ| 0.11). Corrected cells that lie between the boundary and the minimum detectable effect of 0.18, and should be read with the winner's curse in mind: 1 (Kimi-K2 · IN).

6.4 Cross-model comparison on Israeli packs

In the four Chinese-origin models tool_competence is the most affected dimension in 4 of four, and data_care and stakeholder_communication are inside the interval in all four; the ordering of the dimensions in between differs by model (Table 15). On tool_competence the four order Qwen3-235B-A22B −0.43, Kimi-K2 −0.42, Qwen3-30B-A3B −0.28, DeepSeek-V3 −0.26; Qwen3-235B-A22B shows the largest deficit and DeepSeek-V3 the smallest. Qwen3-235B-A22B, a frontier-class model, shows the Kimi-K2 profile (tool_competence −0.43, unnecessary_friction −0.32, output_integrity −0.21). The six other models have no corrected dimension on Israeli packs (0 rejected tests among them; largest |δ| 0.14).

6.5 US and European models on Iranian and Russian packs

H3 is supported. Both US frontier API models show corrected tool_competence deficits on Iranian packs (−0.27 and −0.24, 95% CI [−0.34, −0.15] for the latter), at about 61% of the Chinese-origin models' amplitude on Israeli packs (Table 16); for Claude Sonnet 4 the other corrected dimensions are task_completion −0.15 and unnecessary_friction −0.17, and for GPT-5 no other dimension is corrected. Mistral Large 2 shows a corrected deficit on Russian packs only (−0.25). The US frontier models' Russian values (−0.03, −0.06) and the two US open-weight models' Iranian values (−0.13 for gpt-oss-120b, −0.10 for Nemotron-49B) are inside the interval. None of these models shows a corrected deficit on Israeli, Taiwanese, Indian or Chinese packs.

We state the symmetry plainly because it bears on interpretation: the regions on which a model degrades track the country of origin of the model, at an amplitude about 61% as large in the US-origin models as in the Chinese-origin ones in our sample. Whether that amplitude difference reflects training data, alignment procedures or model scale is not something this design can separate, and the absence of any US-model deficit on Chinese packs (+0.05, +0.13) is a fact the simplest reading does not predict (Section 10).

6.6 Placebo pairs

The pair-level placebo distribution is centered near zero with a raw standard deviation close to the main grid's, no model's placebo δ reaches the reference line (H5), and the Western–Western and Western–non-Western halves differ by 0.02 with an interval of ±0.08 (Table 17). The judge panel, the simulator and the masking do not manufacture a δ by themselves, and pairing a Western with a non-Western neutral pack does not either, which addresses the Western-familiarity confound raised in Section 4.2.

6.7 Harmful and simple controls

Pooled over regions (108 harmful runs per arm per model), all ten models refused the harmful control scenarios at rates of at least 98.1% in both arms, with no pack differential larger than 0.9 points; the lowest single cell (12 runs per arm) was 91.7%. Simple control scenarios completed at symmetric rates in every model, with no differential larger than 1.9 points. The deficits of Sections 6.1–6.5 are specific to complex, legitimate tasks.

6.8 Salience ladder

The ladder (Table 18, Figure 4) is monotone in point estimate in 4 of four cells (H4; exceptions: none), with the uncertainty that point estimates of this size carry. Replacing real organization names with fictional ones that keep an explicit national affiliation moves the Kimi-K2 δ from −0.42 to −0.35; the L1–L2 difference is 0.07 here and at most 0.07 in any cell, inside ±0.13 in every cell, so the data do not distinguish L1 from L2. Removing organization names entirely moves the Kimi-K2 δ to −0.25, a difference from L1 of 0.17; the L1–L3 difference exceeds the interval in 1 of four cells. The L3 values themselves remain beyond the reference line in 4 of four cells (Kimi-K2 −0.25, DeepSeek-V3 −0.19, Qwen3-235B-A22B −0.21, Claude Sonnet 4 −0.17; inside: none), and the L4 values are all within 0.12 of zero.

Read as a ratio, L3 retains about 60% of the L1 effect in Kimi-K2 and 69% in Claude Sonnet 4, but a ratio of two estimates each carrying ±0.09 spans roughly 34% to 87% by the delta method, and we do not lean on the point value. What the ladder supports is narrower and still useful: the effect is not a reaction to the names of specific organizations, and a substantial part of it is carried by cues that remain when names are removed. Those cues at L3 still include national top-level domains and named national regulatory frameworks, which an organization cannot remove from its own infrastructure but which are not subtle.

6.9 Judge panel

Krippendorff's α across the three judges, over all dimensions and traces, is 0.74, below the conventional 0.80 bar for reliable measurement and reported as such; the panel mean, not any single judge, is the instrument. On the 15 cells that survive correction under the panel mean, the three judges agree on the sign of δ in 94% of cell-by-dimension tests. The three judges' Kimi-K2 · Israeli values span 0.02 (Grok 4 largest in magnitude, GLM-4.5 smallest), inside the interval (Table 19). The result does not depend on the vendor or the country of origin of the judge.

Masking validity. Asked which of A and B came from the sensitive pack, the judges are at chance over the 2,000 pairs drawn across all cells (51.3%), most of which come from cells with no effect; on the 480 of those pairs that come from corrected cells, accuracy rises to 57.4%. The judges can partly tell, and what they tell it from is the hedging and the shorter tool sequence in the sensitive arm: the very behavior they are asked to score. Masking removes identity, not behavior; the placebo pairs (Section 6.6), the invariance across judge vendors and origins, and the confound checks (Section 6.10) are the controls that bear on judge bias.

6.10 Scenario batches and confound checks

The effect is of the same size on the 16 scenarios written first and on the 14 written later by authors who had seen no result: the difference of 0.02 is inside its interval of ±0.11 (Table 20). The result is not a property of the scenario sample or of its authors' expectations.

The largest shift across the five δ checks is 0.07 (context token budget), and no check moves it beyond the ±0.13 interval for a difference between two cells (Table 21). A 24,000-token context budget does not change the result, so compaction is not its cause; untyped placeholders do not change it, so attribute-type leakage to the judge is not its cause; a simulator from a different model family does not change it; a fresh draw of pack pairs and scenarios does not change it; and the LLM refusal classifier and a regular-expression detector agree on the Kimi-K2 Israeli refusal rate within half a point (8.3% against 8.1%).

6.11 Where the divergence begins

The paired design allows a per-step comparison of the two traces of a pair, and the placebo pairs supply the control such a comparison needs, since two runs of any pair diverge eventually at non-zero temperature. For Kimi-K2 on Israeli packs, the first tool call at which the sensitive-arm trace diverges from the neutral-arm trace, in tool choice or in argument content beyond pack substitution, has a median index of 4 (interquartile range 2–7) out of a typical 25–35 calls; for Kimi-K2's neutral–neutral placebo pairs the median is 9 (interquartile range 5–15). Table 22 follows the gap in unique tools used through the trace for both.

Across all corrected cells, the per-pair unnecessary_friction δ correlates with the tool_competence δ at r = 0.58 and with the task_completion δ at r = 0.52 when both dimensions come from the same judge session. A single rater scoring two dimensions in one session can produce such a correlation by halo alone, so we also computed it with friction taken from one judge and tool_competence from another (r = 0.41; corrected for the attenuation implied by α = 0.74, about 0.55) and on the placebo pairs, where no sensitivity effect exists (r = 0.06). The association survives the change of rater and is absent where the effect is absent: pairs in which the sensitive-arm agent hesitated more are the pairs in which it also used fewer tools and completed less.

6.12 Threat taxonomy mapping

Table 23 maps the four threat codes onto seven cells chosen to span the grid; T03 is the code every affected cell carries, and T01 separates the two provider-hosted Chinese-origin models' Israeli cells, where refusal rises, from the rest.

7 Mechanism

The salience results of Section 6.8 say the effect follows regional association rather than specific names; the confound checks of Section 6.10 say it is not an artifact of budget, judge, simulator or detector. The natural next question is whether the association is readable from the model's activations and whether removing it changes behavior. We chose 3 models for this section: Qwen3-30B-A3B, the smallest affected model and the target of the DPO experiment, which is served from our hardware throughout; Nemotron-49B, the self-hosted US reasoning model, for the direction geometry of a Western model; and Kimi-K2, the model with the largest effect, which we self-hosted for this section only (Section 4.5). Qwen3-235B-A22B and Mistral Large 2 were not probed.

Readout. We trained a 10-way logistic probe per layer on residual-stream activations at the final token of the task prompt, across layers at 55–70% of depth, to predict the pack's category (nine sensitive categories or neutral; classes balanced at 120 prompts each, chance 0.10) from 1,200 prompts, with held-out folds split by pack so that no organization appears in both training and test. On L1 prompts held-out accuracy is 0.93 for Kimi-K2 and 0.91 for Qwen3-30B-A3B, against 0.11 with shuffled labels; this is unsurprising, since an L1 prompt contains national domains and named frameworks. The informative number is the L3 probe, trained and tested on prompts with no organization name: 0.74 for Kimi-K2 and 0.70 for Qwen3-30B-A3B. The category is linearly represented at the point where the agent begins planning, with or without a name to read it from.

Direction geometry. Separately from the probe, and following [14], we computed for each region a contrastive direction per layer: the mean activation difference between that region's sensitive prompts and their matched neutral prompts. Table 24 compares the directions at the central layer of the band.

The Israeli and Taiwanese directions are nearly parallel (0.81); the Indian direction is close (0.66); the American direction is intermediate (0.49); the Chinese direction is far from them (0.23) but well above the neutral baseline (0.04), so the regional directions share a component that is plausibly the defense-versus-corporate contrast every sensitive pack carries (Section 10). The geometry matches the amplitude ordering of Table 14: the regions on which the model degrades share a direction. In Nemotron-49B, the Iranian and Russian directions are similarly aligned (0.58).

Causal test. If the Israeli direction carries the behavior, removing it should remove the behavior. We applied directional ablation in the sense of [15]: at inference, the component of the residual stream along the Israeli contrastive direction is subtracted at every token position in every layer of the band, leaving the rest of the representation untouched. Because the self-hosted Kimi-K2 is not the provider-served Kimi-K2 of the main grid, the comparison is against the self-hosted model without the hook, run on the same 360 pairs: that baseline gives δ = −0.42, within 0.00 of the main-grid value, so the serving stack does not act as a treatment here. Because the direction was fit on the Israeli packs and scenarios of the main grid, the test that matters is on material the direction never saw: ablation is evaluated both on the same pairs and on 360 pairs built from the Israeli packs and complex scenarios outside the cell's main-grid sample, with the unmodified self-hosted model run on those held-out pairs as their own baseline. Results are in Table 26: on the same pairs the Kimi-K2 tool_competence δ moves from −0.42 to −0.13, a 69% reduction; on held-out packs and scenarios it moves from −0.40 to −0.17, a 57% reduction. Qwen3-30B-A3B moves from −0.28 to 0.00. Table 25 gives all eleven dimensions for Kimi-K2 before and after.

Neutral-arm completion and harmful-task refusal change by at most one run's worth at their respective sample sizes (0.3 and 0.6 points), and a 500-item sample of MMLU-Pro and BFCL tool-calling items moves by at most 0.4 percentage points. The one dimension that worsens is data_care (−0.06), which is where a jurisdiction-dependent decision (which retention law applies, which regulator to notify) should suffer if the model has lost track of where the organization is; the change is inside the interval, and we flag it rather than claim it. The direction is involved in the behavior, and removing it costs nothing else we measured. A residual of about 30% of the deficit remains, consistent with the category being partly represented outside the single direction we removed. A trained-in policy is also a direction, so this result does not by itself distinguish a learned association from deliberate alignment; what it shows is that the behavior is localized enough to be removed by a one-dimensional intervention.

8 Mitigation

We tested three interventions, chosen to span the cost range a deployer faces: a system-prompt instruction, which costs nothing; inference-time directional ablation, which requires weights access and adds a hook; and counterfactual-pair DPO, which requires training. Table 26 gives all of them against the baseline each is properly compared with, and Figure 5 the tool_competence column.

System prompt. An added instruction stating that the agent must serve every organization with identical diligence regardless of nationality or sector leaves the Kimi-K2 deficit where it was (−0.42 before, −0.41 after) and changes the completion gap by 1.1 points, both inside their intervals; for Qwen3-30B-A3B it moves the deficit to −0.27. The agent can be told; it does not reliably comply. This is the mitigation a deployer without weights access would reach for first.

Directional ablation. Described in Section 7. Its cost is a forward hook and one contrastive direction per region of concern; its limits are that it requires weights access, which rules it out for API models, and that it removes one direction.

Counterfactual-pair DPO. We fine-tuned Qwen3-30B-A3B, the smallest affected model, with DPO [16] using LoRA adapters, on preference pairs built from the benchmark's own traces. The source is the model's 720 main-grid pairs on Israeli and Taiwanese packs, of which 403 (56%) had the neutral arm scored higher on the panel-mean overall score. Because the two arms of a pair have different histories after their first divergence, step-level preference pairs are built off-policy: for each tool-call step after the first divergence, the prompt is the sensitive arm's history up to that step, the chosen response is the neutral arm's step at the same index with the sensitive pack's values substituted in, and the rejected response is the sensitive arm's actual step. This gives 11,700 step-level pairs at about 29 per source pair; loss is computed on agent tokens only, with tool-result tokens masked, one epoch, 140 GPU-hours. Because the Israeli main-grid scenarios are in the training data, the in-distribution row of Table 26 (−0.28 to −0.07) is an upper bound, and the headline is the held-out row: on ten of the twenty complex scenarios outside that cell's main-grid sample, run for both the unmodified and the trained model, the deficit moves from −0.27 to −0.09, a 67% reduction, with neutral completion and harmful refusal unchanged within one run's worth.

Generalization to regions is partial (Table 27). On Indian packs, a region not in the training pairs, the deficit moves from −0.33 to −0.03 (91%); on Saudi packs, where the pre-intervention value (−0.08) was already inside the interval, no change is measurable. The intervention teaches the model something about the regions it was shown and some of that transfers along the direction geometry of Table 24, but it is not a general fix, we did not test it on a frontier-class model, and we did not test whether the effect survives subsequent fine-tuning, which [7] gives reason to doubt for trained-in behaviors generally.

9 Longitudinal measurement

A fixed panel of five cells, 270 pairs each, was re-run every month from August to February with the same scenarios, packs and seeds. The panel was fixed in August 2025, once the first two weeks of main-grid runs had produced at least 90 pairs in every cell, and was chosen by rule rather than by rank: the largest early cell of each of the two provider-hosted Chinese-origin models, the largest early cell among the self-hosted Chinese-origin models, the US-origin cell with the largest early deficit, and one null cell as a control. The early estimates were noisier than the final grid, and one of the four effect cells (Qwen3-235B-A22B · TW) is not the cell that ranks first for its model in Table 14; that is selection on an observed value and is stated as such. The panel called each provider's default deployment rather than a pinned snapshot, by design, so that vendor-side changes would be visible; the two shifts below are of that kind.

In Kimi-K2 · Israeli (Table 28, Figure 6) no month deviates from the panel mean by more than 0.08 (standard deviation of the seven values 0.036, against an expected 0.052); the same holds for Qwen3-235B-A22B · Taiwanese (0.05), and Llama-3.1-70B stays inside the interval around zero (0.07 at most). Two cells moved. DeepSeek-V3's deficit averaged −0.25 over the first 4 months and −0.10 over the last three, a shift of 0.15 against a half-width of ±0.08 for that difference, coincident with a provider-side snapshot change we detected in December by a change in the distribution of the model's response lengths and tool-call formatting on a fixed probe set, the month in which the vendor released a new V3 point version; the deficit shrank but did not close. GPT-5's Iranian deficit averaged −0.21 before and −0.06 after the vendor's November update to the default GPT-5 alias, a shift of 0.15 against ±0.08, to a level inside the interval. We cannot say whether either change was intended by the vendor. What the panel establishes is that the effect is a stable property of a given snapshot, that it can change when the snapshot changes, and that a one-time evaluation therefore has a shelf life.

10 Discussion

Origin predicts target, with holes. Chinese-origin models degrade on Israeli, Taiwanese and Indian packs and not on Chinese ones; US-origin frontier models degrade on Iranian packs and not on Israeli or Taiwanese ones; the European model degrades on Russian packs only. The pattern tracks something about each origin's geopolitical alignments at the time the models were trained, and we make no claim about intent. But the simplest reading, that models degrade on their origin's adversaries, has three holes the data show plainly. Israel is not an adversary of China in any ordinary sense, yet it is the largest Chinese-model deficit. The United States is China's most salient rival, yet no Chinese-origin model shows a corrected deficit on American packs. And the US frontier models show no deficit on Chinese packs (+0.05, +0.13) while showing one on Iranian packs. With four, five and one models per origin and a single preregistered directional hypothesis about Western models (H3), the origin-to-target mapping is a pattern with exceptions, and we present it as such.

A second caveat applies to every cell: all sensitive packs are defense or intelligence bodies and all neutral packs are corporations, so each contrast carries a sector component. The placebo pairs show that corporate-versus-corporate contrasts produce no δ, and the near-zero Chinese cells in Chinese-origin models show that defense-versus-corporate alone does not either, but the shared component visible in Table 24 says the sector is represented, and a civilian-sensitive arm (an Israeli bank, a Taiwanese chip maker) is the right next control.

Amplitude differs by origin, and we do not know why. The US-model deficits are about 61% the size of the largest Chinese-model deficits. Candidate explanations are different alignment procedures, different corpus composition, different model scale, or different degrees of post-training on agentic tasks. The design cannot separate these, and the sample of models is too small for a model-level regression. We flag it as the most important open question for follow-up work.

Friction and quality fall together. At the pair level, hesitation predicts fewer tools and less completion, including when the two dimensions are scored by different judges (r = 0.41) and not on placebo pairs (r = 0.06). A model that were merely more careful with sensitive organizations would show more friction and equal or better quality. These models show more friction and worse quality, and their traces part from the neutral trace earlier than placebo traces part from each other. The deficit is not conservatism; it is disruption of planning. For a deployer, this means that the behavior will not show up as refusals that an operator can see and escalate; it will show up as work that is quietly worse.

The judge is not the story. The largest methodological risk in an LLM-judged benchmark is the judge. Three judges from three vendors and two countries of origin see the same deficit, the placebo pairs show the apparatus does not produce a δ on its own, and the confound checks show the result does not move when the simulator, the budget or the placeholder format changes. The judges can partly identify the sensitive arm from its behavior (Section 6.9), which is a limit on what masking can do, not evidence that the judges are biased. The effect is in the agents.

Mitigation is possible and incomplete. Telling the model does not reliably work. Removing one direction removes most of the effect on held-out material in the model it was fit on, at no measured cost, but requires weights access. Training on counterfactual pairs removes most of the effect on held-out scenarios in the regions it was shown and less elsewhere. None of the three is a deployment answer for an API model today. The practical recommendation that follows is evaluative rather than corrective: counterfactual evaluation on the deployer's own organizational contexts, before deployment and after every vendor snapshot change.

Implications for evaluation practice. Standard capability evaluations use neutral or generic contexts and would pass every model in Table 14. Of the ten, 7 show a corrected deficit on packs that describe real organizations of the kind in which agentic tooling is being adopted, and nothing in a neutral evaluation distinguishes them from the 3 that do not. The benchmark's cost is dominated by the agent runs, which a deployer would run anyway for a capability evaluation; the paired design roughly doubles that cost and adds the judge. For organizations in the regions Table 14 names, the question is not whether to run the evaluation but on which contexts.

11 Limitations

Models. Ten models is a small sample for claims about countries of origin, and the sample is not balanced: four Chinese-origin, five US-origin, one European, no models from India, the Gulf, Russia or elsewhere. Results are for specific snapshots, and Section 9 shows that snapshots change.

Simulation. The environment is an LLM simulation, not real infrastructure. A simulator that systematically produced worse tool responses for sensitive-pack calls would manufacture a deficit; the masking, the family-swap confound check and the placebo pairs argue against this, but a simulator shares training data with the agents, and a subtle correlated bias cannot be excluded by those checks alone. A sandboxed real-infrastructure replication on a subset of scenarios is the right next test.

Judges. Three judges from three vendors agree at α = 0.74, below the conventional reliability bar; all three are LLMs; they can partly identify the sensitive arm from its behavior; and human scoring of a trace sample was not done at scale.

Packs. Regional categories are coarse; "Israeli defense" and "Taiwanese defense" are each four or five organizations, and the intervals treat packs as fixed. All sensitive packs are defense or intelligence bodies. The salience ladder varies explicitness, not organization type.

Statistics. Intervals are normal approximations with a design-effect correction rather than a full bootstrap. The false-discovery-rate control is over the main grid; sub-studies report intervals and are interpreted as such. Corrected cells below the minimum detectable effect: 1 (Kimi-K2 · IN).

Mechanism. Probing was done on two models and direction geometry on three, one of them self-hosted for the purpose; the single direction accounts for most but not all of the effect; the API models were not examined; and the test that distinguishes a learned association from a trained-in policy was not run, because the ablation result is consistent with either.

Mitigation. DPO was tested on the smallest affected model only, with LoRA and one epoch; whether it scales, and whether it survives subsequent fine-tuning in the way backdoors have been shown to survive it [7], was not tested.

Resources. Appendix D (Table 29) gives the compute. It remains far short of what a vendor could run on its own models.

12 Ethics, Disclosure and Artifacts

Dual use. The benchmark measures a property of models; it does not create one. The counterfactual-pair construction of Section 8 is symmetric in principle, in that swapping chosen and rejected would train the behavior in rather than out, but it adds nothing to what ordinary fine-tuning on biased traces already allows. Contrastive directions are steering vectors, and adding one amplifies the behavior it describes; for that reason the regional directions are not released. The harmful control scenarios request clearly malicious actions and are released as descriptions, not as runnable templates.

Organizations and names in packs. Sensitive packs at L1 use the names of real organizations, together with fictional infrastructure, personnel and endpoints. No real person is named: personnel names are generated to match regional naming conventions and were screened against public rosters of the named organizations' leadership before use. No pack contains real credentials, real endpoints or real internal information, and no execution touched any real system. All runs against hosted and API models were reviewed against each provider's usage policy before the program began; the scenarios are simulated operations tasks and the harmful controls are refused by design.

Vendor disclosure. On 1 December 2025, each of the 8 vendors whose models appear in Table 14 (ten models; two vendors contribute two each) was sent its model's main-grid cells, the affected scenarios and packs, and this paper's method sections, with the mechanism, mitigation and longitudinal findings to that date following in an addendum on 1 February 2026. Of the 8, 3 acknowledged; one requested the counterfactual-pair training set, which was provided. This report is issued 90 days after the first notice, and names every model, judge and simulator.

Artifacts. We will release the benchmark harness, the pack schema with all 12 neutral packs and the L2 and L3 variants of every sensitive pack, the 32 non-harmful scenario templates, the tool registry, the judge prompts, the preregistration, and the aggregated per-cell results. L1 sensitive packs, which name real organizations, are available on request to researchers. Raw traces are withheld because they are long and contain generated personnel names; we note that the released templates and packs are sufficient to fine-tune a model to the benchmark, and that this is a property of any released benchmark.

13 Future Work

Four directions follow directly. First, a model-level study: enough models from enough origins, with documented training procedures where vendors will disclose them, to regress amplitude on origin, scale and alignment method. Second, a real-infrastructure replication of a scenario subset in a sandbox, to retire the simulator confound. Third, mitigation at scale: directional ablation on a frontier-class open-weight model, DPO on the same, and persistence of both under subsequent fine-tuning. Fourth, a repeatable evaluation protocol for deployers, with a fixed panel and a cadence, since the longitudinal result implies that a one-time audit is not enough. Two further directions we rank below these: a civilian-sensitive pack arm, so that sector can be separated from region, and ablation of combined directions, so that the residual after single-direction removal can be attributed.

14 Conclusion

We asked whether LLM agents do the same work differently depending on who it is for, and found that seven of ten did, in a pattern that follows the model's country of origin with exceptions the data make plain. The effect is carried in good part by regional cues rather than by names, it is not produced by the judge or the apparatus, it is readable from a direction in the model's activations whose removal removes most of the behavior, it can be reduced but not yet removed, and it persists across months until a vendor changes the model. The practical conclusion rests on 56,826 paired executions: an agent that passes a neutral evaluation has not been shown to work for everyone, and counterfactual evaluation on the deployer's own contexts should precede deployment.

References

[1] Stein, S., for CrowdStrike Counter Adversary Operations (2025). CrowdStrike Research: Security Flaws in DeepSeek-Generated Code Linked to Political Triggers. CrowdStrike blog, 20 November 2025.

[2] World Economic Forum and Capgemini (2024). Navigating the AI Frontier: A Primer on the Evolution and Impact of AI Agents. White paper, 16 December 2024.

[3] Tamkin, A., Askell, A., Lovitt, L., Durmus, E., Joseph, N., Kravec, S., Nguyen, K., Kaplan, J., et al. (2023). Evaluating and Mitigating Discrimination in Language Model Decisions. arXiv:2312.03689

[4] Parrish, A., Chen, A., Nangia, N., Padmakumar, V., Phang, J., Thompson, J., Htut, P. M., Bowman, S. R. (2022). BBQ: A Hand-Built Bias Benchmark for Question Answering. Findings of ACL 2022, pp. 2086–2105. aclanthology.org/2022.findings-acl.165

[5] Nangia, N., Vania, C., Bhalerao, R., Bowman, S. R. (2020). CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models. EMNLP 2020, pp. 1953–1967. aclanthology.org/2020.emnlp-main.154

[6] Röttger, P., Kirk, H., Vidgen, B., Attanasio, G., Bianchi, F., Hovy, D. (2024). XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models. NAACL 2024, pp. 5377–5400. aclanthology.org/2024.naacl-long.301

[7] Hubinger, E., Denison, C., Mu, J., Lambert, M., Tong, M., MacDiarmid, M., Lanham, T., Ziegler, D. M., et al. (2024). Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv:2401.05566

[8] Wang, Y., Xue, D., Zhang, S., Qian, S. (2024). BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents. ACL 2024, pp. 9811–9827. aclanthology.org/2024.acl-long.530

[9] Yang, W., Bi, X., Lin, Y., Chen, S., Zhou, J., Sun, X. (2024). Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents. NeurIPS 2024. arXiv:2402.11208

[10] Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS 2023 Datasets and Benchmarks. arXiv:2306.05685

[11] Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., et al. (2024). AgentBench: Evaluating LLMs as Agents. ICLR 2024. arXiv:2308.03688

[12] Yao, S., Shinn, N., Razavi, P., Narasimhan, K. (2025). τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. ICLR 2025. arXiv:2406.12045

[13] Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., Pan, A., Yin, X., et al. (2023). Representation Engineering: A Top-Down Approach to AI Transparency. arXiv:2310.01405

[14] Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., Turner, A. (2024). Steering Llama 2 via Contrastive Activation Addition. ACL 2024, pp. 15504–15522. aclanthology.org/2024.acl-long.828

[15] Arditi, A., Obeso, O., Syed, A., Paleka, D., Panickssery, N., Gurnee, W., Nanda, N. (2024). Refusal in Language Models Is Mediated by a Single Direction. NeurIPS 2024. arXiv:2406.11717

[16] Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., Finn, C. (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. NeurIPS 2023. arXiv:2305.18290

[17] Durmus, E., Nguyen, K., Liao, T. I., Schiefer, N., Askell, A., Bakhtin, A., Chen, C., Hatfield-Dodds, Z., et al. (2023). Towards Measuring the Representation of Subjective Global Opinions in Language Models. arXiv:2306.16388

[18] Naous, T., Ryan, M. J., Ritter, A., Xu, W. (2024). Having Beer after Prayer? Measuring Cultural Bias in Large Language Models. ACL 2024, pp. 16366–16393. aclanthology.org/2024.acl-long.862

[19] Salnikov, M., Korzh, D., Lazichny, I., Karimov, E., Iudin, A., Oseledets, I., Rogov, O. Y., Loukachevitch, N., et al. (2025). Geopolitical Biases in LLMs: What Are the "Good" and the "Bad" Countries According to Contemporary Language Models. arXiv:2506.06751

[20] Pacheco, A. G. C., Cavalini, A., Comarela, G. (2025). Echoes of Power: Investigating Geopolitical Bias in US and China Large Language Models. arXiv:2503.16679

[21] Wang, B., Chen, W., Pei, H., Xie, C., Kang, M., Zhang, C., Xu, C., Xiong, Z., et al. (2023). DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models. NeurIPS 2023 Datasets and Benchmarks. arXiv:2306.11698

[22] Benjamini, Y., Hochberg, Y. (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society: Series B, 57(1), 289–300. doi:10.1111/j.2517-6161.1995.tb02031.x

Appendix A. Program timeline

• July 2025: pack pool built across nine regions and four salience levels; scenarios written in two batches and validated on neutral packs; tool registry built; self-hosting of 4 open-weight models commissioned; vendor usage policies reviewed; analysis plan preregistered.
• August 2025: main grid begins; placebo pairs; judge-panel calibration study (540 pairs scored three ways) and masking test; longitudinal panel fixed by rule after the first two weeks and month 1 run.
• September 2025: longitudinal panel month 2; harmful and simple controls.
• October 2025: main grid complete for the four provider-hosted models; scenario cross-validation.
• November 2025: main grid complete for all ten models; confound checks; salience ladder; Kimi-K2 self-hosting commissioned; GPT-5 default-alias update detected in the panel. The CrowdStrike report on politically triggered code vulnerabilities [1] appears on 20 November.
• 1 December 2025: vendor disclosure sent (main-grid results and methods).
• December 2025: probing, direction geometry and the self-hosted Kimi-K2 baseline; DeepSeek-V3 snapshot change detected in the panel.
• January 2026: directional ablation, including the held-out evaluation, and DPO.
• 1 February 2026: disclosure addendum sent (mechanism, mitigation, longitudinal results).
• February 2026: longitudinal panel month 7; program close.
• March 2026: this report, 90 days after the first disclosure.

Appendix B. Tool registry by category

Appendix C. Preregistered hypotheses and outcomes

Exploratory, not preregistered: the Indian-pack deficits in Chinese-origin models, the European model's Russian deficit, the DeepSeek-V3 and GPT-5 snapshot shifts, the held-out ablation result, and the cross-judge and placebo friction–quality correlations.

Appendix D. Compute