The Snippet Does Not Survive

Summary

Prior work shows that a model's product ranking can be moved by optimized text placed in its context. In production, the context is assembled by a retrieval pipeline that truncates pages to a few hundred words, places them at positions the content owner does not choose, and paraphrases what it keeps, and the companion paper measures how rarely a page enters at all. This paper asks what happens to a ranking signal once content is in the context under those conditions.

Under a perturbation schedule of omission, paraphrase, displacement and competitor text, the strongest snippet method, a tree-of-attacks search (TOA), keeps the best clean rank and loses the most under every perturbation; a gradient-based token method shows no production effect beyond ±0.2; suffix generation does not survive the context. Two methods designed for the context do: Context-Position Independence (CPI), which spreads the signal across page regions so that no single omission removes it, and a preference-optimized whole-page rewriter (POR). CPI raises survival under omission from 18% to 64% and moves rank among named from 2.6 to 2.3 under real retrieval on ChatGPT.

Robustness

The laboratory simulator is rebuilt against the companion paper's production measurements (rank agreement with a real generator, Spearman 0.71) and evaluates every candidate under five conditions. TOA alone has the best clean rank and the worst perturbed ranks. POR trades clean rank for a flat profile. TOA with CPI recovers most of POR's flatness at a fraction of its cost, which is why it is the production stack. Survival under paraphrase rises from 11% to 47%, still below half.

Controlled production

Six engines run their own retrieval over result sets we hold. On ChatGPT's content tier, REO, the companion paper's rewrite, moves entry and mention and barely moves rank among named; CPI moves rank among named (2.6 → 2.3 alone, 2.2 → 1.9 on top of REO) and barely moves entry. The effects are close to additive.

Transfer and native confirmation

Measured in isolation, the ranking signal transfers from ChatGPT to Perplexity (0.78 of the rank gain retained) and much less to Claude (0.30). On real queries against customer sites, net of control products in held-out categories, the stack moves rank among named by −1.4 on ChatGPT and −1.2 on Perplexity. The effect is distinguishable from zero on four of six engines, and not on Claude or Google AI Mode.

Discussion

Every method we tested keeps its laboratory ordering under real retrieval, so laboratory comparisons are incomplete rather than wrong. A method's clean rank says how well it argues when heard; survival says how often it is heard whole. The second number is the one production rewards, and it is not measurable without a context model that truncates, displaces and paraphrases.

The complete methodology, results and limitations are in Full Research.

Abstract

Prior work shows that a model's product ranking can be moved by optimized text placed in its context. In production, the context is assembled by a retrieval pipeline that truncates pages to a few hundred words, places them at positions the content owner does not choose, and paraphrases what it keeps, and the companion paper measures how rarely a page enters at all. This paper asks what happens to a ranking signal once content is in the context under those conditions. We rebuild a laboratory simulator of the browsing context against production measurements (rank agreement with a real generator, Spearman 0.71) and add a perturbation schedule of omission, paraphrase, displacement and competitor text. Under it, the strongest snippet method, a tree-of-attacks search, keeps the best clean rank and loses the most under every perturbation; a gradient-based token method shows no production effect beyond ±0.2; suffix generation does not survive the context. Two methods designed for the context do: Context-Position Independence (CPI), which spreads the signal across page regions so that no single omission removes it, and a preference-optimized whole-page rewriter. We evaluate on four tiers of increasing realism, from the simulator to a controlled-production tier in which six engines run their own retrieval over result sets we hold, and confirm on real queries against customer sites with a concurrent control: CPI raises survival under omission from 18% to 64% and moves rank among named from 2.6 to 2.3 under real retrieval on ChatGPT. Transfer to Perplexity is high and to Claude low. A ranking signal has to be designed for entry, not just for the ranker.

1 Introduction

When a conversational search engine answers a product query, the page that describes a product influences the answer only if it is retrieved, if its content enters the model's context, and if the model, given that content among competitors', ranks the product favorably. A line of work has shown that the last step can be manipulated: Kumar and Lakkaraju [8] with an optimized text sequence, Pfrommer et al. [1] with a tree-of-attacks search that transfers to a production engine, Xing et al. [2] with short, fluent perturbations against listwise rerankers. All of it measures the effect with the manipulated text placed in the context by construction, whole, at a position the experimenter chose.

Production contexts are not like that. The companion paper [11] measures how a page reaches the context of six engines and finds that entry is a minority event, that an engine takes the leading few hundred words of a page and stops, that the page lands at a position the owner cannot choose, and that what enters is sometimes paraphrased. Our earlier, unpublished study (June 2025; "the first study" below) had already met those conditions without being able to name them: a snippet that won in the laboratory lost in production, and every unresolved issue it listed, under every method, was a placement failure.

This paper is about step (iii) under those conditions. Its question is whether a ranking signal can be made to survive entry, and its answer is that snippet methods cannot and page-level methods can. We make the following contributions.

1. A calibrated simulator and a perturbation schedule. The laboratory model of the browsing context samples entry, context share and position from the companion paper's measurements (the values are in Table 1) and evaluates every candidate under omission, paraphrase, displacement and full competitor text (Section 5).
2. Two methods designed for the context. Context-Position Independence (CPI) spreads the ranking signal across independent page regions so that no single omission, move or paraphrase removes it; a Preference-Optimized Rewriter (POR) produces the whole page under a closed loop against a judge panel (Section 4).
3. Negative results on two published methods. Rank-Anything-First [2] shows no production effect beyond ±0.2 rank; the suffix framing of GASP [3] does not survive the context (Section 6).
4. Evaluation under real retrieval. Four tiers: a simulator, an isolated production environment that bypasses retrieval, a controlled-production tier in which six engines run their own retrieval over result sets we hold, and a native confirmation on real queries against consenting customer sites with a concurrent control (Sections 5 and 6).
5. Transfer. The ranking signal transfers from ChatGPT to Perplexity and not to Claude, measured in isolation on the two engines whose interfaces accept a supplied page (Section 6.5).

Throughout, candidates are constrained to natural, non-instructional text that passes a four-stage guardrail filter; the rewrite that moves entry itself, Retrieval-Entry Optimization (REO), is the companion paper's subject and appears here only as a fixed preprocessing step. Methods are described at the level of design and metrics; optimized text and filter configurations are not reproduced here.

2 Related Work

Ranking manipulation. Kumar and Lakkaraju [8] optimize a strategic text sequence with a gradient-based procedure to raise a product's top-recommendation rate in a fictitious catalog. Pfrommer et al. [1] build a dataset of real product sites, show that LLMs weigh product names, content and context position differently, and adapt the Tree of Attacks with Pruning jailbreak [4] to promote low-ranked products, with demonstrated transfer to Perplexity. Xing et al. [2] propose Rank-Anything-First, a two-stage token optimization that produces fluent perturbations against listwise LLM rerankers. Nestaas et al. [7] introduce preference manipulation attacks, in which pages or plugin descriptions induce a model to favor the attacker's product and disparage competitors, demonstrated on Bing and Perplexity, and argue that the resulting dynamics are a prisoner's dilemma among content owners. We use [1], [2] and the fine-tuning idea behind [3] as ranking methods and measure them under the conditions these papers abstract away: partial entry, random position, truncation, paraphrase.

Jailbreak generation and guardrails. Tree of Attacks with Pruning [4] and GASP [3] are automated black-box generators of human-readable adversarial prompts; we repurpose their search procedures with a ranking objective and a benignness constraint, as the first study did. InjecGuard [5] is one of the guardrail models in our filter stage; Li et al. built it to reduce the over-defense of such models; we use it as a strict filter and accept the over-defense it was designed to reduce, since any text a guardrail would flag is also text a reasoning model may discard.

Generative engine optimization and entry. Aggarwal et al. [6] showed that content rewrites raise a source's share of a generated answer by up to 40%, with retrieval presupposed. Greshake et al. [9] and Zou et al. [10] treat retrieval as given or as a corpus the attacker writes into. The companion paper [11] measures retrieval and entry in production and introduces the rewrite that moves them; this paper takes entry as a conditioning event and asks what survives it.

3 Problem Setting

Setting. A content owner controls the page describing its product, publishes natural, non-instructional text that satisfies guardrail models [5], serves the same content to every client, and cannot choose whether, where or in what form the page enters an engine's context. For a query q, engine e and owned page p, let C(q, e) be the set of pages whose content enters the context and A(q, e) the generated answer. The companion paper measures P(p ∈ C); this paper conditions on it.

What entry imposes. From the companion paper's measurements, content that enters is truncated to the engine's context share (a median of 22% of page tokens on content-tier ChatGPT, about 462 words of a median page), lands at a position that is close to uniform on the page-bringing engines and result-ordered on the passage-bringing ones, and is paraphrased by the generator in a share of runs that the introspection harness exposes. The perturbation schedule of Section 5.3 reproduces these as omission, displacement, paraphrase and competitor text.

Metrics.

• Mean rank among named: the product's position among the products named in A, averaged over runs in which it is named; the scale is the number of products in the scenario. In the laboratory and isolated tiers every product is in the context, so this is the familiar rank; under real retrieval it is reported beside the canary rate so that rank given entry is well defined.
• Survival: P(rank ≤ 2 | perturbation), for a specified perturbation of the context.
• Mention rate: P(the product is named in A), reported where real retrieval makes it informative; its movement is the companion paper's subject.
• Canary rate: the share of runs in which the owned content demonstrably entered the context, by a canary phrase that the companion paper validates per engine; used here to condition.

Realism tiers. We evaluate in four environments of increasing realism: a laboratory simulator in which the context is fully modeled and controlled; an isolated production environment in which the engine's generator is real but retrieval is bypassed by supplying URLs directly; controlled production, in which the engine's full pipeline runs on long-tail queries whose top results we hold, so that retrieval is real but the result set and its competitors are known; and native production, in which the full pipeline runs on real queries against real customer sites. The laboratory assumes entry; the isolated tier forces retrieval and measures entry with a canary; the last two measure rank under real retrieval, the first with the competitor set controlled and the second without.

Other parties. The engine is a defender that can change what enters and how at any time; the companion paper measures what that cost over the program, and Section 8 lists it as a limitation here. The user is the affected party; Section 8 treats the native tier's effect on users as a cost. Competitors can run the same methods; we do not model the equilibrium.

4 Methods

The first study asked a single question: given that we enter the context, how do we ensure rank #1 with natural text that satisfies guardrail models? Its own production data answered that entering the context was the binding constraint (a canary rate of 34% with three to seven competitors), not what happened after entry. The program therefore split the question in two. Given that we are in the result set, how do we enter the context at all? That is Retrieval-Entry Optimization (REO), the companion paper's subject, which this paper applies as a fixed preprocessing step wherever a configuration names it. Given that we enter the context, how do we ensure rank #1 with natural text that satisfies guardrail models? That is the subject of the ranking methods (Sections 4.1 to 4.4) and of the Context-Position Independence track (Section 4.5), which targets the first study's unresolved issues directly.

Each method is described in the first study's format: the base method and why it was chosen, how it works, the major issues met and the modifications made to address them, minor modifications, and the issues that remain open. Methods are described at the level of design and metrics. Optimized text and filter configurations are not reproduced here.

Shared filter stage. Every candidate produced by any method passes four filters before it is scored: the generating model's built-in guardrails; Llama Prompt Guard 2 and a GPT-4.1 pass instructed to check for prompt injection; InjecGuard [5]; and, new in this program, a review by a reasoning model (o4-mini) that discards any candidate the reviewer describes as instruction-like. A candidate rejected by any filter invalidates its branch of the search.

4.1 Tree-of-Attacks (TOA)

Base method and why we chose it. Pfrommer et al. [1] adapt Tree of Attacks with Pruning [4] to ranking. It is the most robust documented attempt to influence production AI rankings in state-of-the-art models, and it includes documented success on Perplexity, indicating transferability between systems. In this program it is the reference method: every production number not explicitly attributed to another method is TOA on top of the companion paper's REO rewrite.

How it works. The method builds a search tree of candidate snippets over multiple iterations. At each step an attacking model generates several candidate snippets (branching); each is prepended to the product page text and fed to a recommender model to see how highly it ranks the target product (evaluation); the highest-scoring candidates are kept as the new leaves (pruning); and the process repeats until the attack reliably pushes the target to the top of the ranking distribution.

Modifications retained from the first study. The attacker is instructed not to generate adversarial content, and a filter stage invalidates leaves that trip any guardrail. The context model was extended with the components a realistic production scenario requires: position randomization, so that the owned content appears at a random position; user prompts and system prompts randomly biased against the owned content; dropout layers that randomly omit context with varying aggressiveness; and, barring dropout, the full text of competitors in the context. Competitor modeling was changed from keyword-dense synthetic pages to real citations that occurred naturally in production conversations, since competitors in production were much less clustered and much less keyword-dense than the original paper assumed. Baseline modeling was added to ensure empirical measurement, and fuzzy ranking extraction was replaced with an LLM-driven extraction for speed and precision.

Major issues and modifications in this program.

• Issue: A single attacker and a single recommender overfit to each other; snippets that won in the laboratory lost on the next model version. Modification: a panel of three attacker models (GPT-4.1, Claude Sonnet 4 and Gemini 2.5 Pro) and four judge models (the same three and Llama 3.3 70B), all pinned to dated snapshots for the program, with pruning on the panel's mean rank and a penalty on variance across judges. The program issued 1.2M judge calls for TOA and CPI; POR's judging is counted separately in Section 4.4.
• Issue: The reasoning model in agentic search flagged a share of snippets that had passed the filter stage. Modification: the o4-mini review filter described above. The flag rate for surviving leaves in agentic production runs fell to 4% (unconstrained attacker output had been flagged in 71% of agentic runs).
• Issue: Snippets tuned under the first study's simulator did not survive dropout or paraphrase in production. Modification: pruning runs under the rebuilt simulator's perturbation schedule (Section 5.3), so that leaves are scored on their worst perturbed rank rather than their best clean rank.
• Issue: Failure rates of 34% (Product A) and 41% (Product B) in the first study (runs that produced no final snippet, from configuration issues and the cost of generating snippets within API budget). Modification: retries with budget carry-over and a fixed per-run token cap. The failure rate is 11% for Product A and 13% for Product B.

Additional minor modifications. Baselines are carried over from the first study; a monthly re-estimate served as a drift check and never differed from the carried-over value by more than 0.3. The GPT-4.1 mini rank extraction is validated monthly against hand labels on a sample; agreement stayed above 0.95.

Unresolved issues. TOA still degrades under aggressive context shifts; the worst perturbed rank remains markedly worse than the clean rank even after pruning on it (Table 6). Cross-engine transfer is partial, and per-engine re-optimization recovers some but not all of it (Table 10). Cost scales with the judge panel: in a side experiment (n = 600, not tabulated), a panel-of-one run cost a quarter of a panel run and lost about a third of the robustness gain.

4.2 Rank-Anything-First (RAF)

Base method and why we chose it. Xing et al. [2] appeared in October 2025, in the program's final two months, and was added to the comparison at once. Its optimization is gradient-based rather than search-based, and its results did not include transferability testing to production, but it showed a more rigorous context model and better results than other similar learned methods. The authors report mean ranks between roughly 2 and 5 depending on model and category, without reaching #1.

How it works. RAF constructs an adversarial control prompt token by token and appends it to the target item's description in an LLM-based reranking pipeline. In Stage 1 it scans the whole vocabulary with gradient-based scoring to shortlist candidate tokens that both push the target toward rank 1 and look locally fluent; in Stage 2 it evaluates the candidates under exact ranking and readability losses, combining them with entropy-based dynamic weighting and temperature sampling to pick the next token. Repeating the process yields short, natural-sounding control snippets that promote the target in listwise rankings, evaluated across several open-weight rerankers, while remaining hard to flag as adversarial.

Major issues and modifications.

• Issue: Appended control sequences, as in the original method, did not survive the context in the first laboratory runs, as the first study had found for appended snippets under TOA. Modification: do not append new text; rewrite the original lightly with near-synonyms, using the rewrite machinery built for CPI (Section 4.5), with the swap accepted only if the rank score improves and the fluency gate passes, in at most one or two passes.
• Issue: The gradient stage needs white-box access, so RAF optimizes against open-weight rerankers and transfers blind to the production models. Modification: a transfer set of three open-weight rerankers (Llama 3.1 8B Instruct, Qwen2.5 7B Instruct and Mistral 7B Instruct v0.3, each prompted as a listwise reranker), with candidates kept only if they improve on all three. Transfer improved in the laboratory and not in production.

Additional minor modifications. The weight of the ranking loss was increased with hard perplexity cutoffs.

Unresolved issues. Not robust to not being in the immediate context; not robust to rewrites; weak cross-model transferability; weak cross-retrieval-system transferability, all four exposed by its first laboratory run (Section 6.2). None moved in the six weeks RAF was in the program, and it was retired from the production track.

4.3 Generative Adversarial Suffix Prompter (GASP)

Base method and why we chose it. There are strong similarities between jailbreaking an LLM and manipulating its rankings: in both cases the content works against the model's alignment, the process by which the model is constrained to its intended behavior. Basani and Zhang [3] is one of the papers that articulates how to automatically create state-of-the-art, effective jailbreaking suffixes that otherwise look benign.

How it works. GASP trains a suffix generator on a corpus of human-readable adversarial suffixes and embeds its outputs in a continuous latent space, so that search happens over embeddings rather than discrete tokens. A latent Bayesian optimization loop uses a black-box evaluator to score candidates while a readability constraint keeps suffixes natural. For each prompt, the most effective suffixes are compared against less effective ones and the generator is fine-tuned to favor the successful ones; top latent points are decoded back to text and the process repeats, producing fast, scalable, human-like candidates from black-box feedback alone.

Major issues and modifications.

• Issue: Generated snippets led to adversarial jailbreaking rather than rank manipulation. Modification (first study): start from our own Llama 3.1 8B generator fine-tuned on AI-preferred content, since no dataset of effective rank-manipulating snippets exists and we could not create one; change the evaluation function to measure a snippet's effect on document rank; and model a cheap version of SonicBerry for more effective evaluation, by issuing fanned-out queries to a self-hosted SearXNG instance configured for Google and Bing, inserting the retrieved documents directly into a browsing context, and adding a bge-reranker-v2-m3 cross-encoder with temperature-sampled scores to approximate the backend's non-determinism. All retained, with the rebuilt simulator (Section 5.2) as the evaluator.
• Issue: Once the generator was good, latent-space search gained nothing over sampling from the generator and keeping the best candidate. Modification: none within GASP; this observation is what moved the effort to the rewriter of Section 4.4.

Additional minor modifications. Experimentation with different layers for the latent space yielded no better results in the first study, and the same held with the larger generator. Experimentation with the SearXNG engine weights likewise.

Unresolved issues. The suffix framing itself. A suffix is a single snippet in a single position, which is exactly what the context does not preserve.

4.4 Preference-Optimized Rewriter (POR)

Origin and why we chose it. The first study elected to fine-tune a model using online direct preference optimization (DPO) within a reinforcement-learning-from-AI-feedback (RLAIF) loop. Because the objective is defined directly by AI-generated rankings, a closed optimization loop is possible: an optimizer model (Qwen3-8B, trained with online DPO) generates rewritten content, the judge panel of Section 4.1 ranks it against competitor items, the rankings are converted into preference pairs, and the pairs update the model's weights, after which the cycle repeats. In the first study this served GASP; in this program it is a method of its own. Every snippet method shares one weakness, that the snippet must survive being in a particular place in the page and in the context. A rewriter that produces the whole page has no single place to survive. It is also the only method here that could run as a deployed system rather than a research tool.

How it works. The loop above, with the judge panel ranking under the rebuilt simulator and its perturbation schedule, and a separate readability judge (Claude Sonnet 4) whose veto discards a pair. The program trained on 310k preference pairs over 6 cycles, 2,900 GPU-hours in total.

Major issues and modifications.

• Issue: Mode collapse toward keyword density after the third cycle; pages ranked well and read badly. Modification: the readability judge moved from a soft penalty to a hard veto, with a human review sample each cycle. Readability regressions were flagged in 7% of final outputs, concentrated in one vertical.
• Issue: The optimizer learned the simulator's quirks rather than the engines'. Modification: a share of each cycle's judgments is replaced by isolated-production outcomes, which are slow and expensive but real, so that the preference signal is partly grounded.
• Issue: Training data is per product, and the first of the four further products needed a cold start. Modification: a category-level base model, followed by a short product-level cycle.

Unresolved issues. An order of magnitude more work, compute and data than TOA, as the first study anticipated; per-product cost is acceptable only because the category base model amortizes it. Peak rank is slightly behind TOA in the laboratory; the advantage lies entirely in robustness and in not needing a snippet in the context's immediate neighborhood. Readability regressions are rare but visible to the site owner's readers.

4.5 Context-Position Independence (CPI)

Why this track. The first study's unresolved issues, under every method, were placement failures: not robust to not being in the immediate context, not robust to rewrites or non-minor omissions, highly susceptible to changes in the immediate context. Its 1vMany production result was not a ranking failure but a placement failure: the one snippet that carried the effect was dropped, moved or paraphrased. The companion paper shows truncation and position effects at entry, and its introspection harness shows paraphrase of fetched content. If the effect is spread across the page, no single omission removes it.

How it works. The owned content carries the ranking signal in several independent regions: the title and lede, a structured attribute block that describes the owner's product only, a question-and-answer block, and the closing summary; no competitor is named in any region, and the filter stage rejects a candidate that names one. Each region is optimized separately with TOA against the simulator under the condition that only that region survives, then all regions jointly. A near-synonym rewrite built for this track makes each region's phrasing self-contained, so that an engine's own paraphrase preserves the sense: it finds key words by masking and perturbation saliency, preferring early tokens and entities, chooses for each from a small set of close alternatives (same part of speech, embedding or masked-language-model neighbors, domain terms), and swaps only where a fluency gate passes. The measurement is survival: the share of runs in which the target still ranks in the top two after the perturbation.

Major issues and modifications.

• Issue: Regions optimized separately contradicted each other on the page. Modification: a joint pass after the separate ones, with a consistency judge.
• Issue: More regions meant longer pages, and longer pages lose context share (companion paper). Modification: regions are placed within the smallest measured context share among the target engines; the closing summary lies outside content-tier ChatGPT's share, is kept because Perplexity and Claude take it, and carries no signal that the earlier regions do not.

Unresolved issues. Survival under paraphrase is better but still below half. Gains are smaller on engines that bring passages rather than pages (Gemini, Google AI Mode), where only one region enters in any case.

5 Experimental Setup

5.1 Products and competitors

Six products: Product A and Product B, carried over from the first study so that results are directly comparable, and four further products across three verticals, reported pooled on a normalized rank scale (rank divided by scenario size). Each product has an owned blog set and a competitor blog set sourced from its platform. New in this program, each also has a competitor set drawn from 12,400 production conversations in which the product's category arose, so that competitors are modeled as they appear in generated answers.

5.2 Simulator

The first study's laboratory model of the browsing context (position randomization, biased user and system prompts, dropout layers, full competitor text) was rebuilt against the companion paper's measurements: entry, context share and position distributions are sampled from the per-engine values of Table 1 rather than set by hand. Fidelity is the Spearman rank correlation between simulator ranks and isolated-production ranks on identical content: 0.44 for the first study's simulator, 0.71 for the rebuilt one. All laboratory results use the rebuilt simulator.

Simulator parameters. The entry, context-share and position distributions are the companion paper's measurements; Table 1 prints the values the simulator samples from, so that this paper is self-contained.

5.3 Perturbation schedule

Each laboratory evaluation is repeated under five conditions: no perturbation; 30% random omission of the owned content (dropout); LLM paraphrase of the owned content; the owned content moved to the last position in the context; and the full text of every competitor present in the context. TOA pruning and POR training use the worst rank across conditions.

5.4 Evaluation tiers

Laboratory. Owned and competitor blog sets, context fully modeled and controlled, self-hosted Llama 3.3 70B and Qwen3-32B as the simulator's generators, and the OpenAI, Anthropic and Google APIs for the panel; 82,800 ranking runs over 12,820 unique content versions (37,000 TOA runs over 5,920 versions and 45,800 runs over 6,900 versions for the other methods, 6.5 evaluations per version on average), plus 36,000 evaluations under the perturbation schedule of Section 5.3 (2,400 per method and condition).

• Laboratory sample sizes include repeat evaluations of the same content version, typically five to eight per version in this program (down from fifteen to twenty in the first study, because the judge panel gives a more stable estimate per evaluation); unique version counts are stated separately.
• Not all runs produce a final snippet and thus a final ranking, so the sample size for final ranks is the number of attempted runs less the failures. Failure rates are reported in Tables 4 and 5.
• "Mean final rank" is over runs in which the product is named. "Canary and mention" columns restrict to runs in which the canary proves that the owned content entered the context and the product is named.

Isolated production, independent of the native search stack. As in the first study: owned and competitor URLs are supplied directly to the engine, bypassing its retrieval, with the model asked to rank the products and rankings extracted by a GPT-4.1 mini pass. For ChatGPT this is the consumer interface, through fresh, isolated sessions (to avoid organization-level caching and memory contamination) under the research agreement described in Section 8. For Perplexity and Claude, used only in the transfer measurement of Section 6.5, it is the URL-fetch tool of each provider's API. In addition to the optimized content, the page carries an instruction-style canary request, which the first study found to correlate one-to-one with the page entering the context on ChatGPT; it is appended after the filter stage, is used only to measure entry, and is absent from every native-production page. Three scenarios on ChatGPT: 1v1 (the owned URL and a single competitor URL), 1vMany (three to seven competitor URLs), and 1vCrowd (eight to twenty), the last new in this program; 17,800 runs.

Canary validity in this tier. The instruction-style canary is itself an instruction in a fetched page, which is the mechanism this paper does not rely on, and it could in principle move the rank it is used to condition on. We therefore ran the 1vMany scenario on the same URLs with and without the canary (1,200 runs per arm, an independent sample from Table 7), and separately compared the instruction-style canary with the passive canary of the owned network on the same pages (1,400 runs). Table 2 shows mention and rank indistinguishable between arms: the rank difference is 0.1 with a 95% interval of ±0.2. The passive canary recovered 65% of entries against 70% for the instruction-style one, the ChatGPT passive-canary sensitivity the companion paper reports. The instruction-style canary is retained in this tier for comparability with the first study.

Controlled production. 18 product clusters on the owned network of the companion paper (240 domains across 6 top-level domains, every client served the same page content, a canary sentence varying by requesting user-agent so that a response can be attributed to the user-agent that fetched the page). Each cluster holds the top three conventional results for 30 long-tail queries in a product's category (540 queries in all), and its three pages describe the owned product and two other products in the category from public information, so that the engine's result set contains 3 known products plus whatever it retrieves beyond the top three. The engines run their own retrieval on these queries through fresh sessions, twice a week, and every configuration of the stack is published on the owned page in turn, one version at a time, never concurrently. REO, the companion paper's rewrite of the leading section against the engine's observed fan-out queries, is applied as a fixed preprocessing step in the configurations that name it. Outcomes are the canary rate, the mention rate, and the mean rank among the products the response names. The competitive setting is artificial by construction, since we chose the result set; it is the native tier's job to relax that. 47,500 runs over the program.

Native production. 5 consenting customer sites, instrumented by the companion paper, with the engines' retrieval stacks untouched: a panel of 2,050 real queries run twice a week in independent sessions across the six engines, an eight-week baseline window before optimized pages were enabled (late July 2025 to late September 2025), an eight-week close-out window (October to November 2025), and 410 control queries in 4 product categories on the same sites in which no product was optimized, measured in the same weeks. A run is rankable when the response names at least two products in the category. This paper reports one outcome from the panel, rank among named net of the control (Section 6.6); entry, mention and the full design are in the companion paper.

5.5 Uncertainty

Intervals on proportions (Tables 2, 7, 8 and 9) are 95% Wilson intervals computed on an effective sample size n/d, where d is a design effect for clustering: repeat evaluations of the same content version in the laboratory and isolated tiers (d = 2.0, estimated from the between-version variance of baseline runs); cluster-weeks on the owned network (d = 1.6); and site-weeks in the native tier (d = 1.8), where the critical value is a t statistic on 40 − 1 degrees of freedom rather than 1.96, and rises to 2.78 if the site is taken as the cluster. Rank intervals are ± half-widths around the mean, computed over runs in which the product is named. The difference in differences of Table 11 uses a Wald interval on its four means with the same effective sample sizes and critical value. Runs per arm are stated in every table. The intervals are a floor on uncertainty: they do not capture engine-side changes between weeks, which the companion paper measures, and in the native tier they do not replace the concurrent control, which addresses bias rather than variance.

6 Results

6.1 Laboratory

TOA attains the best clean rank on every product (Product A 4.8 → 1.5, Product B 3.2 → 1.3); the first study's final ranks were 1.7 and 1.4, with failure rates of 34% and 41%, so the gain at scale is modest in rank and large in reliability. RAF reaches 1.9 on the seven-product scale (SD 1.1, n = 2,400), a normalized rank of 0.27 against the 0.53 of its first run on the first study's four-product laboratory setup (Section 6.2); it remains behind TOA. GASP remains the weakest ranking method, as in the first study (Product A 3.1, SD 1.5, n = 1,600; Product B 2.4, SD 1.2, n = 900); the generator trained for it is the part of that work that carried forward. POR is slightly behind TOA on clean rank (Product A 1.9, SD 1.1, n = 6,200; Product B 1.5, SD 0.8, n = 3,900); its value appears under perturbation.

6.2 Robustness, and the RAF negative result

TOA alone has the best clean rank and the worst perturbed ranks (Table 6, Figure 1); the table is the first study's list of unresolved issues expressed numerically. POR trades clean rank for a flat profile. TOA with CPI recovers most of POR's flatness at a fraction of its cost, which is why it is the production stack. In survival terms, the probability that Product A remains in the top two under 30% omission rises from 18% (TOA) to 64% (TOA + CPI), and under paraphrase from 11% to 47%.

RAF in production. RAF entered the program in October 2025. A first laboratory run on the first study's four-product setup, for comparability with its TOA baseline, gave 2.7 → 2.1 (n = 150); the seven-product comparison of Table 3 followed. In isolated 1vMany production, RAF moved mean rank from 3.8 to 3.6 (n = 1,100 runs in which the product was named), a change of 0.2 whose interval by the method of Section 5.5 is ±0.15, while the weekly baseline itself ranged from 3.6 to 4.1 over the same period. We report this as no improvement beyond ±0.2, not as a null; an attribution to rewriting and partial inclusion of the control sequence is consistent with the robustness profile of Table 6 but is not established by this measurement. The transfer-set modification improved laboratory transfer and not production transfer. RAF was withdrawn from the production track in November and retained as a laboratory baseline.

6.3 Isolated production

The first study's 1vMany figures were: baseline mention 68% (n = 875) and mean baseline rank 3.8 over 651 ranked runs, final mention 71% and mean final rank 2.9 over 698 ranked runs (both pairs quoted as reported; in each, the ranked count exceeds the mention rate times n, which the first study did not reconcile), canary rate 34%, mean final rank omitting no-canary responses 2.3 (n = 297). The rank conditional on entry moved less (2.3 → 1.8, of which the controlled-production ablation of Section 6.4 attributes about 0.3 to CPI); what moved most is the frequency with which the content enters at all (34% → 71%) and therefore the frequency with which the rank effect applies. The 1v1 scenario, which the first study had already found close to saturated (final rank 1.2 of 2, canary 94%), moves to 1.1 with canary 97%. In the 1vCrowd scenario, which the first study did not have and which is closest to a real "best product" answer, the stack moves mean rank from 9.6 to 5.4 and mention from 41% to 57%. Entry does not imply mention here: the canary appears in 50% of runs, the product is named in 46 of those points at a mean rank of 4.8, and the 11% of runs that name it without a canary average 7.9, which is a visibility effect more than a ranking effect.

The same six configurations were also run in this tier (1,300 to 1,600 runs per arm); their canary, mention and rank values agree with the controlled-production ablation of Section 6.4 within the intervals of Table 8, and are not tabulated separately.

6.4 Controlled production

This tier is where the stack meets real retrieval for the first time, under conditions we can still read: the engine chooses what to fetch and what to put in the context, but we know which products are in the result set. Table 8 gives the ChatGPT content-tier ablation and Table 9 the full stack on every engine.

The ablation separates the contributions under real retrieval (Table 8, Figure 2). REO moves entry and mention (35% → 61% and 44% → 66%, TOA only against TOA + REO) and barely moves rank among named; CPI moves rank among named (2.6 → 2.3 alone, 2.2 → 1.9 on top of REO) and barely moves entry. TOA alone, re-run at scale with panels and perturbed pruning, improves rank among named from 3.1 to 2.6 without changing how often the product is named, which is the first study's production result restated under real retrieval. The effects are close to additive, and the small departures favor the combination.

The per-engine picture (Table 9, Figure 3) follows the entry rates the companion paper reports. Perplexity moves most (55% → 78% mention, 2.8 → 1.7 rank), the content tier of ChatGPT next, and the metadata tier of ChatGPT least (3.9 → 3.6 rank among named, with mention near its floor), since the stack cannot reach a context it does not enter. Google AI Mode (24% → 32%) and Gemini move by their entry gains and little more. Mention exceeds the canary rate on every engine, because a product can be named from a result's title and description without its page content entering the context; rank among named is the quantity that depends on entry.

6.5 Cross-engine transfer

Transfer is measured on the isolated tier, where the URL of a page version is supplied directly, so that the comparison isolates the generator from retrieval. Only engines with an interface that accepts a URL without retrieval can be measured this way: ChatGPT through its consumer interface, Perplexity and Claude through their APIs' URL-fetch tools. Per-engine re-optimization produces a separate page version, published on the owned network in place of the previous one and never alongside it, so that at any moment every client sees one page (companion paper); the isolated tier supplies the URL of the version under test, and the two columns of Table 10 are therefore different weeks. The transfer to Perplexity reported in [1] holds (0.78 retained, 0.91 after re-optimization). Claude retains little (0.30, 0.46 after re-optimization), consistent with its reasoning-step filtering. Copilot, Gemini and Google AI Mode cannot be measured in isolation; their absolute native effects are in Table 11, and since the content served there was optimized for ChatGPT, Table 11 already includes whatever transfers. Because the isolated tier bypasses retrieval, these shares measure the transfer of the ranking signal, not of REO; the companion paper's rewrite carries the coverage for all six engines on the single page.

6.6 Native confirmation

The native panel (Section 5.4) gives the in-context outcome under real queries and real competitors: rank among named, net of the control products in the held-out categories. Table 11 reports it. The ranking stack moves rank among named by −1.4 on ChatGPT and −1.2 on Perplexity net of control; the effect is distinguishable from zero on ChatGPT, Perplexity, Copilot, Gemini and not on Claude, Google AI Mode, the same two engines on which the companion paper finds no mention effect. The engines fall in the same order as on the controlled-production tier (Table 9). The magnitudes are not comparable across the two tiers, since the native scale is set by however many products the engine names rather than by a result set of three; what carries over is the order and the sign. Whether the product is named at all is the companion paper's result.

7 Discussion

Pages, not snippets. Every snippet method shares a single point of failure: the snippet's location. POR, which rewrites the page, and CPI, which distributes the signal, are both answers to the same observation, and the robustness table separates them from the methods that are not. We read this as a statement about the production context rather than about any method: truncation, reordering and paraphrase are properties of entry, and a ranking signal must be designed for them.

Engines differ in kind. The six engines fall into three groups: page-bringing engines (ChatGPT content tier, Perplexity, and to a degree Copilot), passage-bringing engines (Google AI Mode, Gemini), and engines whose reasoning step filters what entered (Claude). The rank effect under real retrieval is largest in the first group and smallest in the second and third (Table 9), and transfer of the ranking signal from ChatGPT, measurable in isolation on two engines (Table 10), is high to Perplexity and low to Claude, consistent with the grouping. For the other three engines the grouping rests on the controlled-production results alone, and the separation is suggestive rather than clean.

Cost. TOA with a judge panel costs about four times a single-judge run and buys about a third more robustness; CPI multiplies that by the number of regions; POR costs an order of magnitude more than TOA in compute and data (2,900 GPU-hours, 310k pairs) and is viable only because a category-level base model amortizes it. The cheapest configuration that survives the context is TOA with CPI, which is why it is the production stack.

What this means for the ranking literature. Every method we tested keeps its laboratory ordering under real retrieval, so laboratory comparisons are not wrong; they are incomplete. A method's clean rank says how well it argues when heard; survival says how often it is heard whole. The second number is the one production rewards, and it is not measurable without a context model that truncates, displaces and paraphrases, which is what Section 5 supplies.

8 Limitations and Ethical Considerations

Measurement. The simulator's rank correlation with isolated production is ρ = 0.71; it is a rank correlation, not a share of variance explained, and it is measured against the isolated tier rather than native production. The simulator's parameters are the companion paper's measurements and inherit their limits, in particular that canary-based entry rates are lower bounds. The isolated tier ranks on one engine and measures transfer on two, the engines whose interfaces accept a supplied page. The controlled-production tier chose its own result sets, so its competitive setting is artificial; the native confirmation rests on 5 sites, which bounds its generality, and under site-level clustering its intervals widen by about 38%. Every measurement was taken against engines that changed during the program; the companion paper reports one backend change that removed the production effect for 19 days.

Scope. We do not measure effects on users beyond rank among named on the native panel, and we do not measure the equilibrium that Nestaas et al. [7] describe when many content owners optimize at once. Agentic-mode search is reported as small in the companion paper and is not analyzed here.

User impact. The native tier changed what real users were told on real queries, on 5 sites, for the duration of the treatment window; the companion paper reports the mention-rate effect and the full treatment, and the constraints are the same here: optimized pages carried only the site owner's own product information, naming no competitor and making no claim about one; every page passed the filter stage of Section 4 and the owner's approval; the same page was served to every client; and the effect is one of visibility among products the engines were already citing. Our research ethics board, with two external members, reviewed the tier on those conditions before it was enabled, and site owners could withdraw any page at any time.

Responsible disclosure. The pipeline findings this paper's simulator rests on were shared with the operators of the six engines in August 2025, before the native treatment window opened (companion paper); a description of the methods here at the level given was included. Optimized text, prompts and filter configurations were not shared and are not included here.

Ethics. Candidates were constrained to natural, non-instructional text, and the filter stage was applied to every one; a candidate that named a competitor was rejected. The automated, isolated sessions of the isolated tier on the ChatGPT consumer interface were covered by a research agreement with the interface's operator obtained before the program began; no other consumer interface was automated, which is why the isolated tier ranks on one engine and uses provider APIs for the transfer measurement. The OpenAI fine-tuning API and Vertex AI tuning behind the introspection harness were used on our own data within their published terms; the rewriter was trained on our own hardware. Optimized text, prompts and filter configurations are withheld from this paper.

9 Conclusion

Given that content is in the context, the question for a ranking signal is not how well it ranks but how much of it survives being there. Snippet methods, including the strongest published one, lose most of their effect under the omission, displacement and paraphrase that entry imposes; methods that spread the signal across the page or rewrite the whole page keep most of theirs, in the laboratory, in isolated production, under real retrieval on result sets we hold, and on real queries net of a control. The resulting effect is real and partial: survival under omission from 18% to 64%, rank among named from 2.6 to 2.3 on ChatGPT under real retrieval, −1.4 on real queries net of control. It transfers to Perplexity and not to Claude, and it is worth nothing on a page that does not enter, which is where the companion paper [11] begins.

Appendix A. Open problems by method

The first study closed each method with a list of unresolved issues; RAF, which entered the program in October 2025, is listed with the issues its first laboratory run exposed. The table records what this program did to each.

New open problems introduced by this program: survival under paraphrase is still below half; POR's cost and readability failure mode; transfer is measurable in isolation on two engines only; and the decay of every measurement against engines that change monthly (companion paper).

References

[1] Pfrommer, S., Bai, Y., Gautam, T., Sojoudi, S. (2024). Ranking Manipulation for Conversational Search Engines. EMNLP 2024, pp. 9523–9552. arXiv:2406.03589

[2] Xing, T., Li, J., Du, Y., Hu, X. (2025). Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization. arXiv preprint, October 2025. arXiv:2510.06732

[3] Basani, A. R., Zhang, X. (2025). GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs. NeurIPS 2025. arXiv:2411.14133

[4] Mehrotra, A., Zampetakis, M., Kassianik, P., Nelson, B., Anderson, H., Singer, Y., Karbasi, A. (2024). Tree of Attacks: Jailbreaking Black-Box LLMs Automatically. NeurIPS 2024. arXiv:2312.02119

[5] Li, H., Liu, X., Xiao, C. (2024). InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models. arXiv preprint. arXiv:2410.22770

[6] Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A. (2024). GEO: Generative Engine Optimization. KDD 2024. arXiv:2311.09735

[7] Nestaas, F., Debenedetti, E., Tramèr, F. (2025). Adversarial Search Engine Optimization for Large Language Models. ICLR 2025. arXiv:2406.18382

[8] Kumar, A., Lakkaraju, H. (2024). Manipulating Large Language Models to Increase Product Visibility. arXiv preprint. arXiv:2404.07981

[9] Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., Fritz, M. (2023). Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. AISec 2023. arXiv:2302.12173

[10] Zou, W., Geng, R., Wang, B., Jia, J. (2025). PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models. USENIX Security 2025. arXiv:2402.07867

[11] Natapov, A., Oz Cohen, T. (2025). Does Your Page Even Get In? A Measurement Study of Six Production AI Search Pipelines. sqrr.ai.