Does Your Page Even Get In?

Summary

Conversational search engines answer product queries by retrieving web pages into a model context and generating an answer from it. Work on manipulating the generated ranking assumes the manipulated page is in the context; this paper measures how often that assumption holds in production and shows that it can be moved. Using 240 owned domains that serve every client the same content, 48,200 canary-tagged page versions and 3.1M logged crawler requests, we map the retrieval pipelines of six engines (ChatGPT, Perplexity, Google AI Mode, Gemini, Claude, Copilot) and measure context entry, the probability that a retrieved page contributes its content to the context.

Entry is a minority event on five of the six engines (34% for content-tier ChatGPT) and is capped by a context share of a few hundred words. Retrieval-Entry Optimization (REO), which rewrites a page against the engine's observed fan-out queries, raises ChatGPT entry from 34% to 61%. On 5 consenting customer sites with a concurrent held-out control, it raises mention rate on real queries by 14 points on ChatGPT and 18 on Perplexity, with no detectable effect on Claude or Google AI Mode. One backend change removed the effect for 19 days, and query drift erodes it monthly.

Context entry

Entry is measured with a canary sentence unique to each page version and fetching user-agent, whose sensitivity is validated per engine (0.74 to 0.93), so every measured rate is a lower bound. On content-tier ChatGPT the median context share is 22%, about 462 words of a median 2,100-word page, and content beyond roughly the first 500 words rarely entered. Metadata-tier sessions are closed to on-page optimization.

Retrieval-Entry Optimization

REO rewrites the leading section of a page to answer as many of the engine's observed fan-out queries as possible, within a self-contained block of a few hundred words, with explicit entities and a visible last-updated signal. The same page is served to every client, and its conventional rank is a hard constraint on the rewrite. Query coverage was the strongest predictor of entry: pages covering fewer than 2 of the 6 fan-out queries observed for their category on ChatGPT entered 19% of the time, and pages covering 4 or more entered 68% of the time. In the controlled-production ablation, REO is the only component that moves entry and mention; the ranking methods of the companion paper change neither by more than two points.

Native production

A panel of 2,050 real queries was run twice a week against the customer sites, with 410 control queries in categories where no product was optimized, measured in the same weeks. Control mention rates rose by 1 to 3 points on every engine, drift that a pre-period comparison alone would have attributed to the treatment. Net of control, the effect is distinguishable from zero on ChatGPT, Perplexity, Copilot and Gemini, and not on Claude or Google AI Mode.

Conclusion

Context entry decides whether on-page influence is possible at all. It is a minority event on five of six engines, capped by a context share of a few hundred words, closed entirely on one retrieval tier, and uninfluenceable in its position. The effect of a rewrite that targets it is engine-specific and perishable. For the ranking-manipulation literature, the result is a recommendation to measure entry before measuring rank; the companion paper does the latter.

The complete methodology, results and limitations are in Full Research.

Abstract

Conversational search engines answer product queries by retrieving web pages into a model context and generating an answer from it. Work on manipulating the generated ranking assumes the manipulated page is in the context; this paper measures how often that assumption holds in production and shows that it can be moved. Using 240 owned domains that serve every client the same content, 48,200 canary-tagged page versions and 3.1M logged crawler requests, we map the retrieval pipelines of six engines (ChatGPT, Perplexity, Google AI Mode, Gemini, Claude, Copilot) and measure context entry, the probability that a retrieved page contributes its content to the context, with a canary whose sensitivity we validate per engine. Entry is a minority event on five of the six engines (34% for content-tier ChatGPT) and is capped by a context share of a few hundred words. We introduce Retrieval-Entry Optimization (REO), which rewrites a page against the engine's observed fan-out queries, and evaluate it on two production tiers: controlled production, where the engines run their own retrieval over result sets we hold, and native production, on 5 consenting customer sites with a concurrent held-out control. REO raises ChatGPT entry from 34% to 61% and, with a fixed ranking stack from the companion paper, raises mention rate on real queries by 14 points on ChatGPT and 18 on Perplexity net of control, with no detectable effect on Claude or Google AI Mode. One backend change removed the effect for 19 days and query drift erodes it monthly. Entry, not ranking, is where influence over these systems is decided.

1 Introduction

Product discovery is moving from ranked lists of links to synthesized answers. When a user asks a conversational search engine for the best product in a category, the engine issues queries to one or more web indexes, fetches a subset of the results into a model context, and generates an answer that mentions and orders a handful of products. The web page that describes a product therefore influences the answer only to the extent that (i) it is retrieved, (ii) its content enters the context, and (iii) the model, given that content among competitors', ranks the product favorably.

The research community has concentrated on step (iii). Aggarwal et al. [6] showed that rewriting a source can raise its visibility in generative answers; Kumar and Lakkaraju [8] and Pfrommer et al. [1] showed that optimized text can move a product to the top of a model's recommendation; Xing et al. [2] showed that short, fluent perturbations suffice. These studies share an experimental convention: the manipulated text is placed in the context by construction. Our earlier, unpublished study (June 2025; “the first study” below) adopted the same convention in the laboratory and then measured one of these methods against the production ChatGPT interface. The laboratory effect was large; the production effect, in the presence of three to seven competitors, was small, and a canary phrase embedded in the manipulated page appeared in only 34% of responses. In two thirds of runs the page had been supplied directly and its content never reached the model.

This paper is about steps (i) and (ii). Its premise is that retrieval and context entry are measurable, engine-specific, and optimizable, and that in production they decide more than step (iii) does. The companion paper [11] takes up step (iii): what happens to a ranking signal once content is in the context, and which methods survive the truncation, reordering and paraphrase that entry imposes. We make the following contributions.

1. A measurement methodology for context entry. Owned domains that serve every client the same page content, marked with a canary sentence unique to the fetching user-agent and the page version, let us attribute every model response to the page versions that entered its context. We validate the canary's sensitivity per engine against two reference instruments (Section 4.2).
2. Pipeline maps and entry rates for six engines. Discovery crawlers, live fetchers, indexes, fan-out, crawl-to-index latency, fetch-cache lifetime, entry rate, context share and entry position, including two retrieval tiers of ChatGPT of which one is closed to on-page influence (Sections 4.3 to 4.5).
3. Retrieval-Entry Optimization. A rewrite that treats entry as its own objective, modeled from the engines' observed fan-out queries, under a constraint that the same page serves every client and keeps its conventional ranking (Section 5).
4. Two production tiers under real retrieval. A controlled-production tier on the owned network, where the engines retrieve over result sets we hold, and a native tier on 5 consenting customer sites with a 2,050-query panel and a concurrent held-out control, which together separate the entry effect from drift (Sections 6 and 7).
5. Durability and negative results. One backend version change erased the production effect for 19 days; query drift erodes entry monthly; the metadata tier of ChatGPT and entry position are beyond on-page influence (Sections 7.4 and 8).

Throughout, every page is served identically to every client, optimized pages on customer sites are approved by the site owner, and the ranking stack placed on top of REO is held fixed and treated as a black box from the companion paper. Methods are described at the level of design and metrics; optimized text is not reproduced here.

2 Related Work

Generative engine optimization. Aggarwal et al. [6] formalized generative engines and showed on a 10,000-query benchmark that content rewrites (adding citations, statistics, quotations) raise a source's share of the generated answer by up to 40%. Their visibility metric attributes answer text to sources; it presupposes retrieval and does not model whether a source enters the context at all. Our entry rate is the missing prior, and REO is the rewrite that targets it rather than the answer.

Ranking manipulation. Kumar and Lakkaraju [8], Pfrommer et al. [1] and Xing et al. [2] optimize text against a model's ranking with the text already in context; Nestaas et al. [7] show that pages and plugin descriptions can induce a model to favor the attacker's product on Bing and Perplexity and argue that the resulting dynamics are a prisoner's dilemma among content owners. The companion paper [11] evaluates these methods under the conditions entry imposes. Here they appear only as a fixed stack that we place on top of REO so that the native tier has a ranking signal to carry.

Indirect prompt injection and retrieval poisoning. Greshake et al. [9] named indirect prompt injection and demonstrated it against LLM-integrated applications, including a production search assistant. Zou et al. [10] show that a handful of crafted passages in a retrieval corpus can steer a RAG system's answer to a chosen question. Both treat retrieval as given or as a corpus the attacker writes into directly. Our setting differs in that the content owner controls only its own pages, retrieval is a production system observed from outside, and the content must be natural and non-instructional; the one place an instruction appears is a measurement-only calibration page (Section 4.2). InjecGuard [5] is one of the guardrail models that every page in this program had to satisfy.

3 Threat Model and Problem Setting

Actors. A content owner controls one or more web pages describing a product. An engine operates a conversational search pipeline: a query classifier, a query fan-out step, one or more web indexes, a live fetcher, a selection and truncation step that assembles a browsing context, and a generator. Competitors control other pages in the same category. A user issues a natural-language product query and reads a generated answer.

Capabilities and constraints. The content owner can publish arbitrary content on its own pages and, with the site owner's consent, vary a canary phrase by requesting user-agent. It has no access to engine internals, cannot influence which competitor pages are retrieved, and observes the engine through crawler requests to its pages, through the engine's responses, and through the introspection harness of Section 4.2. Published content must be natural, non-instructional text that satisfies guardrail models [5]; the one exception is the measurement-only calibration page of Section 4.2, which is never optimized content and never appears on a customer site. Page content is identical for every client at any moment, so conventional search ranks the same page the engines retrieve.

Objectives and metrics. For a query q, engine e and owned page p, let R(q, e) be the set of pages the engine fetches, C(q, e) the set whose content enters the context, and A(q, e) the generated answer.

• Entry rate: P(p ∈ C | p ∈ R). Measured by a passive canary phrase unique to each page version appearing in A, with per-engine sensitivity validated in Section 4.2. For engines that serve from an index without a live fetch, R is the controlled top three of the cluster's conventional results.
• Context share: the fraction of p's tokens present in the context when p ∈ C, estimated from which of the page's depth markers (Section 4.2) appear in A.
• Entry position: where in the context p's content lands relative to competitor content, where an instrument exists to observe it.
• Mention rate: P(the product is named in A).
• Mean rank among named: the product's position among the products named in A, averaged over runs in which it is named. It is the companion paper's outcome; we report it only where it is needed to separate entry from ranking.

The content owner's objective is to maximize mention rate under the constraints above. The ranking literature optimizes rank conditional on entry; we treat entry rate as the first objective, because for p ∉ C every other quantity is undefined.

Other parties. The engine is also a defender: it can change retrieval tiers, selection and filtering at any time, and Section 7.4 measures what that costs the content owner. The user is the affected party: an optimized page changes which products the user is told about, and Section 9 treats that as a cost of the native tier rather than a side effect. Competitors can run the same program; we do not model the resulting equilibrium (Section 9).

Realism tiers. The companion paper evaluates in four environments; this paper uses the two in which retrieval is real. In controlled production the engine's full pipeline runs on long-tail queries whose top results we hold, so that retrieval is real but the result set and its competitors are known. In native production the full pipeline runs on real queries against real customer sites. Both measure entry under real retrieval; the first with the competitor set controlled, the second without.

4 Measuring Context Entry

The question of this section is the one the first study opened with: how does what we write reach the context of the model through search? We answer it for ChatGPT in detail, because it is the engine with the most complex and most variable pipeline, and then for five further engines with the same instruments.

4.1 Terminology

The following terms are used throughout. Several are OpenAI-internal names recovered from request metadata and tool traces; we keep them because they are the only stable identifiers the backend exposes.

• OAI-Index: OpenAI's proprietary web index. It crawls pages after ChatGPT has visited them, from minutes to hours later (median 2.6 h on our network), under the “OAI-SearchBot” user-agent, and stores page content rather than metadata alone. Its contents sometimes enter the browsing context.
• SonicBerry: OpenAI's search backend; “Sonic” is the internal name for search generally. It connects to several web indexes (Google, OAI-Index, and probably Bing) and, among other responsibilities, decides what enters the browsing context.
• Chat context: the context passed to the generator for producing response tokens: system prompt, retrieved memories, user messages, tool responses and more.
• Browsing context: the separate context the search tool uses. It is not reachable from the chat context. In recent tool versions it contains the user's custom instructions, probably not the user's memories, and, on public evidence that we did not independently verify, a summarization of the chat context filtered of user-identifiable information.
• Retrieval tier: whether a backend version inserts page content into the browsing context (content tier) or only result metadata (metadata tier). Only content-tier versions can be influenced by on-page text.
• Context entry, entry rate, context share, entry position and canary are defined in Section 3; entry position is where in the browsing context the owned content lands relative to competitor content, which affects rank independently of content.

4.2 Measurement infrastructure

Owned result-set network. The first study used dozens of sites, indexed in conventional search, to control specific result sets and serve different content to four user-agents. The program scaled this to 240 domains across 6 top-level domains, organized into 38 topical clusters so that each cluster holds the top three conventional results for a set of long-tail queries (1,140 in total). Unlike the first study, every client receives the same page content; what varies by requesting user-agent is only the canary phrase, so that a response can be attributed to the fetcher that retrieved the page. 13 AI user-agents are distinguished, grouped by vendor and function (Appendix B): for OpenAI, ChatGPT-User (the live fetcher, whether invoked through a URL-opening tool or as part of SonicBerry), OAI-SearchBot (the OAI-Index crawler, which arrives after ChatGPT-User has found a page not yet indexed) and GPTBot (the training-data crawler); for Perplexity, PerplexityBot and Perplexity-User; for Anthropic, ClaudeBot, Claude-User and Claude-SearchBot; and Googlebot, Bingbot, Applebot, Amazonbot and Meta-ExternalAgent. Unlisted clients receive the page with a default canary. Because the content is the same for every client, the REO rewrites of Section 5 reach Google- and Bing-served engines through Googlebot and Bingbot, and conventional ranking is part of the objective rather than a bystander. All requests stream to a central log store; the program logged 3.1M AI-crawler requests. From this data we derived the pipeline maps, latencies, cache behavior, entry rates and the week-by-week mix of backend versions reported below.

Canary corpus. Each page version carries one primary canary, a unique sentence placed in the leading section, and five depth markers, distinct sentences at 10, 25, 50, 75 and 90 percent of the page's length; 48,200 versions were deployed. Canaries and depth markers are short, grammatical sentences specific to the page (a dated review note, a localized unit, a named example), unique only in their wording, so that a human reader sees ordinary prose; site owners approved the sentences on their pages. A canary in a response attributes the response to one page, one version, one date and the user-agent that fetched it, without relying on citations, which several engines omit or merge. The depth markers give context share (Section 3).

Canary validity. The first study established a one-to-one relation between a page entering the context and its canary appearing in the response for an instruction-style canary, on ChatGPT, in isolated production only. The passive canary used on the owned network carries no instruction, so a generator may paraphrase it or leave it out, and its sensitivity has to be measured per engine. We measured it on a separate set of calibration pages on the owned network, which carry an instruction-style canary as the reference instrument, are excluded from every ranking measurement, and never appear on a customer site. The reference instrument's own one-to-one property was re-established on the two engines where the introspection harness exposes the context directly, ChatGPT and Gemini grounding, and is assumed on the other four; Table 1 states this per engine. Sensitivities range from 0.74 to 0.93. Every canary-based entry rate in this paper is therefore an underestimate: the true rate is approximately the measured rate divided by the sensitivity, and we report measured rates uncorrected, as lower bounds, lowest on Claude and Google AI Mode, which drop phrases that read as out of place.

Introspection models. To observe the search context from the generator's side, the first study used OpenAI's fine-tuning API to train a model that appends its full prior context and tool-call history to the end of every completion, and then used that model through the ordinary API to perform search. The procedure does not expose the full search context, because much of the work is performed by an opaque, separate model; it does expose what the generator receives, including the order of fetched content and the classifier output, which is what the ranking methods act on. In this program it became a standing harness, run weekly on two tuned models: a gpt-4.1 tuned through OpenAI's fine-tuning API, which retains the web search tool and exposes the ChatGPT search tool's context, and a Gemini 2.5 Flash tuned through Vertex AI supervised tuning, which retains Google Search grounding and exposes that context. No such harness exists for Perplexity, Claude, Copilot or Google AI Mode. Three backend behavior changes occurred during the program: the version change of early September 2025 (Section 7.4), a change in the ordering of fetched content in October, and a change in fan-out query count in November. The harness detected the first two before the production metrics did.

Access analytics. The per-domain, per-user-agent dashboard of the first study was extended with a per-engine entry-rate view and a canary-attribution view (which responses contained which page versions).

Content-management plugin and customer sites. The first study built a WordPress plugin that lets a site owner change the content served for specific user-agents, with UI-based overrides for the owner's own governance and API-based overrides so that the program could try many variants without returning to the owner each time, with the owner's consent. In this program the plugin's per-user-agent mappings carry only canary phrases; an optimized page, once the owner approves it, replaces the previous page for every client. The plugin was deployed on 5 customer sites alongside the owned network (two of the six products share a site). These sites are what made the native production tier possible: real products, real competitors, real queries, and the engines' retrieval stacks untouched. They received 212k AI-crawler requests in addition to the 3.1M on the owned network.

4.3 The ChatGPT retrieval pipeline

Backend versions and tiers. We observed the following SonicBerry versions in the wild over the program: alpha.sonicberry_v7 and current_sonicberry_unpaid_default throughout; current_sonicberry_paid and current_sonicberry_unpaid_oai early on; and alpha.sonicberry_v8 from early September 2025 (Section 7.4). The “alpha” versions insert data from websites' content into the browsing context (content tier); the default version inserts only metadata, in the form of result-page-like entries (metadata tier). The version a session receives depends on time rather than on account tier, and the content-tier versions appear to be subject to a global rate limit: the share of our probe sessions routed to a content-tier version fluctuated from week to week with no stable pattern by account tier, consistent with the first study's observation that access to the more expensive version varied over time. The generator behind the interface changed from GPT-4o to GPT-5 on 7 August 2025; the backend versions are independent of it, and Section 7.4 reports that the change did not move entry.

Search modes. ChatGPT has two search modes, which OpenAI's own documentation distinguishes as a non-reasoning search, in which the model passes the user's query to the search tool and relays the tool's answer without internal planning, and an agentic search, in which a reasoning model manages the search process, decides whether to keep searching, and trades latency for depth. We call them instant and agentic. On the customer sites, 81% of ChatGPT-User fetch patterns matched instant search (a burst of fetches within seconds of one another, consistent with the second turn described below) and 19% matched agentic search (one or two targeted fetches at irregular intervals); patterns were classified by inter-fetch timing and request count per session. Instant search is the more prevalent mode and the one this paper concentrates on.

Instant search. Instant search is triggered automatically when the system decides that it needs up-to-date information, through a classifier (sonic_classifier) that emits a search probability (search_prob), or by the user through the search control, which forces the function; the two paths are equivalent. From a panel of prompts with known outcomes, with the classifier output read from the tool-call history exposed by the introspection harness, we estimate the automatic decision boundary at search_prob ≈ 0.42; product-comparison and “best X for Y” prompts sit well above it. The search operates in turns. Each turn issues its own queries, mostly three, and the results of the first turn affect the queries generated in the second, so at least the second turn's queries are generated from the browsing context. Most runs have two turns, with exceptions. The first turn appears to search OAI-Index, possibly also Bing. The second turn is present only in the consumer interface, not in the API, and searches Google: the content-tier versions consistently fetch result URLs from Google through ChatGPT-User and insert some of the fetched content into the browsing context. These fetches pass through a continental-level HTTP cache whose lifetime we measured at a median of 24 min by serving timestamped content and observing when fetched content changed. If a fetched site is not yet in OAI-Index, a crawl is scheduled and the site is added later, frequently with a visit from the training-data crawler (GPTBot) as well.

Agentic search. Agentic search has finer-grained control over the process. The reasoning model decides for itself when to invoke the search tool, rather than relying on the classifier; it sends its queries to SonicBerry directly, often circumventing the tool's tendency to fan out into several similar queries; its turn count varies between one and two; and in every turn the use of Google specifically is apparent. This mode is less valuable to a content owner, being less prevalent and harder to affect. In the first study, an attempt to influence it through a web source failed: the reasoning model recognized the source as adversarial, ignored it, and logged possible malicious behavior in its reasoning trace. What changed in this program is that benign, filtered content does enter and is used; the agentic effects reported in Section 7 are small but no longer zero.

Indirect injection and reasoning-mode filtering. Indirect prompt injection through a fetched page [9] remains possible in the content-tier versions of instant search; we re-confirmed it monthly on controlled result sets, by the first study's method of finding a result set we control. It is not the mechanism of this paper. The companion paper's filter stage keeps optimized content on the benign side of that line: in agentic runs, the reasoning model flagged 71% of unconstrained attacker outputs and 4% of outputs that had passed the filter stage. All optimized content below passed the filter stage; the one exception is the measurement-only canary request of the companion paper's isolated tier, which appears on no page in this paper's tiers.

4.4 Pipelines of five further engines

We applied the same instruments to Perplexity, Google AI Mode, the Gemini 2.5 Flash API with Google Search grounding, Claude Sonnet 4 with its web search tool, and Microsoft Copilot: owned result sets, per-user-agent canaries, timestamped responses. Table 2 summarizes the components relevant to entry.

Perplexity is the closest analog of content-tier ChatGPT: wide fan-out, full page content in context, fast indexing. It is the easiest engine to measure and the one on which [1] demonstrated transfer. Google AI Mode and Gemini both serve from the Google index and mostly bring passages or snippets rather than pages; entry depends on a passage-selection step that we could observe but not model well, and indexing is slow for new domains and fast for already-ranking ones. Claude's search is agentic by construction, with the model deciding on queries and fetches, uses Brave's index for discovery, and applies the strictest reasoning-model filtering of the six; content enters generously when it enters. Copilot serves from Bing, indexes most slowly, and once indexed behaves like Perplexity with a smaller context share; its live fetcher could not be distinguished from the index crawler, so Table 2 lists Bingbot for both.

4.5 Entry rates and context share

Entry rates are measured on the owned network over the full program, for pages present in the top three of the fetched result set; for the two index-served engines, for pages in the controlled top three of the cluster's conventional results. “After REO” refers to pages rewritten under the Retrieval-Entry Optimization track (Section 5). Table 3 gives the rates and Figure 1 plots them.

Four observations shape the rest of the paper.

1. Entry at baseline is a minority event on every engine except Perplexity. On content-tier ChatGPT it is 34%, of the same order as the canary rate of the first study's production runs, although the two are different quantities: the first study's rate was conditional on a page supplied directly to the interface, this one on a page present in a fetched result set.
2. Crawl-to-index latency is dominated by the discovery crawler's scheduling, not by the live fetch. A page first fetched by ChatGPT-User appeared in OAI-Index results after a median of 2.6 h; roughly a tenth of pages took more than two days.
3. Context share is capped. Content-tier ChatGPT takes the leading portion of a page and stops: with a median owned page of 2,100 words, a median context share of 22% is about 462 words, and content beyond roughly the first 500 words rarely entered. This motivates the front-loading constraint of REO.
4. Entry position, where it could be measured, follows the engine. The harness exposes the order of fetched content on ChatGPT and on Gemini grounding. On ChatGPT, owned content landed in each available position at similar rates, so the position randomization of the first study's simulator was the right call; on Gemini, position followed conventional result order. Position was not measured on the other four engines or in the consumer interface. Metadata-tier sessions are closed to on-page optimization: entry moved from 6% to 9% under REO and consists of title and description only.

5 Retrieval-Entry Optimization (REO)

Why this track. Every production measurement in the first study was conditioned on a canary, and with three to seven competitors the canary appeared in 34% of responses: two thirds of the time, the page had been supplied and its content never reached the model, so nothing downstream could act. No ranking method addresses this. REO treats entry as its own objective.

How it works. Entry is modeled as a function of the page, the engine's fan-out queries and the engine's selection step. From the fetch logs we recover, per prompt category and engine, the queries each engine generates (ChatGPT's two turns of three, Perplexity's four to six, Google AI Mode's five to twelve), so for each owned page we can compute its coverage, the number of those queries whose answer the page's leading section plausibly contains. Pages are rewritten under three constraints and then handed to the ranking stack of the companion paper, which operates on top and is held fixed in this paper:

1. Query coverage. The leading section answers as many of the observed fan-out queries for its category as possible, in the engine's own phrasing patterns.
2. Front-loading. Whatever must enter sits inside the smallest context share among the engines the page targets: a leading block of a few hundred words that is also a self-contained passage, so that it serves content-tier ChatGPT and the passage-bringing engines alike. One page serves every engine; there are no per-engine variants (Section 4.2).
3. Entity and freshness signals. Explicit product and category entities early in the page, structured data, and a visible last-updated signal, which the fetch logs show alters recrawl scheduling.

Major issues and modifications.

• Issue: Query coverage pulled pages toward keyword stuffing, which conventional search punished; two clusters lost their controlled result sets in August. Modification: coverage is scored on the leading section only, and the conventional rank of the rewritten page is a hard constraint on the rewrite, since the same page is served to every client; on customer sites a rewrite is published only once the site owner approves it.
• Issue: Entry rate is noisy per page; a page's weekly entry rate moved by ±15 points with no change on our side. Modification: entry is measured per cluster, per engine, per week, with page-level numbers reported only when n ≥ 200.
• Issue: Metadata-tier SonicBerry does not read the page. Modification: none is possible. We report it and stop optimizing for it.

Unresolved issues. Fan-out queries drift with model and version changes, so coverage decays and must be refreshed (Section 7.4). Google AI Mode passage selection remains poorly modeled, and entry gains there are the smallest of the six engines. On the engines where entry position could be measured, it could not be influenced by on-page changes.

6 Experimental Setup

6.1 Products and the ranking stack

Six products: Product A and Product B, carried over from the first study so that results are directly comparable, and four further products across three verticals, reported pooled. On every tier the optimized page carries, on top of REO, the ranking stack of the companion paper (a tree-of-attacks snippet search with the signal spread across page regions, “TOA + CPI” there). The stack is held fixed in this paper and treated as a black box; where a configuration without it is needed to separate entry from ranking, it is named.

6.2 Controlled production

18 of the owned network's 38 clusters are product clusters: each holds the top three conventional results for 30 long-tail queries in a product's category (540 queries in all), and its three pages describe the owned product and two other products in the category from public information, so that the engine's result set contains 3 known products plus whatever it retrieves beyond the top three. The engines run their own retrieval on these queries through fresh sessions, twice a week, and each configuration (the unoptimized page, REO alone, and REO with the fixed ranking stack) is published on the owned page in turn, one version at a time (Section 4.2). Outcomes are the canary rate, which is the entry rate of Table 3 on the same runs, the mention rate, and the mean rank among the products the response names. The competitive setting is artificial by construction, since we chose the result set; it is the native tier's job to relax that. 47,500 runs over the program.

6.3 Native production

The customer sites, with the engines' retrieval stacks untouched. A panel of 2,050 real queries across the six products and 4 control categories, drawn from our platform's conversation data (Section 9) and from the customer sites' own search logs, is run twice a week in independent sessions across the six engines (26,240 treated and 6,560 control runs possible per engine over an eight-week window). A run is rankable when the response names at least two products in the category; the yield varies by engine and is reported in Table 6. 51,000 rankable responses fall in the eight-week close-out window (October to November 2025). The baseline is the same panel run against the unmodified sites in the eight weeks before optimized pages were enabled (late July 2025 to late September 2025); baseline-window run counts were within 5% of the close-out counts on every engine, and yield differed between the windows by at most 2 points, so the rankable-run denominator is not treatment-dependent in practice. The 410 control queries concern 4 product categories on the same sites in which no product was optimized; their pages were never rewritten, they are measured in the same weeks as the optimized pages, and because the control outcome is the mention of a control product, spillover through a shared product is impossible. They provide the concurrent control of Section 7.3.

6.4 Uncertainty

Intervals on proportions (Tables 3, 4, 5, 6 and 7) are 95% Wilson intervals computed on an effective sample size n/d, where d is a design effect for clustering: cluster-weeks on the owned network (d = 1.6), and site-weeks in the native tier (d = 1.8, from the between-site-week variance of baseline runs), where the critical value is a t statistic on 40 − 1 degrees of freedom rather than 1.96. The difference in differences of Table 7 uses a Wald interval on its four proportions with the same effective sample sizes and critical value. The site-week is the clustering unit because it is the level at which the design effect was estimated; if the site itself is taken as the cluster, the degrees of freedom fall to four, the critical value rises to 2.78, and every native interval widens by about 38%. Section 7.3 reports which effects survive under both choices. Runs per arm are stated in every table. The intervals are a floor on uncertainty: they do not capture engine-side changes between weeks, which Section 7.4 treats separately, and they do not replace the concurrent control of Section 7.3, which addresses bias rather than variance.

7 Results

7.1 Retrieval-Entry Optimization on the owned network

Entry rates before and after REO are those of Table 3 and Figure 1, measured on the same runs as the controlled-production tier below: 34% → 61% on content-tier ChatGPT, 52% → 70% on Perplexity, and 6% → 9% on the metadata tier, which is closed. The single strongest predictor of entry was query coverage: pages whose leading section covered fewer than 2 of the 6 fan-out queries observed for their category on ChatGPT entered 19% of the time; pages covering 4 or more entered 68% of the time. Front-loading accounts for most of the remainder: a page whose relevant content began beyond the first few hundred words entered at close to the baseline rate regardless of coverage. Entry position did not move under REO on either engine where it could be measured (Section 4.5).

7.2 Controlled production

This tier is where REO meets real retrieval under conditions we can still read: the engine chooses what to fetch and what to put in the context, but we know which products are in the result set. Table 4 separates REO from the ranking stack on ChatGPT's content tier; Table 5 and Figure 2 give the full stack on every engine.

REO is the only component that moves entry and mention (35% → 61% and 44% → 66%, TOA only against TOA + REO); the ranking methods, alone or combined, change neither by more than two points. That is the result the first study's production data implied and could not show: a ranking signal that never enters the context has no effect, and the methods that make it enter are not ranking methods.

The per-engine picture follows the entry rates of Table 3. Perplexity moves most (55% → 78% mention), the content tier of ChatGPT next, and the metadata tier of ChatGPT least, since no rewrite reaches a context the page does not enter. Google AI Mode (24% → 32%) and Gemini move by their entry gains and little more. Mention exceeds the canary rate on every engine, because a product can be named from a result's title and description without its page content entering the context.

7.3 Native production

Native effects are smaller than controlled-production ones (Table 5), as expected: the native pipeline includes the metadata-tier sessions we cannot influence (29% of ChatGPT sessions in the window) and the result positions we do not control on real queries. The canary rates of Table 6 are unconditional on retrieval; the owned page was retrieved for roughly half to three fifths of panel queries (Table 6), and that retrieval rate together with the tier mix accounts for the gap between the entry rates of Table 3 and the canary rates here. On ChatGPT, mention rises from 23% to 39% and rank among cited from 4.1 to 2.6. Perplexity shows the largest native effect (31% → 52%, rank 3.4 → 2.1), consistent with its entry rate and context share. Copilot (17% → 29%) and Gemini (14% → 21%) track entry rate. Claude (19% → 24%) and Google AI Mode (12% → 16%) move least, and the concurrent control below shows that neither change is distinguishable from zero.

Concurrent control. The pre-period baseline is confounded with anything that changed in the engines over the program. Table 7 therefore compares, within the same weeks, the optimized products with control products in held-out categories on the same sites (Section 6.3). Control mention rates moved by 1 to 3 points over the program, upward on every engine, which is the drift a pre-period comparison alone would have attributed to the treatment. The difference in differences removes it, and its interval decides which effects survive. With site-weeks as the clustering unit, ChatGPT, Perplexity, Copilot and Gemini are distinguishable from zero and Claude and Google AI Mode are not; with the site as the cluster (Section 6.4), the same four effects survive both choices. We report no effect on Claude or Google AI Mode. Figure 3 plots the differences with their site-week intervals. The ordering of the engines by effect size is unchanged by the correction.

The same control applied to rank among named, the companion paper's outcome, gives −1.4 on ChatGPT and −1.2 on Perplexity net of control and is reported there.

7.4 Durability

Three backend changes occurred during the program (Section 4.2). Two of them measured the lifetime of an optimized state; the third, the change in fan-out query count in November, fell inside the close-out window, moved the owned-network entry rate by under two points, and is not treated separately. A generator change also fell inside the program: ChatGPT's default model changed from GPT-4o to GPT-5 on 7 August 2025, inside the native baseline window. Owned-network entry rate and isolated-production rank moved by less than their weekly variation across it, which is what one expects if retrieval rather than the generator decides entry.

• In early September 2025, alpha.sonicberry_v8 replaced v7 for a growing share of sessions. For the production stack of the time, the companion paper's isolated-production rank moved from 2.6 to 3.4 within a week, while the owned-network entry rate was unchanged in the same weeks, which locates the change downstream of retrieval; in the introspection harness, context share and ordering looked different. Re-optimization of the ranking stack against updated context parameters (companion paper) recovered the previous level after 19 days. The customer sites were in their baseline window at the time and were not affected.
• Independently of any version change, REO coverage for a fixed page decayed by about 3 points of entry rate per month as the generated fan-out queries drifted. A monthly refresh held entry rate flat; a set of pages deliberately left unrefreshed from August lost about 3 points a month through November.

Both argue for treating optimization as a cadence rather than an event.

8 Discussion

Entry is the binding constraint. The ranking literature measures step (iii) under the assumption that the manipulated text is in the context. In production, content supplied directly to the ChatGPT interface entered the context in a third of three-to-seven-competitor runs in the first study, and a retrieved page entered in a third of runs on the owned network here. Of the components in the production stack, the one that moved entry and mention is the one that does not act on ranking at all (Table 4); the ranking methods, re-run at scale, land where the first study did. We expect the same to hold for any ranking method evaluated only with its text in context, including [2] and [8], and the companion paper [11] measures how much of such a method survives entry once it happens.

Engines differ in kind. The six engines fall into three groups: page-bringing engines (ChatGPT content tier, Perplexity, and to a degree Copilot), passage-bringing engines (Google AI Mode, Gemini), and engines whose reasoning step filters what entered (Claude). Entry is highest in the first group and lowest in the second and third (Table 3), the REO gain tracks entry (Table 5), and the native effect net of control is detectable only in the first group and on Gemini (Table 7). The grouping is suggestive rather than clean: Copilot and Gemini sit close together on every measure.

Implications for engines. Our measurements suggest where defenses would bind. Metadata-tier retrieval is closed to on-page influence, at the cost of answer quality. Passage selection, as in Google AI Mode, limits entry to what the engine chooses rather than what the page front-loads. Reasoning-mode review discounts text that reads as instruction; our content is not instruction, and the discount is partial. Because the program served one page to every client, nothing here depends on per-user-agent content differences; an engine comparing its crawler's fetch with an anonymous fetch would find the page identical apart from the canary sentence, and that comparison remains a sound defense against variants of this work that do serve different content to crawlers.

Cadence. One backend change cost the full effect for 19 days, and query drift removes about 3 points of entry rate per month without any change on the engine's side. An optimized state therefore needs a refresh cadence of weeks to months, and the program's durable output is the measurement network and that cadence rather than any particular content.

9 Limitations and Ethical Considerations

Measurement. Canary-based entry rates are lower bounds on every engine; the true rate is approximately the measured rate divided by the sensitivity of Table 1, which is lowest on Claude and Google AI Mode, and we report measured rates uncorrected. The reference instrument behind Table 1 is itself validated on two engines and assumed on four. The native tier rests on 5 sites, which bounds the generality of Table 6; the held-out control shares those sites, so a site-level confound would survive it, although a product-level one cannot; under site-level clustering every native interval widens by about 38% and the same four effects survive both choices (Section 7.3). Confidence intervals use a design-effect approximation rather than a full cluster bootstrap and are floors on uncertainty (Section 6.4). Four of six products are pooled and anonymized. Every measurement was taken against engines that changed during the program, and Section 7.4 shows the consequences.

Scope. We do not measure effects on users, such as whether optimized content changes purchase decisions or answer quality, and we do not measure the equilibrium that Nestaas et al. [7] describe when many content owners optimize at once. Agentic-mode effects are reported as small and are not analyzed further.

User impact. The native production tier changed what real users were told on real queries, on 5 sites, for the duration of the treatment window. We do not consider this harmless and did not treat it as such. Four constraints bounded it: the optimized pages carried only the site owner's own product information, rewritten, naming no competitor and making no claim about one, and nothing the owner would not publish; every page passed the guardrail filters of the companion paper's stack and the owner's approval; the same page was served to every client, so users of conventional search and users of AI search saw the same content; and the effect is one of visibility among products the engines were already citing, not the insertion of a product into a category it does not belong to. The magnitude is in Table 7. Our research ethics board, with two external members, reviewed the tier on those conditions before it was enabled; site owners could withdraw any page at any time through the plugin, and were given the results. No user was identified or contacted, and we do not measure effects on users beyond what is in Table 7, which the Scope paragraph above lists as a limitation. What we cannot bound is the aggregate: if every content owner ran this program, the equilibrium described by Nestaas et al. [7] follows, and answer quality falls for everyone.

Responsible disclosure. The pipeline findings of Section 4 (crawl-to-index latencies, cache lifetimes, the entry and context-share behavior of each tier) were shared with the operators of the six engines in August 2025, before the treatment window of the native tier opened, together with a description of the REO and CPI tracks at the level given here. Two operators acknowledged; one requested the canary corpus to test a detector and received it after the close-out window ended, so that the native measurements were not affected; none requested an embargo. Optimized text, prompts and filter configurations were not shared and are not included here.

Ethics. Page content was identical for every client on every site in the program; per-user-agent mappings carried only canary sentences, on domains we own or on customer sites with the owner's consent. Optimized content was constrained to natural, non-instructional text, and the filter stage was applied to every candidate. The production conversation data used for competitor modeling and for the query panel is our own platform's, collected under terms that permit research use, de-identified before use and reduced to product-category mentions and query text; no engine user's data was collected. The OpenAI fine-tuning API and Vertex AI tuning were used on our own data within their published terms, which do not restrict what a tuned model emits about its own context. The automated sessions on the ChatGPT consumer interface, each run in a fresh, isolated session, were covered by a research agreement with the interface's operator obtained before the program began; no other consumer interface was automated. Monthly injection re-confirmation ran only on result sets we control and only with content we own. Optimized text, prompts and filter configurations are withheld from this paper.

10 Conclusion

We measured context entry across six production AI search engines and found it to be the step that decides whether on-page influence is possible at all: a minority event on five of six engines, capped by a context share of a few hundred words, closed entirely on one retrieval tier, and uninfluenceable in its position. A rewrite that targets entry rather than ranking moves it, under real retrieval on result sets we hold and on real queries against customer sites net of a concurrent control: 34% to 61% entry on content-tier ChatGPT, and +14 points of mention on real ChatGPT queries. The effect is engine-specific and perishable, removed for 19 days by one backend change and eroded monthly by query drift. For the ranking-manipulation literature, the result is a recommendation to measure entry before measuring rank; the companion paper [11] does the latter. For engines, it is a map of where retrieval, not generation, decides what their users are told.

References

[1] Pfrommer, S., Bai, Y., Gautam, T., Sojoudi, S. (2024). Ranking Manipulation for Conversational Search Engines. EMNLP 2024, pp. 9523–9552. arXiv:2406.03589

[2] Xing, T., Li, J., Du, Y., Hu, X. (2025). Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization. arXiv preprint, October 2025. arXiv:2510.06732

[3] Basani, A. R., Zhang, X. (2025). GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs. NeurIPS 2025. arXiv:2411.14133

[4] Mehrotra, A., Zampetakis, M., Kassianik, P., Nelson, B., Anderson, H., Singer, Y., Karbasi, A. (2024). Tree of Attacks: Jailbreaking Black-Box LLMs Automatically. NeurIPS 2024. arXiv:2312.02119

[5] Li, H., Liu, X., Xiao, C. (2024). InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models. arXiv preprint. arXiv:2410.22770

[6] Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A. (2024). GEO: Generative Engine Optimization. KDD 2024. arXiv:2311.09735

[7] Nestaas, F., Debenedetti, E., Tramèr, F. (2025). Adversarial Search Engine Optimization for Large Language Models. ICLR 2025. arXiv:2406.18382

[8] Kumar, A., Lakkaraju, H. (2024). Manipulating Large Language Models to Increase Product Visibility. arXiv preprint. arXiv:2404.07981

[9] Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., Fritz, M. (2023). Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. AISec 2023. arXiv:2302.12173

[10] Zou, W., Geng, R., Wang, B., Jia, J. (2025). PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models. USENIX Security 2025. arXiv:2402.07867

[11] Natapov, A., Oz Cohen, T. (2025). The Snippet Does Not Survive: Evaluating Ranking Manipulation Under Production Context Assembly. sqrr.ai.

Appendix A. Program timeline

• July 2025: owned network expanded to 240 domains; user-agent split extended to 13; canary corpus and attribution view live; pipeline mapping of the five additional engines starts; customer sites instrumented; native panel baseline window opens late in the month.
• August 2025: simulator rebuilt against the first weeks of measurement; four further products onboarded; REO track starts; first measurable move in ChatGPT entry rate; two clusters lose conventional rankings to a keyword-dense rewrite and the leading-section-only scoring rule is adopted; CPI track starts late in the month; pipeline findings disclosed to the six engine operators.
• September 2025: alpha.sonicberry_v8 appears early in the month; isolated-production rank for the TOA + REO stack regresses for 19 days; POR v1 trained; optimized pages enabled on the customer sites late in the month with owner approval.
• October 2025: RAF [2] appears and is added to the laboratory comparison and the isolated tier; change in the ordering of fetched content detected by the introspection harness; close-out window opens.
• November 2025: change in fan-out query count; RAF withdrawn from the production track; eight-week close-out window ends; program close; canary corpus shared with the operator that requested it; this report.

Appendix B. AI user-agents distinguished on the owned network

Every client, listed or not, receives the same page content; only the canary phrase differs, so that a response can be attributed to the user-agent that fetched the page. Google-Extended is a robots.txt token rather than a request user-agent and is therefore not listed.