How AI Retrieves and Ranks Passages (Not Pages)

When an engine retrieves, it isn’t crawling the open web live and reading your latest page in real time. It searches an index, a pre-built, searchable copy of the web, and different engines use different indexes: their own, or a partner’s. Perplexity and ChatGPT lean on large web indexes; Google’s AI surfaces use Google’s index; Claude uses its own search backend. The practical effect is that the pool of candidate pages is different on every engine, and it is only ever as fresh and as complete as that index.

Two things follow. First, if your page isn’t in the index, it cannot be retrieved, no matter how good it is. That’s the whole subject of the companion series on why AI search engines can’t see your website. Second, being in the index is necessary but not sufficient: the index might hold millions of eligible pages, and only a few are pulled for any single query.

Before anything is indexed for semantic search, long pages are split into chunks, passages of roughly a few hundred words. This isn’t a stylistic choice; it’s a hard constraint. The models that encode text into searchable form and the models that generate answers both have token limits and can’t process an entire long document in one pass, so documents are broken into self-contained pieces that each carry one coherent idea.

This passage-level view isn’t unique to AI chat. Google moved the same direction years ago with passage ranking, announced in 2020: it started ranking a specific, highly relevant section of a page even when the rest of the page was about something else, affecting roughly 7% of queries. The lesson generalizes. Machines increasingly evaluate the passage, not the page.

The practical implication is about structure. If your best answer only makes sense after three preceding paragraphs of setup, the chunk that gets indexed may not stand on its own, and a passage that can’t stand on its own retrieves poorly. Sections that state their point directly, under a clear heading, survive chunking intact.

The engine matches on meaning. Both your passage and the query are turned into embeddings, lists of numbers that act as a fingerprint of meaning, so that two pieces of text that mean the same thing land close together even if they share no words. Finding relevant passages then becomes a geometry problem: retrieve the passages whose vectors sit nearest the query’s vector, measured by cosine similarity or dot product. This approach, dense passage retrieval, was shown to beat classic keyword search (BM25) substantially on open-domain question answering.

That’s why keyword stuffing does nothing here: matching is semantic, so a passage that genuinely explains a concept will be found under many different phrasings of the question. In practice most production systems run hybrid search, combining this semantic matching with old-fashioned keyword matching, so exact terms and names still matter, they just aren’t the whole game.

First-stage retrieval is built for speed, not precision. It casts a wide net and returns a large candidate set, often hundreds of passages, because at that scale each passage and the query are encoded separately and compared with fast math. That’s great for recall, but the ordering is rough. So a second stage re-reads the shortlist much more carefully.

That second stage is a reranker, typically a cross-encoder: instead of encoding the query and passage separately, it reads them together and scores how well that specific passage answers that specific query. It’s slower, so it’s only run on the shortlist, but it’s far more accurate. The technique that established this re-scored the top candidates from a fast first stage and improved ranking quality dramatically. Only the top few passages, often three to five, survive to be handed to the model.

Now the frustrating pattern resolves. ‘We have a great page and still don’t get cited’ almost always traces to one specific stage of this funnel, and each has a different fix.