BLUF: Modern AI search is rarely one search technique. The strongest systems usually combine fast keyword retrieval, semantic retrieval, rank fusion, and a more precise reranking step before giving a small set of passages to an LLM.
When people first learn Retrieval Augmented Generation, or RAG, the retrieval step can look almost magical:
User question
↓
Retrieve relevant information
↓
LLM
↓
AnswerBut the box labeled retrieve relevant information can contain an entire search architecture.
This article opens that box and explains the pieces in plain language, with examples.
Start with a simple search problem
Imagine a company knowledge base containing thousands of documents. One document says:
Employees may work from home up to three days each week with manager approval.
A user asks:
How many days can I work remotely?
The words are not identical. The document says work from home. The user says work remotely.
This small example explains why modern search systems often use more than one retrieval method.
1. Sparse retrieval: search by words
Traditional search engines are very good at finding documents that contain the same words as the query. Algorithms such as BM25 score documents based on which terms appear, how important those terms are, and how often they occur.
Query: remote work policy
Document A: Remote work policy for employees → strong match
Document B: Employees may work from home... → weaker lexical match
Document C: Office cafeteria opening hours → poor matchThis is often called sparse retrieval because documents and queries are represented using a very large vocabulary where most terms have no value for a particular document.
Sparse retrieval is especially useful when exact wording matters.
- product names
- error codes
- account identifiers
- people and place names
- exact phrases
If someone searches for ERR-4821, a semantic model does not need to understand the philosophical meaning of the code. Exact matching is exactly what we want.
2. Embeddings: represent meaning as numbers
Semantic search approaches the problem differently. It converts text into a numeric representation called an embedding.
"work from home"
↓
embedding model
↓
[0.18, -0.42, 0.73, ...]The numbers themselves are not useful to a person. What matters is where the vectors sit relative to one another.
Text with related meaning tends to produce vectors that are closer together than unrelated text.
"work from home"
↕ close
"remote work"
"office cafeteria"
↕ far awayThis allows a system to find useful content even when the exact words differ.
3. The encoder creates the embedding
An encoder is the model that turns text into a machine usable representation.
Input text
↓
Tokenization
↓
Encoder
↓
Vector representationFor search, we are often interested in an encoder that creates one useful vector for a query or document.
For example:
"How many days can I work remotely?"
↓
encoder
↓
query vector4. Bi encoders make semantic search scalable
A bi encoder, also called a dual encoder, processes the query and documents separately.
Documents Query
↓ ↓
document encoder query encoder
↓ ↓
document vectors query vector
└──────────── similarity ─────┘This separation matters because the document vectors can be calculated ahead of time and stored in a vector index.
When a user searches, only the query needs to be encoded.
1,000,000 documents
↓ done earlier
stored vectors
new query
↓
query vector
↓
find nearest vectorsThat makes bi encoders practical for searching very large collections.
5. Dense retrieval: search by semantic similarity
When search is performed using these dense embedding vectors, we usually call it dense retrieval.
Return to our example:
User query:
"How many days can I work remotely?"
Possible semantic matches:
1. "Employees may work from home up to three days each week..."
2. "Hybrid employees can use a home office..."
3. "Remote access requires multifactor authentication..."The first document may rank highly even though the user never typed the phrase work from home.
That is the main strength of dense retrieval: it can match concepts rather than only matching literal words.
Dense retrieval is not automatically better
It is tempting to conclude that vector search replaces keyword search. In practice, that is often a mistake.
Consider another query:
ERR-4821Keyword search may find the exact troubleshooting document immediately. Dense retrieval may also return documents about similar errors that are conceptually related but wrong.
Now consider:
"Why does the application sign me out after I leave it idle?"The relevant document might be titled:
Session timeout policy
Dense retrieval can help connect those two ideas.
So the real question is usually not:
Should we use keyword search or vector search?
It is:
How should we combine them?
6. Hybrid search combines sparse and dense retrieval
Hybrid search runs more than one retrieval method and combines the results.
User query
↓
┌──────────┴──────────┐
↓ ↓
Sparse retrieval Dense retrieval
BM25 embeddings
↓ ↓
ranked results ranked results
└──────────┬──────────┘
↓
fusion
↓
combined rankingThis gives the system two useful signals:
- Lexical signal: Do the words match?
- Semantic signal: Does the meaning match?
Suppose the query is:
reset employee VPN password
Sparse search might rank an exact phrase very highly. Dense search might also find a document titled Recovering remote access credentials. Hybrid retrieval gives the system a chance to use both.
7. Why we cannot always just add the scores
At first it seems easy to combine the results:
final score = BM25 score + vector similarity scoreThe problem is that the two scoring systems may operate on completely different scales.
BM25 score 18.7
Vector similarity 0.84Adding them directly can make one system dominate simply because its numbers are larger, not because its ranking is better.
There are several ways to solve this. One of the most practical is Reciprocal Rank Fusion.
8. RRF combines rankings instead of raw scores
Reciprocal Rank Fusion, usually shortened to RRF, looks mainly at where a document appears in each ranked list.
Imagine two retrieval systems return:
Keyword ranking Semantic ranking
1. Document A 1. Document B
2. Document B 2. Document A
3. Document C 3. Document DDocument A and Document B both appear near the top of both lists. RRF rewards that agreement.
The common form is:
RRF score = Σ 1 / (k + rank)You do not need to memorize the formula. The useful idea is simpler:
A document that ranks well in multiple retrieval systems should receive a strong combined rank.
Because RRF uses rank positions instead of trying to compare incompatible raw scores, it is a convenient way to combine retrieval systems.
9. Retrieval gets candidates. Reranking chooses the best ones
Fast retrieval systems are designed to search large collections quickly. They are good at finding a useful candidate set, but their ordering is not always perfect.
Suppose hybrid retrieval finds 50 candidate passages.
1,000,000 passages
↓
fast retrieval
↓
50 candidatesAt this point we can afford to use a slower but more precise model.
That stage is called reranking.
50 candidate passages
↓
reranker
↓
10 best passagesA reranker does not usually search the entire corpus. It improves the ordering of a much smaller candidate set.
10. Cross encoders are powerful rerankers
A common reranking model is a cross encoder.
A bi encoder processes the query and document independently. A cross encoder sees them together.
Bi encoder
query → vector
→ similarity
text → vector
Cross encoder
[query + document]
↓
joint model
↓
relevance scoreBecause the cross encoder can examine interactions between the query and every part of the candidate document, it can make a more precise relevance judgment.
For example:
Query: How many days can I work remotely?
Candidate A:
Employees may work from home up to three days each week with manager approval.
Candidate B:
Employees working remotely must use the corporate VPN.
Both are semantically related to remote work. But only Candidate A actually answers the question. A good reranker can move Candidate A above Candidate B.
Cross encoder and reranker are not the same thing
This distinction causes a lot of confusion.
Reranker describes a job in the pipeline.
Cross encoder describes a model architecture.
A cross encoder can be used as a reranker, but reranking can also be implemented with other models or scoring methods.
11. The complete modern retrieval pipeline
Now we can put the pieces together.
User query
↓
├───────────────────────┐
↓ ↓
Sparse retrieval Dense retrieval
BM25 Bi encoder
↓ ↓
keyword ranking semantic ranking
└───────────┬───────────┘
↓
RRF
↓
top candidates
↓
reranker
cross encoder
↓
best passages
↓
LLM
↓
answerThis is the architectural idea worth remembering:
Use inexpensive methods across a large search space, then spend more computation as the candidate set becomes smaller.
12. Think of retrieval as a funnel
1,000,000 documents
↓
BM25 + vector search
↓
100 candidates
↓
RRF / fusion
↓
50 candidates
↓
cross encoder reranking
↓
5 to 10 passages
↓
LLMThis explains why production AI search systems can combine multiple models without running the most expensive model against every document.
13. Where ColBERT and late interaction fit
There is another interesting family of retrieval approaches between simple bi encoder search and full cross encoder scoring.
One well known example is ColBERT.
A basic dense retriever often compresses an entire query or document into one vector. ColBERT keeps richer token level representations and performs a later interaction between query and document tokens.
Single vector retrieval
document → one vector
query → one vector
Late interaction
document → many token vectors
query → many token vectors
↓
interaction laterThe goal is to preserve more detailed matching information while still allowing document representations to be indexed ahead of time.
You do not need ColBERT for every application. It is useful to know because retrieval architecture is not limited to a simple choice between bi encoders and cross encoders.
14. A practical example: searching technical documentation
Suppose an engineer asks:
Why are users logged out after 30 minutes of inactivity?
The documentation contains:
Document A
"Session timeout is configured to 30 minutes."
Document B
"Authentication cookies are encrypted."
Document C
"Idle browser sessions are terminated according to the session timeout policy."
Document D
"ERR-301 indicates an invalid access token."A possible search flow is:
Sparse retrieval finds Document A because 30 minutes and session timeout are strong lexical signals.
Dense retrieval may find Documents A and C because logged out after inactivity is semantically close to idle sessions are terminated.
RRF combines those rankings.
The reranker recognizes that Document A directly explains the 30 minute behavior and places it first.
The LLM then receives the strongest passages and can generate a grounded answer.
15. What should you choose?
| Need | Good starting point |
|---|---|
| Exact IDs, codes, names, or phrases | Sparse retrieval / BM25 |
| Conceptual or meaning based search | Dense retrieval |
| General knowledge or enterprise search | Hybrid retrieval |
| Combine multiple ranked lists | RRF |
| Improve precision on top candidates | Reranker |
| High precision reranking | Cross encoder |
| Richer retrieval interaction at scale | Late interaction such as ColBERT |
| Ground an LLM in retrieved information | Retrieval pipeline + RAG |
16. Common mistakes
Replacing keyword search with vector search everywhere
Exact words still matter. Codes, names, numbers, product identifiers, and quoted phrases are often handled extremely well by lexical search.
Sending too many retrieved passages to the LLM
More context is not automatically better. Irrelevant passages can distract the model and consume context space.
Treating similarity as proof of relevance
A passage can be semantically similar to a question without answering it. That is one reason reranking helps.
Evaluating only the final LLM answer
If the wrong evidence was retrieved, the generation model starts from a weak foundation. Retrieval should be measured independently.
Ignoring filters and permissions
Enterprise AI search often needs metadata filters, tenant boundaries, authorization, freshness, and document level permissions in addition to relevance ranking.
17. Measure retrieval before blaming the LLM
When a RAG system gives a poor answer, people often change the prompt first. Sometimes the real problem happened much earlier.
Question
↓
Did we retrieve the right evidence?
↓
Did we rank it near the top?
↓
Did the LLM receive it?
↓
Did the LLM use it correctly?This separation makes debugging much easier.
Useful retrieval evaluation measures include whether the correct document appears in the top results, how highly it ranks, and whether improvements work consistently across a representative test set.
The mental model to keep
You do not need to memorize every retrieval algorithm. Remember the roles:
Keywords find exact signals.
Embeddings find semantic signals.
Hybrid search uses both.
RRF combines rankings.
Rerankers improve precision.
The LLM receives the final evidence.Modern AI search is therefore not simply "vector search." It is usually a sequence of increasingly precise decisions about which information deserves to reach the model.
Where this fits with RAG
RAG describes the larger pattern of retrieving external information and supplying it to a generative model. The techniques in this article explain what can happen inside the retrieval part of that architecture.
RAG
question
↓
retrieval
├── sparse search
├── dense search
├── hybrid fusion
└── reranking
↓
context
↓
LLM
↓
answerIf you already understand the basic idea of RAG, this is the next layer down: how the system actually decides which information is relevant enough to become context.