A Field Guide to RAG
A field guide to the whole RAG landscape: naive RAG, the pre- and post-retrieval tricks, the modular mindset, and the five axes every system sets.
Retrieval-augmented generation started as a single idea and turned into a whole family of designs. This is the opening map: naive RAG, the advanced pre- and post-retrieval tricks, the modular mindset, and the five axes every variant moves along. It anchors a five-part series that walks every kind of RAG.
Think of an open-book exam. You do not memorize the whole textbook; you keep it beside you, and when a question comes you flip to the relevant pages and answer from those. Retrieval-augmented generation hands a language model that open book: rather than relying only on what it absorbed in training, it looks up the passages that matter and answers from them.
Retrieval-augmented generation, RAG, is how you give a language model knowledge it was never trained on: your documents, your database, this morning's data. Instead of hoping the answer is baked into the weights, you retrieve the relevant material at question time and put it in front of the model. Done well, it grounds answers in real sources, cuts hallucination, lets you cite where an answer came from, and updates the moment your data does, with no retraining.
That is the elevator pitch, and it has not changed since 2020. What has changed is that "RAG" is now an umbrella stretched over dozens of designs that look nothing like each other. Someone doing keyword search over ten PDFs and someone running a knowledge-graph-backed agent that reasons across sources over multiple hops will both tell you they "do RAG." They are both right, and that is exactly why the word has stopped being useful on its own.
This series is a map of the whole space. This first piece lays out the baseline and the axes; the four that follow go deep on each. My goal is that by the end you can look at any RAG system, or any paper with "RAG" in the title, and place it.
Naive RAG: the baseline everything else edits
Start with the simplest version, because every other kind of RAG is a modification of one box in this picture.
It happens in two phases. Ahead of time, you do the prep: you cut each document into small chunks, and you turn every chunk into an embedding. Think of an embedding as a pin on a map, placed so that chunks about the same topic sit close together. You save all the pins in an index. Then, when a question arrives, you put it on the same map, find the pins nearest to it, and take the closest few. Those chunks go into the prompt, and the model answers from them. Retrieve, then read.
Step through it once on a concrete question and the four stages become obvious:
It works surprisingly often, and it fails in surprisingly many ways. The chunks might be split badly, so the answer is cut in half. The nearest vectors might not be the most relevant ones, because "close in embedding space" is not the same as "useful." The right chunk might be retrieved but buried in the middle of a long prompt where the model glosses over it. And nothing about the pipeline actually guarantees the answer is grounded in what was retrieved; the model can still ignore the context and make something up. Every advanced technique in this series exists to patch one of those failures.
Advanced RAG: fix the query going in, fix the results coming out
The first wave of improvements keeps the pipeline shape but adds a step before retrieval, after it, or both. Each of these is a preview; the deep-dives that follow cover the machinery.
Before retrieval, fix the query
A user's question is often a bad search query, so the first family of tricks cleans it up before it ever hits the index. Three show up again and again.
Query rewriting. Picture walking up to a librarian and mumbling, "you know, that thing about taxes for freelancers." The librarian nods, quietly turns it into a precise request in their head, and only then heads to the shelves. Query rewriting is that clean-up step. Real questions are messy: full of filler, or leaning on earlier context, like a chat follow-up that just says "and how much does it cost?" On its own that is unsearchable, so the model rewrites it into a clear, standalone query ("how much does metformin cost?") before it ever hits the index.
On its own the follow-up is unsearchable: the index has no idea what “it” refers to.
Query expansion, or multi-query. One wording can quietly miss the right document, simply because the document happened to use different words. So instead of trusting a single phrasing, you ask the same thing several ways, the way a detective puts the same question to three witnesses in slightly different words because one version jogs a memory the others do not. A search for "car insurance claim" also goes out as "auto policy claim" and "vehicle accident reimbursement," so a passage that only ever says "automobile" is not missed. You run all the phrasings and pool everything they bring back.
You start with a single wording, which may not match documents that use different words.
HyDE (hypothetical document embeddings). Think of a police sketch. A witness describes a suspect from memory, and even if the sketch gets the nose wrong, it is close enough in overall shape to pick the right person out of a lineup. You do not arrest the sketch; you use it to find the real person. HyDE does this for search: instead of searching with the question, it has the model draft a rough, made-up answer and searches with that. The guess can have wrong details, but it is shaped like a real answer passage, so it lands right beside the true ones on the map, closer than the bare question ever would. You use only its position to search, then throw the guess away and read the real passages it found.
The question is worded nothing like the passages that answer it, so it sits far away on the map and pulls back weak matches.
After retrieval, fix the results
The first search is fast but rough, so the second family cleans up what came back before it reaches the model. There are two common moves.
Re-ranking. Think of a quick clerk who grabs a rough stack of files, then an expert who actually reads each one against your question and puts the best on top. That is re-ranking. The first search is quick but sloppy; the re-ranker is a slower, sharper model that reads each chunk next to the question and reorders them, floating the truly relevant ones up and pushing the near-misses down.
- 1Subscription pricing and plans0.88
- 2Cancel your subscription0.82
- 3Subscription terms of service0.79
- 4Pause your subscription temporarily0.60
- 5How to update your payment method0.55
- 6End your membership and stop billing0.41
Contextual compression. Now take a highlighter to each chunk and hand over only the highlighted lines, not the whole page. That is contextual compression. It trims each chunk down to the few sentences that actually answer the question, so the prompt fills with signal instead of filler and the model is not distracted by padding.
The retrieved chunk is on-topic but mostly padding for this exact question. Sending all of it wastes prompt budget.
None of this changes the basic loop. It just stops trusting the raw question and the raw search results.
Modular RAG: it is a design space, not an algorithm
Think of a sandwich shop rather than a single fixed meal. You do not order "the sandwich"; you choose the bread, then the filling, then the sauce. Once you have query rewriters, several retrievers, re-rankers, and compressors, RAG becomes exactly that: not one fixed pipeline but a set of swappable parts you pick from, one choice per step. This is the modular view, and it is the right mental model for everything that follows. There is no single "RAG." There is a menu, and a real system is one particular order off it.
The five axes
Picture a control panel with five dials. Every RAG system, however exotic, is just a particular setting of these five dials, and the rest of this series turns them one at a time, one or two per article.
- How it retrieves. Dense embeddings, sparse keyword search, a hybrid of both, re-ranking, late interaction. This is the matching problem, and it is the subject of How RAG Finds Things.
- How it indexes. What unit you chunk into, and what unit you hand back: fixed chunks, sentence windows, parent documents, summary trees like RAPTOR. Covered in How RAG Reads a Corpus.
- How its control flows. Retrieve once, or loop; decide whether to retrieve at all; grade and correct the results; hand the whole thing to an agent. That is RAG That Thinks.
- How its knowledge is structured. A flat pile of vectors, or a graph, or a mix of modalities. GraphRAG and multimodal RAG live in Beyond the Vector.
- Where it sits in the lineage. From the original RAG paper through RETRO and Fusion-in-Decoder to the modern "do you even need RAG with long context" debate and cache-augmented generation. Also in Beyond the Vector.
A production system that quietly does hybrid retrieval, re-ranks, uses parent-document chunking, retrieves once, over flat vectors, is one point in this space. An agentic GraphRAG system is another. Naming the axes is what lets you talk about the difference precisely instead of arguing about what "real RAG" is.
How to read the rest
The four deep-dives stand alone, so you can jump to whichever axis you are fighting with today. But they are ordered the way I would teach them: get retrieval and indexing right first, because a clever control loop over bad retrieval just fails more expensively. Then add intelligence to the loop. Then, if flat vectors genuinely cannot answer your questions, reach for graphs and the frontier.
The single most useful thing you can carry into all of them: RAG quality is mostly retrieval quality. The model is rarely the bottleneck. What you feed it is.
