RAG stands for retrieval-augmented generation, and in practice it means one thing: AI that answers from your documents instead of from whatever it absorbed during training — and shows you which document each part of the answer came from.
The acronym is unhelpful and you can forget it. What follows is the idea, without the vocabulary.
Why a general assistant fails on your own material
Ask a general-purpose assistant what the notice period is in your supplier contract and it will tell you. Confidently. It will be wrong, because it has never seen the contract.
This is not a defect so much as a category error. A general assistant learned patterns from an enormous amount of public text. It knows what supplier contracts usually say. It has no access whatsoever to what your supplier contract says, and nothing in how it works signals the difference between recalling something and constructing something plausible.
For an organisation, that gap is the whole problem. The questions worth asking an AI at work are nearly all specific: what did we agree with this customer, what does our own policy say, what did we decide in March and why.
The order of operations
Retrieval changes the sequence.
Before answering, the system searches your own material, finds the passages that bear on the question, and writes an answer from those passages. Then it tells you which document each part came from.
That is it. There is no additional magic, and the simplicity is worth dwelling on, because it explains both the strengths and the limits.
Why the citation is the point
Consider what happens after the answer appears.
Someone in your finance team asks what the notice period is in a supplier contract. They get a sentence back. The next thirty seconds decide whether the tool was worth buying.
If the answer carries a link to clause 14.2 of the actual contract, they open it, read it, and either act or escalate. The system saved them twenty minutes of searching and they verified it in half a minute.
If the answer carries nothing, they have two options, and both are bad. Verify from scratch — in which case the tool saved nothing. Or don’t verify, and act on a sentence nobody checked.
That is not a preference about user interfaces. It is the difference between a system that survives an audit and one that quietly manufactures liability. The mechanics of how the tracing works are in how citation-backed RAG works.
What it does not do
Every one of these gets oversold somewhere, so they are worth stating plainly.
- It does not make the model smarter. Retrieval gives the model better material. A weak model with excellent retrieval gives you accurate but clumsy answers; a strong model with poor retrieval gives you eloquent, well-cited nonsense.
- It does not fix badly organised documents. If three versions of a policy sit in the folder, the system may cite any of them. Retrieval makes your content problems visible and specific — which is genuinely useful, but it is not the same as solving them.
- It does not remove the need for an owner. Someone has to decide what gets indexed, what gets removed when it is superseded, and who is allowed to see which material. Without that person the quality curve bends downward from month two.
- It does not decide access for you. The moment a system can read a folder, “who can see what” becomes a live question with a real answer. Better to decide it deliberately than discover it in an audit.
The part with the technical word in it
You can stop here and lose nothing. But one term comes up in every vendor conversation, so here it is.
To search meaningfully, the system needs to find passages that answer the question rather than merely repeat its words. Someone asking “how long is the notice period” should find a clause that says “termination requires ninety days’ written notice,” even though the two share almost no vocabulary.
Systems do this by converting text into embeddings — long lists of numbers positioned so that passages about similar things sit near each other. Searching is then a matter of finding the nearest neighbours. That is the whole concept, and the useful consequence for a buyer is that retrieval quality is a real engineering variable rather than a given: two systems over the same documents can return meaningfully different passages, which is why evaluating on your content matters more than any benchmark. That is what the quality loop is about.
What you need to try it
Three things, and the third is the one projects skip
- Documents people actually ask about. Start with the three sources that generate the most internal questions. Indexing everything on day one is the most common way to guarantee a mediocre pilot.
- Somewhere to run it. For public or low-sensitivity content this can be a hosted service. For contracts, personnel files or client data it usually means your own infrastructure — see sovereign AI in plain English.
- A named owner for the content. Not a project sponsor — someone who decides what goes in, what comes out when it is superseded, and who can see what.
If you want the next level of detail, the RAGSuite triad explains how search, an assistant and the connectors that bring your material in fit together, and choosing an LLM for regulated RAG covers the model decision for confidential content.
Frequently asked questions
What does RAG stand for, and do I need to remember it?
Retrieval-augmented generation. You do not need to remember it. The useful sentence is “AI that answers from our own documents and shows its sources” — that phrase will get you through any vendor conversation, and most people in the room will be quietly grateful.
How is this different from asking a general AI assistant?
A general assistant answers from what it absorbed during training. It has never seen your contracts, your handbooks or your project history, so when you ask something specific to your organisation it produces something fluent and frequently wrong. A retrieval system searches your own material first and writes the answer from the passages it found, which is why it can point at where each claim came from.
Does it stop the AI making things up?
It reduces it substantially and does not eliminate it. Grounding an answer in retrieved passages removes the most common failure — inventing facts wholesale — but a system can still misread a passage, retrieve the wrong document, or blend two sources badly. This is precisely why citations matter: they turn an invisible error into a checkable one.
What if our documents are a mess?
Then you will get a fast, confident tour of the mess. Retrieval reflects what it is given: superseded policies still in the folder, two versions of the same document, an approval that lives only in an email thread. Most disappointing pilots are content problems wearing a technology costume. The upside is that the failures are visible and specific, which makes them fixable.
What do we actually need to start?
Three things. Documents people genuinely ask questions about — start with the three sources that generate the most internal questions, not everything. Somewhere to run it, which for confidential material usually means your own infrastructure. And a named person who owns the content: who decides what goes in, what comes out, and who can see what. The third is the one that decides whether a pilot survives.
Sources & further reading
- Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020) — the original paper that named the approach