Context engineering: why context matters more than the prompt
A language model only knows what is in its context for a given request. Context engineering makes sure the right things are there: the fitting instructions, the knowledge you need from your sources, a short history and results from tools. This guide explains how that works and why it matters more than perfect wording.
With a token budget you can recalculate.
Key points
Context engineering controls everything a model sees for a request. The prompt is only one part of it.- For company knowledge, the key tool is
RAG : for each question, matching passages from your documents are found and passed along with their sources. - More context is not automatically better.
Selected, ordered context usually gives more accurate answers than a full window.
What is context engineering?
Context engineering is the deliberate assembly of the context a language model processes for a request. It includes instructions, the actual question, retrieved knowledge, the conversation so far, results from tools and a memory of earlier steps. The goal is for the model to see exactly the information it needs for a correct answer.
The term spread mainly with AI agents that work over many steps. German Wikipedia offers an overview. The basic idea of loading knowledge for each request is described in the research paper on retrieval-augmented generation (Lewis et al. 2020).
Why it matters: many wrong answers come not from poor wording but from the model not seeing the right information. If the right passage is missing from the context, the model guesses. If it is there, with its source, the answer can be checked.
Context engineering vs prompt engineering.
The two are linked. Prompt engineering is one part of context engineering: it words the instruction. Context engineering decides what else goes along, and does so automatically for every request.
| Feature | Prompt engineering | Context engineering |
|---|---|---|
| Focus | the instruction | everything the model sees |
| Who does it? | whoever writes the instruction: user or admin | the system, for every request |
| Main means | role, task, format, examples | search, selection, compressing, order |
| Typical mistake | unclear task | wrong or too much knowledge in the context |
Prompt engineering: the eight techniques →Prompt generator →
Six techniques for better context.
These six techniques shape answer quality in RAG systems and AI agents. Most of them can be measured and tuned.
1 · Select
Only passages that fit the question. Hybrid search and reranking bring the best hits to the top.
2 · Compress
Summarise long history and tool results instead of sending them in full.
3 · Order
Put important things at the start or end. Models often use the middle of long contexts less well (Liu et al. 2023, “Lost in the Middle”).
4 · Separate
Mark instructions and data clearly. Text from documents and emails is content, not instructions.
5 · Budget
A fixed token budget per part. That leaves room for the answer and keeps the cost per request predictable.
6 · Cite
Pass every passage with its source and require the source in the answer. That way every statement can be checked.
How much context does a request need?
An example from a typical RAG system: the system prompt takes 500 tokens, five passages of 512 tokens come to 2,560, the history to 1,000, and 1,000 are reserved for the answer. Together that is 5,060 tokens. A context of 8,192 tokens per request therefore leaves room for longer questions.
Every extra token costs compute and, with cloud models, money. On your own server, memory per concurrent request also grows with the context. Use the context window calculator and the tokenizer to work out what fits your window.
| Part | Tokens | Lever |
|---|---|---|
| System prompt | 500 | keep it short and fixed |
| Retrieved knowledge | 2,560 (5 × 512) | number and size of passages, reranking |
| History | 1,000 | summarise older messages |
| Answer | 1,000 | set the length in the format |
| Total | 5,060 of 8,192 | buffer for long questions and tools |
Context engineering with RAGSuite.
RAGSuite handles the part of context engineering that concerns your knowledge: ingesting documents and splitting them into passages, finding the matching passages for each question and passing them to the model with sources. Through the MCP server, other AI tools and agents get this knowledge with sources too.
How the stages work together, from chunking to selecting the passages, is explained with starting values in the RAG pipeline guide.
For quality, measure instead of guessing. Set up real questions with expected answers, and after every change to chunking, search or instructions, check whether answer and source are right. Small changes to the context can matter more than switching models.
Is your context well built?
Tick what is done. Nothing is stored.
Questions about context engineering.
What is context engineering in simple terms?
Context engineering means deliberately controlling what a language model gets to see for a request: instructions, matching passages from your documents, the conversation so far and results from tools. The better this context is selected and ordered, the more accurate and verifiable the answer becomes.
What is the difference between context engineering and prompt engineering?
Prompt engineering words the instruction well. Context engineering makes sure everything needed is in the context, anew for every request: the right knowledge, the right amount, in the right order. Prompt engineering is therefore one part of context engineering, not a replacement for it.
What does context engineering have to do with RAG?
A great deal. Retrieval-augmented generation, RAG for short, is the main technique for bringing knowledge into the context: matching passages are found for each question and passed along. How well that works depends on chunking, search, reranking and the order in the context. That is exactly context engineering.
Will a larger context window make context engineering unnecessary?
Probably not. A larger window allows more text but costs more compute and money per request. Models also often use information in the middle of long contexts less well than at the start or end. Carefully selected context therefore usually remains better than as much context as possible.
Does context engineering matter for AI agents too?
Especially there. Over many steps, an agent collects tool results, intermediate states and history. Without compressing and ordering, the context quickly fills with irrelevant material and quality drops. Good agents summarise intermediate results and fetch knowledge with sources when they need it.
The right context for every answer.
Sanjay shows you how RAGSuite selects matching passages, ranks them by relevance and passes them on with sources.