All posts

AI engineering · AI agents · Guides

Context engineering: deciding what the model sees

Context engineering is the discipline of curating an AI model’s working memory - writing, selecting, compressing, and isolating what goes into the context window.It’s what prompt engineering grew up into, it’s where most real AI application quality comes from, and it’s learnable. The complete guide, with the tactics that matter.

Deric YeeDeric Yee Updated 24 August 2026 10 min read
Analytics dashboards glowing on dark screens

Why “prompt engineering” stopped being the job

For a couple of years, getting good output from AI meant writing better instructions - and “prompt engineering” named a real skill. Then people started building actual applications and agents, and discovered the instruction was the smallest part of the problem. A production AI request isn’t one clever sentence: it’s a system prompt, conversation history, retrieved documents, tool descriptions, tool results, and memory - thousands to hundreds of thousands of tokens, of which the user’s prompt is a sliver. Quality turned out to be governed by everything else in the window, and the discipline of managing that became context engineering.

The mental model that makes everything click: the context window is a desk, not a warehouse. Million-token windows mean the desk got huge - but attention is still finite, and everything you put on the desk competes for it. Models attend unevenly across long contexts; a small, curated context reliably beats an enormous unfiltered one on the same task. So the engineering question is never “how much can I fit?” It’s the librarian’s question: what does the model need on the desk right now, and what is merely nearby?

This guide slots into our engineering series as the layer the others stand on: the agent is a model, tools, and a loop; loop engineering governs when it runs; graph engineering wires many of them together. Context engineering governs what any of them is thinking about on any given turn - which is why it’s where debugging almost always ends up.

The four operations

Every context-engineering tactic in the wild is one of these four moves.

Write - put the right things in

The system prompt, the task spec, the examples, the tool descriptions. Everything the model needs to act well, stated once, precisely, in stable order. Most “model is being dumb” complaints trace back to context that never contained what the model needed - or buried it under what it didn’t.

Select - retrieve only what’s relevant

The model shouldn’t see your whole knowledge base; it should see the five chunks that matter for this request. Retrieval (search, RAG, file reads, memory lookups) is selection - and over-retrieving is as damaging as under-retrieving, because every irrelevant token competes for the model’s attention.

Compress - shrink what must stay

Long histories get summarised, verbose tool results get truncated to what mattered, old conversation turns collapse into notes. Compression trades detail for headroom - done well, the model keeps the plot without drowning in the transcript of getting there.

Isolate - keep contexts from polluting each other

Separate concerns get separate contexts: a sub-agent gets a fresh window with only its task, a verifier never sees the author’s reasoning, a new phase starts clean with a written handover. Isolation is the operation people discover last - and the one that fixes the weirdest failures.

Context rot: the failure mode that defines the field

Run any agent long enough and you’ll watch it happen. Around some point in a long task, the model starts repeating steps it already completed. It forgets a constraint you stated an hour ago. It fixates on a plan that was superseded twenty turns back. Nothing “broke” - the window simply filled with the sediment of getting here: stale tool results, dead ends, old drafts - and the signal drowned in its own history. That’s context rot, and it’s the reason context engineering exists as a named discipline rather than a nice-to-have.

The rot has a subtler cousin worth knowing: poisoning.One wrong fact that enters the context early - a hallucinated detail, a misread file, a bad assumption - gets treated as established truth by every subsequent turn, because models trust their context more than they doubt it. The fix for both is the same pair of operations: compress (summarise history into verified conclusions, clear stale results) and isolate (start important phases with a clean window and a deliberate briefing, so sediment and poison don’t carry forward).

The tactics, in code

Here are the three highest-value tactics for anyone building on the raw APIs - trimming tool results before they enter history, compacting on a token trigger, and delegating reads to isolated sub-agents:

context-hygiene.ts - compress and isolate
// Context hygiene for a long-running agent - the three highest-value
// tactics, in ~40 lines. (Both major SDKs also ship server-side versions:
// automatic compaction and tool-result clearing.)

const MAX_TOOL_RESULT = 4_000 // chars kept per tool result

// 1. COMPRESS tool results before they enter history. A 60KB API response
//    usually contains one paragraph the model needs.
function trimResult(raw: string): string {
  if (raw.length <= MAX_TOOL_RESULT) return raw
  return raw.slice(0, MAX_TOOL_RESULT) + '\n[...truncated - ask to re-read if needed]'
}

// 2. COMPRESS history on a trigger: when the transcript passes a budget,
//    summarise everything but the recent turns into a briefing note.
async function compactIfNeeded(messages: Msg[]): Promise<Msg[]> {
  if (tokenEstimate(messages) < 60_000) return messages
  const old = messages.slice(0, -6)              // keep the live tail verbatim
  const summary = await summarise(old)           // one model call:
  // "Facts established, decisions made, work completed, open items."
  return [
    { role: 'user', content: `[Context summary of earlier work]\n${summary}` },
    ...messages.slice(-6),
  ]
}

// 3. ISOLATE sub-tasks: a fresh context that receives a briefing,
//    not the transcript. The reader sub-agent returns facts; the
//    parent's window never absorbs the 200KB it read to find them.
async function delegateRead(files: string[], question: string) {
  return runAgent({
    system: 'You read files and answer precisely. Return facts only.',
    task: `Read ${files.join(', ')} and answer: ${question}`,
  }) // parent gets the answer - never the file contents
}

Two more tactics that cost nothing and pay constantly. Order for stability: put stable content first (system prompt, tool definitions) and volatile content last (the current question, timestamps). Models weight beginnings heavily, and this ordering is also exactly what makes prompt caching work - good context engineering and cheap API bills are the same discipline. And externalise memory:durable facts (user preferences, project decisions, what’s been tried) belong in a file or store the agent reads on demand - not in a transcript that will eventually be compressed away. The window is working memory; give long-term memory its own home.

The meta-tactic above all of them: look at your actual context.Log the final assembled prompt your system sends. Nearly every “the model is being stupid” bug becomes obvious within a minute of reading what the model actually saw - the missing constraint, the twelve stale tool results, the two contradictory instructions from different code paths. It’s the AI era’s version of reading the logs, and it’s shocking how few people do it.

Why this is a career skill, not a trick

Notice what context engineering actually is, underneath the vocabulary: deciding what information matters, structuring it so the reader can act on it, cutting what doesn’t earn its place, and briefing collaborators cleanly. Those are senior- engineer instincts - communication and judgement - applied to a machine reader. That’s why experienced professionals pick this up fast, and why it’s become a screening skill in AI-era hiring: an engineer who manages context well ships AI features that work; one who doesn’t ships demos that fall apart on real data.

It’s also thoroughly learnable by building. In the programme, students hit context limits with their own AI product feature in Phase 3 - and learn these operations by needing them, which is the only way they stick. If you’re earlier in the journey, start free: the free trial gets you building real projects this week, no card needed.

FAQ

  • What is context engineering?

    Context engineering is the discipline of deciding what an AI model sees: deliberately writing, selecting, compressing, and isolating the information in its context window - the system prompt, conversation history, retrieved documents, tool results, and memory. Where prompt engineering crafts one instruction, context engineering manages the model’s entire working memory across a task. The term took over from “prompt engineering” as the field’s centre of gravity because in real applications and agents, what fills the window matters far more than how any single request is phrased.

  • What is the difference between context engineering and prompt engineering?

    Prompt engineering optimises an instruction: phrasing, structure, examples. Context engineering optimises everything around it: which documents get retrieved, how much history survives, what order information appears in, what gets summarised away, and what each sub-agent is allowed to see. Prompt engineering is writing a good question; context engineering is deciding what’s on the desk when the question is asked. In agents and production systems, the second dominates - a perfect prompt inside a polluted context still fails.

  • What is context rot?

    Context rot is the degradation of model performance as the context window fills with stale, redundant, or irrelevant content - old tool results, superseded plans, dead conversation branches. Symptoms: the model repeats completed steps, forgets constraints stated early on, or fixates on outdated information. Bigger context windows didn’t cure it, because attention is finite even when capacity isn’t: everything in the window competes for relevance. The cures are the compression and isolation operations - summarise, clear stale tool results, and split long jobs into fresh-context phases.

  • Why does context engineering matter more than bigger context windows?

    Because capacity and attention are different resources. Million-token windows mean you can put more in - not that the model weighs it all well. Research and production experience both show models attend unevenly across long contexts, and that a small, curated context reliably beats a huge, unfiltered one on the same task. The window is a desk, not a warehouse: a bigger desk helps, but the skill that determines output quality is deciding what’s on it.

  • How do I get better at context engineering?

    Practise on real systems and inspect ruthlessly. Concretely: log the exact final context your app or agent sends (most bugs are visible right there), apply the four operations deliberately (write precisely, select narrowly, compress on a trigger, isolate sub-tasks), keep stable content first and volatile content last (which also makes caching work), and when output quality drops mid-task, check the window before blaming the model. Building a real AI-powered product end to end - the fastest way to learn all of this - is exactly what Phase 3 of our programme has students do.

Back to all postsNext in the series: loop engineering →

The window is a desk. Learn to keep it.
Build the judgement by building.

Context engineering sticks when you need it for something real. In the AI-Native Software Development Programme you ship an actual AI product feature - and hit, then solve, every problem in this guide. Start free, no card.

Start the free trialSee the programme