Context Engineering: The Skill That Replaced Prompt Engineering

Context is the new code

Prompt engineering is dead. Not in the "newspapers declared it dead" sense, but in the way assembly language is dead. You can still write it. Most professionals have moved on to something higher-level. That something is context engineering: the discipline of structuring, selecting, and sequencing information so that AI agents produce reliable outputs in production systems. At product.engineer, we define context engineering as the practice of designing the complete information environment that an AI model operates within, including system instructions, retrieved documents, tool definitions, conversation history, and structured metadata. It determines not just what the model knows, but how it reasons, what it prioritizes, and where it directs its attention. If prompt engineering was about writing a clever question, context engineering is about building the entire room the model thinks inside.

This distinction matters deeply for the product engineer. When you own the full cycle from problem definition to shipped feature, you need AI that works reliably at production scale. Not AI that works impressively in a demo and then falls apart on edge cases. That reliability comes from context, not from prompts.

The shift became visible in 2025. At the AI Engineer World's Fair, Shopify's engineering team presented their internal framework for what they called "context-first agent design." The talk has since accumulated over 73,000 views. Their core argument: the quality of an agent's output is 90% determined by the context it receives and 10% determined by the model's raw capability. They showed production data from their merchant-facing AI tools. When they improved prompts without changing context architecture, accuracy moved from 71% to 74%. When they restructured the context without changing the prompt, accuracy jumped from 71% to 93%.

That 22-point gap is the entire case for context engineering in one data point.

Why prompt engineering hit a ceiling

Prompt engineering worked when the use case was simple: human types a question, model returns an answer. You could iterate on phrasing, add few-shot examples, specify output format, and get meaningfully better results. It was a valid discipline for a specific era.

That era ended when we started building agents.

An agent is not a single prompt-response pair. It is a system that takes actions over time, maintains state across interactions, calls tools, reads documents, and makes decisions with consequences. When your AI feature books a restaurant, modifies a database, or commits code, the stakes of a wrong answer go from "mildly annoying" to "production incident."

The problem with prompt engineering for agents is structural. A prompt is a single string. An agent's context is a dynamic, multi-source system that changes on every turn. Consider what a coding agent like Cursor or Windsurf needs to work correctly:

• The user's current instruction
• The file currently open
• Related files in the project
• The project's coding conventions (from CLAUDE.md or similar)
• Recent git history for change context
• Tool definitions for file operations, terminal commands, web search
• Conversation history (compressed or full)
• Error messages from previous attempts
• Test results from the last run

That is nine different information sources, each with different update frequencies, different relevance windows, and different priority levels. "Writing a better prompt" addresses exactly one of those nine dimensions. Context engineering addresses all of them simultaneously.

Research from Anthropic's evaluations team demonstrates that structured context (system prompts with explicit sections, tool descriptions with examples, and retrieved documents with relevance scores) significantly improves task completion rates compared to unstructured context with the same information content. Same information. Different structure. Meaningfully better outcomes.

That is not a prompting trick. That is architecture.

The anatomy of a context system

A well-engineered context has five layers. Each layer serves a different function, and the product engineer who understands all five ships AI features that actually work in production.

Layer 1: Identity and constraints

This is the system prompt, but thinking of it as "a system prompt" undersells what it does. It establishes the model's operating parameters: who it is, what it can and cannot do, how it should handle ambiguity, and what its failure modes should look like.

Linear's AI features use identity layers that specify not just tone ("concise, technical") but decision boundaries ("if the user's request would affect more than 50 issues, confirm before proceeding"). Stripe's docs assistant has constraint layers that prevent it from generating API calls with invalid parameter combinations, even if the user explicitly requests them.

The identity layer answers: what are the boundaries of this agent's world?

Layer 2: Knowledge (retrieved and injected)

This is where RAG (retrieval-augmented generation) lives, but it is broader than RAG. Knowledge context includes:

• Documents retrieved by semantic search
• Structured data pulled from databases
• Real-time information from API calls
• User-specific data (preferences, history, permissions)

The engineering challenge here is not retrieval. It is selection and compression. A model with 128K context can hold a lot of information. But research from Google DeepMind (published in their "Lost in the Middle" paper, 2023) demonstrated that information placed in the middle of long contexts is recalled 20-30% less accurately than information at the beginning or end. Position matters. Order matters. Context engineering means deciding not just what to include, but where to place it.

Layer 3: Tools and capabilities

Tool definitions are context. When you describe a function's parameters, its expected behavior, its failure modes, and its side effects, you are engineering the model's understanding of what it can do. Poor tool descriptions produce models that call the wrong tool, pass wrong parameters, or fail to use available tools at all.

Vercel's v0 product demonstrates this well. Their tool definitions include not just the function signature but explicit examples of when to use each tool, common mistakes to avoid, and the expected output format. This is harness engineering in practice: building the infrastructure that makes agent behavior predictable.

Layer 4: Memory and conversation state

What happened before this turn? What did the user say three messages ago? What did the agent try that failed? Memory management is a context engineering problem because models have finite windows and real conversations have infinite potential length.

The naive approach is to include the full conversation history until you hit the context limit, then truncate from the beginning. The engineered approach is to maintain a compressed summary of earlier interactions, preserve critical decision points verbatim, and discard low-information turns entirely.

Notion's AI features implement what they call "progressive summarization" for context. Recent messages are included verbatim. Messages from 10+ turns ago are summarized into key facts. Messages from 50+ turns ago are reduced to entity relationships only (user prefers X, project requires Y). This keeps the context window efficient while preserving the information the model actually needs.

Layer 5: Output scaffolding

The final layer tells the model how to structure its output. This is not "respond in JSON." It is providing the shape of the expected response, including intermediate reasoning steps, required validations before final output, and format constraints that downstream systems depend on.

When Figma's AI features generate design suggestions, the output scaffolding includes required metadata (component type, variant compatibility, accessibility notes) that the rendering system needs. The model does not get to decide what metadata to include. The context engineering defines it.

The context engineering framework

After working with AI systems at scale, both at AWS and in my own companies, I have found that effective context engineering involves five key dimensions: Priority, Relevance, Instruction, Structure, and Measurement.

Priority determines what information goes where in the context window. Critical constraints go first (system prompt). Tool definitions go near the user's query. Retrieved documents go between, ordered by relevance score.

Relevance is the filter that decides what enters the context at all. Not everything the model could know should be in the context. A coding agent working on a React component does not need the project's database schema unless the component fetches data. Relevance filtering is the difference between a 4K-token focused context and a 90K-token noisy context. The focused context wins almost every time.

Instruction is the constraint layer. Explicit rules the model must follow, phrased as imperatives. Not "you might want to consider" but "you must validate inputs before calling external APIs." Anthropic's own research showed that imperative phrasing in system prompts increased instruction adherence by 18% compared to suggestive phrasing.

Structure means using consistent formatting. Headers, bullet points, XML tags, JSON schemas. Models parse structured text more reliably than prose. Anthropic recommends XML tags for section delineation. OpenAI's documentation recommends markdown headers. Both work. Consistency is what matters.

Measurement is the feedback loop. You instrument your context to understand what the model actually uses. If you include 50 retrieved documents and the model only references 3, your relevance filtering needs work. If the model violates a constraint, your instruction layer needs strengthening.

Context engineering in the product engineer's workflow

This is not an academic exercise. For anyone building AI features, context engineering is daily work. It shows up in three places.

Designing agent features

When you spec an AI feature (following spec-driven development), the context architecture is part of the spec. You define what information sources the agent needs, how they get retrieved, and what priority they take. This is as much a part of the feature design as the UI mockup.

At AWS, I have seen teams lose months because they designed the UX for an AI feature without designing the context. They built a beautiful chat interface. The responses were terrible because nobody engineered what the model actually received. The fix was always the same: go back, design the context system, ship again.

Configuring development tools

Every engineer using Cursor, Claude Code, Windsurf, or GitHub Copilot is doing context engineering whether they realize it or not. Your CLAUDE.md file is context engineering. Your .cursorrules file is context engineering. The way you structure your codebase so that an AI can navigate it is context engineering.

This connects directly to agentic engineering. The engineers who get 10x productivity gains from AI tools are not writing better prompts. They are engineering better context: clearer file structures, better naming conventions, explicit documentation of decisions and constraints.

Building production AI systems

When you ship an AI feature to users, the context system is production infrastructure. It needs monitoring, versioning, A/B testing, and incident response, just like any other production system.

PostHog treats their AI feature contexts as code. They version them in git, review them in PRs, and measure their impact with the same rigor they apply to any product change. When a context change produces a regression in output quality, they revert it like they would revert a bad deploy.

The product engineer's context engineering toolkit

Here is what the toolkit looks like in practice:

Real-world patterns from production

Let me share three patterns I have seen work repeatedly across teams.

Pattern 1: The context contract. Before building an AI feature, write a document that specifies exactly what the model receives in every scenario. Treat it like an API contract. When the product changes, update the context contract first, then update the implementation. This prevents drift between what you think the model sees and what it actually sees.

Pattern 2: The context budget. Assign token budgets to each layer. Example: 2,000 tokens for system prompt, 4,000 for tools, 8,000 for retrieved documents, 2,000 for conversation history. When new information needs to enter, something else has to leave. This forces prioritization decisions and prevents context bloat.

Pattern 3: The context regression test. Maintain a suite of input scenarios with expected outputs. When you change the context architecture, run the suite. This is not an eval in the ML sense; it is a functional test that ensures your context changes do not break existing behavior. Stripe runs over 2,000 context regression tests on their AI features before any deploy.

From my own experience

Having coached over 12,000 engineers and hired more than 600, I can tell you the pattern I see over and over: the engineers who struggle with AI features focus on the model. The engineers who ship reliable AI features focus on the context. This is true whether they are building internal tools at AWS or consumer products at a startup.

In my own work as a Sr. Product Engineer at AWS, the shift was dramatic. Early AI features I worked on spent 80% of development time on prompt iteration. We would test hundreds of prompt variations, find one that worked well in our test cases, deploy it, and then watch it fail on real-world inputs we had not anticipated. The problem was never the model. It was always the context: missing information, poorly structured instructions, or tool definitions that did not account for edge cases.

When we shifted to context-first development, our iteration cycles dropped from weeks to days. We spent less time tweaking language and more time engineering what information the model received, how it was structured, and when it was updated. The quality improvements were consistent and predictable in a way that prompt tweaking never was.

The shift in hiring

This has implications for how companies hire product engineers. Prompt engineering was never a real engineering discipline. It was a skill, like "writing good Google search queries." Useful, but not sufficient for building production systems.

Context engineering is different. It requires systems thinking, information architecture, testing methodology, and production engineering skills. It is why companies like Vercel, Linear, and Notion increasingly look for engineers who understand both product and AI systems; who can design a context architecture with the same rigor they bring to a database schema or an API design.

If you are a product engineer looking to build AI features, invest in context engineering. Not prompt templates. Not "magic prompts" from Twitter threads. The fundamentals: how models process context, how information priority affects output quality, how to test and measure context effectiveness.

Common mistakes

Five mistakes I see teams make repeatedly:

1. Stuffing everything in. More context is not better context. A 100K-token context with 80K tokens of marginally relevant documents performs worse than a 20K-token context with precisely relevant information. Models get confused by noise.

2. Ignoring position effects. Critical instructions buried in the middle of a long context get followed less often. Put your most important constraints first and last.

3. Static context for dynamic tasks. If your agent handles different types of requests, the context should adapt. A customer support agent answering billing questions needs different context than the same agent answering technical questions. One-size-fits-all context produces mediocre results for all cases.

4. No measurement. If you do not instrument what the model actually attends to and uses, you are guessing. Add logging that tracks which retrieved documents get referenced in outputs, which tools get called, and which instructions get followed.

5. Treating context as a prompt. Context is infrastructure. It needs versioning, testing, monitoring, rollback capability, and ownership. If your context lives in a string literal inside a function and nobody reviews changes to it, you will have production incidents.

Key takeaways

• Context engineering structures the entire information environment an AI operates within, not just the prompt text.
• Restructuring context without changing prompts jumped accuracy from 71% to 93% in Shopify's production AI tools.
• A well-engineered context has five layers: identity, knowledge, tools, memory, and output scaffolding.
• Treat context as production infrastructure that needs versioning, testing, monitoring, and rollback capability.
• The engineers shipping reliable AI features focus on context architecture, not prompt iteration.

FAQ

What is the difference between context engineering and prompt engineering?

Prompt engineering focuses on crafting the text of a single query to get better responses from a model. Context engineering is broader: it encompasses the entire information environment the model operates within, including system instructions, tool definitions, retrieved documents, conversation history, and output structure. Prompt engineering is one input. Context engineering is the whole system.

Do I need to know context engineering if I am not building AI products?

Yes, if you use AI development tools. Every engineer using Cursor, Claude Code, GitHub Copilot, or similar tools benefits from context engineering. How you structure your codebase, write documentation, and configure tool settings directly affects the quality of AI assistance you receive. The engineers getting 10x productivity gains have engineered their context, not just their prompts.

How do I measure whether my context engineering is working?

Three metrics matter: task completion rate (does the agent finish what it starts?), instruction adherence (does it follow the constraints you set?), and relevance utilization (does it use the information you provide?). Track these across context changes to understand what improvements actually move the needle.

What tools should I use for context engineering?

The tools are less important than the practice. That said, version control (git) for context definitions, evaluation frameworks (like Anthropic's or OpenAI's eval tools) for measuring quality, and observability platforms (like LangSmith or Helicone) for production monitoring form a solid foundation. The key is treating context as code that gets reviewed, tested, and monitored.

Is context engineering only relevant for LLM-based systems?

The principles apply anywhere an AI system needs structured input to produce reliable output. But the specific techniques (token budgeting, position optimization, progressive summarization) are most directly applicable to transformer-based language models with finite context windows. As context windows grow and architectures evolve, the specifics will change. The discipline of designing information environments will not.

The future is context-shaped

Prompt engineering was a single-player game. Write a better prompt, get a better answer. Context engineering is a team sport. It touches product design (what information does the user need to provide?), backend engineering (how do we retrieve and structure knowledge?), data engineering (how do we keep context fresh?), and frontend engineering (how do we present multi-step agent interactions?).

This is why the product engineer is uniquely suited to own it. Not the ML engineer who optimizes models. Not the data scientist who builds evals. The person who understands the full system, from user intent to shipped feature, and can make the architectural decisions that determine whether an AI feature works in production or falls apart on the first edge case.

Context is the new code. Engineer it accordingly.

Related reading

• What Is a Product Engineer? The Definitive Guide
• Harness Engineering: Building Infrastructure for AI Agents
• Agentic Engineering: How Product Engineers Build with AI
• Spec-Driven Development for Product Engineers
• Don't Build Agents, Build Skills