The rise of "context engineering"
Blog

The rise of "context engineering"

Harrison Chase
2025.07.06
·Web·by Anonymous
#LLM#Context Engineering#Agent#Prompt Engineering#LangGraph

Key Points

  • 1Context engineering is defined as building dynamic systems to provide large language models with the right information, tools, and format to plausibly accomplish tasks.
  • 2This discipline is crucial because many LLM agent failures stem from insufficient or poorly formatted context, not just the model's limitations, making it a broader concept than prompt engineering.
  • 3Tools like LangGraph enable comprehensive context engineering by offering granular control over LLM inputs and processes, while LangSmith provides observability to debug and refine the context provided to agents.

Context engineering is defined as the process of building dynamic systems to provide the right information and tools in the right format, enabling a Large Language Model (LLM) to plausibly accomplish a given task. This approach extends beyond static prompt engineering, addressing the complexities of evolving LLM applications into dynamic, agentic systems.

The methodology of context engineering is characterized by several core components:

  1. Systemic Nature: Context for complex agents is sourced from various origins including the developer, user input, prior interactions, outputs from tool calls, and external datasets. Integrating these diverse information streams necessitates a sophisticated system.
  2. Dynamism: Many contextual elements are inherently dynamic, requiring the logic for constructing the final prompt to be equally dynamic, adapting in real-time rather than relying on fixed, static prompt templates.
  3. Provision of Right Information: A fundamental principle is that LLMs are not mind-readers; they require explicit, relevant information. Failure to provide appropriate context ("garbage in, garbage out") is a primary cause of poor agent performance.
  4. Provision of Right Tools: For tasks that cannot be solved purely by input data, LLMs must be equipped with suitable tools (e.g., for information retrieval, external actions). The availability of these tools is as critical as the information itself.
  5. Format Optimization: The manner in which information and tool parameters are presented to the LLM significantly impacts its comprehension and utility. Clear, concise, and digestible formats are preferred over verbose or poorly structured data (e.g., a descriptive error message over a large JSON blob).
  6. Plausibility Assessment: A critical self-evaluation question in context engineering is "Can the LLM plausibly accomplish the task with the given context?" This helps identify whether failures stem from inadequate context/tools or inherent model limitations, guiding debugging efforts.

Context engineering is paramount because most agentic system failures are attributed to insufficient or poorly presented context rather than the underlying LLM's inherent limitations, especially as models advance. Common issues include missing context (LLMs cannot infer what they are not given) and poorly formatted context (hindering effective communication).

The distinction from traditional prompt engineering lies in its scope: while prompt engineering focuses on crafting effective phrases, context engineering encompasses the entire process of structuring and delivering comprehensive, dynamic information. Prompt engineering, particularly the articulation of core instructions for LLM behavior, is considered a subset of context engineering.

Practical examples of context engineering include:

  • Tool Use: Ensuring agents have access to and can effectively interpret outputs from external tools, with outputs formatted for maximal LLM digestibility.
  • Short-Term Memory: Implementing mechanisms like conversation summarization to maintain context over ongoing interactions.
  • Long-Term Memory: Retrieving and incorporating user preferences or past data from previous sessions.
  • Retrieval-Augmented Generation (RAG): Dynamically fetching relevant external information and inserting it into the prompt before LLM invocation.

Frameworks like LangGraph facilitate context engineering by offering granular control over agent execution flow, precise input data passed to the LLM, and output storage, thereby enabling full customization of context construction. Observability platforms like LangSmith aid in debugging context engineering by providing detailed traces of agent calls, showing exact inputs/outputs to the LLM and the preceding steps for context gathering, thereby allowing engineers to verify the completeness and formatting of the provided context. The discipline aligns with the broader philosophy that effective communication with LLMs is foundational to reliable agentic systems.