AI & Agents

Context Engineering in OpenClaw: Design What Your Agent Sees

Chroma's research on context rot found that GPT-4 accuracy drops from 98.1% to 64.1% based purely on how information is structured, not on how much context is available. OpenClaw gives you direct control over this through seven workspace bootstrap files, configurable truncation limits, and a pluggable context engine system that lets you replace the entire prompt assembly pipeline.

Fast.io Editorial Team 14 min read
Neural network diagram representing AI agent context architecture and data flow

Why Context Structure Outweighs Context Size

Chroma's research on context rot found that GPT-4's task accuracy drops from 98.1% to 64.1% based solely on how information is structured within the context window, not on how much information is present. That 34-point gap means the difference between a useful agent and one that misses instructions, hallucinates capabilities, or forgets constraints mid-conversation. Context engineering exists because of this gap.

Most context window optimization guides focus on fitting more tokens. Context engineering focuses on choosing the right tokens and putting them in the right place. OpenClaw treats this as a first-class design problem rather than an afterthought.

Instead of dumping everything into a growing prompt, OpenClaw assembles a system prompt from discrete, inspectable components: tool descriptions, skill metadata, runtime information, and workspace bootstrap files that you control. Each piece has a clear role, a measurable size, and configurable limits.

The distinction between context and memory is central to OpenClaw's design. Memory persists on disk across sessions in the form of files, notes, and stored preferences. Context exists only within the current model window. Your AGENTS.md file is memory until OpenClaw injects it into the system prompt. At that point it becomes context. Understanding this boundary is the first step toward engineering it.

The practical payoff is predictability. When you know exactly what your agent sees, you can diagnose why it ignores certain instructions, why it invents capabilities it doesn't have, or why it loses track of constraints mid-conversation. You stop guessing and start designing.

What OpenClaw Sends to the Model

OpenClaw rebuilds the system prompt fresh on every run. The prompt comprises several distinct blocks, each serving a different purpose:

  • Tool list with descriptions: Every registered tool's name and a short description of what it does
  • Skills metadata: Name, description, and file location for each installed skill
  • Workspace location and time: Current directory, hostname, and timestamp
  • Runtime metadata: Host OS, active model, thinking mode, and version info
  • Injected workspace files: Any workspace bootstrap files present in the project directory, grouped under a "Project Context" header

Full skill instructions are not part of the system prompt. OpenClaw lists skills as compact metadata and lets the model read the full SKILL.md file on demand using read commands. This works like a developer who knows what documentation exists but only opens the relevant page when needed. It keeps the system prompt lean while preserving access to deep instructions.

Beyond the system prompt, the model's context window includes conversation history (your messages and the agent's replies), all tool calls and their returned results, any attached files or transcripts like images and audio, and compaction summaries from earlier in the session.

Tools create two separate context costs that catch people off guard. The tool list text is visible in the system prompt, so you can read it and measure it. But tool schemas, the JSON definitions that tell the model how to call each tool, are also sent to the model. These schemas consume tokens that don't show up as plain text in your prompt. If you have 30 tools registered, their JSON schemas might consume more context than all your workspace files combined.

When the conversation grows long, OpenClaw manages context pressure through two mechanisms. Compaction creates a summary of older messages and preserves recent ones. Pruning removes old tool results from the in-memory prompt but keeps the full history inspectable on disk. Neither mechanism touches your bootstrap files, which are re-injected fresh on every run.

Designing Your Workspace Bootstrap Files

OpenClaw automatically injects these files into the system prompt when they exist in your project directory:

  • AGENTS.md: Primary instruction file. Define your agent's behavior, rules, workflows, and domain knowledge here. This is the file most teams start with and the one that typically consumes the most characters.
  • SOUL.md: Personality and communication style. Tone guidelines, response formatting preferences, and interaction patterns. Keep this concise because personality directives don't need the same detail as operational rules.
  • TOOLS.md: Tool-specific guidance. When to use which tool, parameter conventions, and patterns for combining tools. Useful when your agent has access to many tools and needs guidance on which to reach for first.
  • IDENTITY.md: Agent identity declarations. Name, role, and self-description the agent uses when asked who it is. Rarely needs more than a few hundred characters.
  • USER.md: User-specific context. Preferences, background, technical level, and details about the person the agent works with. Helps the agent calibrate its responses.
  • HEARTBEAT.md: Persistent runtime notes. Information the agent needs to carry across compaction cycles within a session. Think of it as a sticky note the agent can update.
  • BOOTSTRAP.md: First-run only. Initial setup instructions that execute once when you start a new project and don't reappear in later sessions.

Each file has a hard character limit. By default, OpenClaw truncates any single bootstrap file at 20,000 characters and caps the total bootstrap injection at 60,000 characters. Both thresholds are configurable in your OpenClaw settings, so you can raise or lower them to match your project's needs.

When truncation happens, OpenClaw warns you inline by default. You can configure truncation warnings to fire every time, only on the first occurrence, or not at all. Keeping warnings active is recommended until you've established stable file sizes, because silent truncation is one of the most common causes of "my agent stopped following instructions" bugs.

The 60,000-character total cap means you need to budget across files. A common mistake is writing a comprehensive AGENTS.md at 18,000 characters, adding a detailed TOOLS.md at 15,000, a SOUL.md at 10,000, and then wondering why USER.md content gets silently cut. Research suggests models maintain peak performance when context usage stays between 60% and 80% of available capacity, so leaving headroom in your bootstrap budget gives the model room for conversation history and tool results.

A practical allocation strategy:

  • AGENTS.md: 10,000 to 15,000 characters for core behavior rules and domain knowledge
  • TOOLS.md: 5,000 to 8,000 characters for tool-specific guidance
  • SOUL.md: 2,000 to 4,000 characters for personality
  • USER.md: 2,000 to 5,000 characters for user preferences
  • HEARTBEAT.md: 1,000 to 3,000 characters for cross-compaction persistence
  • IDENTITY.md: 500 to 1,000 characters

That totals roughly 20,500 to 36,000 characters, well within the 60,000 cap and leaving room for future additions.

AI analysis dashboard showing document structure and content breakdown
Fastio features

Store your OpenClaw agent's outputs where the next session can find them

50GB free workspace with Intelligence Mode for semantic search and MCP access at /mcp. No credit card, no trial, no expiration.

Inspecting and Debugging Your Context

OpenClaw provides built-in commands to see exactly what the model receives. These commands are your primary tools for diagnosing context issues.

Quick checks:

  • /status shows window fullness and current session settings at a glance. Run this when you want a fast read on how much context room remains.
  • /usage tokens appends a per-reply token breakdown footer to each response, so you can watch consumption grow in real time as the conversation progresses.

Detailed inspection:

  • /context list shows every injected file with its rough character count, both per-file and as a combined total. This is the first command to run when debugging context issues, because it immediately reveals which files are present, how large they are, and whether any hit the truncation cap.
  • /context detail gives a deeper breakdown: per-file sizes, tool schema sizes, skill entry sizes, system prompt length, and the count of compactable messages. This is where you discover hidden context costs like tool schemas consuming more space than your workspace files.
  • /context map generates a treemap visualization showing how every context contributor compares in size. It requires a cached run report, so you need to complete at least one model call before this command produces output.

A five-step debugging workflow:

  1. Run /context list to see what files are injected and their sizes
  2. Check whether any files hit the 20,000-character per-file cap
  3. Run /context detail to compare tool schema sizes against workspace file sizes
  4. If context is tight, decide whether to trim bootstrap files or disable unused tools
  5. Use /context map for a visual overview after a few turns of conversation

The most common surprise is tool schema size. A project with 30 registered tools might dedicate 40% or more of its context budget to JSON schemas alone. Disabling tools you don't actively use, or reducing their schema verbosity, frees significant space for workspace files and conversation history.

Another common issue is truncation happening without the developer noticing. If you disable truncation warnings and your AGENTS.md grows past 20,000 characters, your agent silently loses instructions from the bottom of the file. Keep warnings active until you've established stable file sizes.

Building a Custom Context Engine

OpenClaw's built-in context engine handles prompt assembly and compaction using default logic. For most projects, it works well. But if you need different assembly strategies, custom compaction, or cross-session recall, you can replace it entirely with a plugin.

A context engine plugin uses kind: "context-engine" and participates at four core points in the prompt lifecycle:

  1. ingest: Called when a new message arrives. Preprocess, classify, or index messages before they enter history. Use this for embedding messages into a vector store or tagging them for later retrieval.
  2. assemble: Called before every model run. Return an ordered set of messages that fits within the token budget. This is where you implement custom retrieval, relevance scoring, or cross-session recall.
  3. compact: Called when context is full or when the user triggers /compact. Summarize or compress older history. You decide what to preserve and what to condense.
  4. afterTurn: Called after each model response. Handle post-processing, state persistence, or background compaction scheduling.

Three optional hooks cover advanced scenarios: bootstrap for engine initialization at session start, prepareSubagentSpawn for setting up isolated context before a child agent launches, and onSubagentEnded for cleanup when a subagent completes.

To activate a custom engine, configure it in your OpenClaw settings under plugins.slots.contextEngine with the engine's registered ID. The system loads the plugin at startup and routes all context operations through it. If the plugin fails at runtime, OpenClaw quarantines it for the current process and falls back to the legacy engine automatically. This failure isolation means a buggy engine won't permanently break your agent.

One important flag to understand: ownsCompaction. When set to true in the engine's info object, your engine takes full responsibility for compaction and OpenClaw disables built-in auto-compaction entirely. When unset, OpenClaw's built-in compaction may still run during execution alongside your engine's compact method. If you want to delegate back to OpenClaw's summarization from inside your engine, the delegateCompactionToRuntime() helper is available.

The assemble method can return a systemPromptAddition, a block of text prepended to the system prompt on every call. This lets your engine inject dynamic recall guidance, retrieval results, or context-aware hints without requiring static workspace files. For example, a RAG-focused engine could embed relevant document snippets into the system prompt based on the current conversation topic.

Most teams don't need a custom context engine. The built-in legacy engine handles standard workflows well. Consider building one when you need cross-session memory recall during assembly, custom compaction that preserves specific information types, or specialized handling for subagent context boundaries.

Persisting Agent Work Beyond the Context Window

Context engineering solves what your agent sees during a session. But sessions end. Context gets compacted. The work products your agent creates, research notes, generated files, analysis reports, need to live somewhere durable.

Local file storage works for single-developer setups. Your OpenClaw agent writes files to disk, you read them, and everything stays on one machine. This breaks down when agents collaborate with other agents, when you need to hand results off to colleagues, or when you want the next session to query what was built previously without manually copying files back into context.

Cloud storage services like S3, Google Drive, or Dropbox solve the durability problem but add a different one: they're passive containers. You upload files and the storage layer is done. No indexing, no semantic search, no way for the next agent session to ask "what did we produce last week?" without downloading and re-processing everything.

Fast.io provides workspaces where agents and humans share files with built-in AI that indexes content automatically. When an OpenClaw agent writes output to a Fast.io workspace, that content becomes searchable by meaning through Intelligence Mode, queryable through chat with citations, and accessible to both agents and humans through the same interface.

The Fast.io MCP server exposes workspace operations through Streamable HTTP at /mcp and legacy SSE at /sse. An OpenClaw agent can read from and write to workspaces using MCP tools, treating the workspace as persistent context that survives session restarts, compaction cycles, and machine changes. Full integration details are available in the MCP documentation.

Ownership transfer matters for agents that build deliverables for clients. An agent account creates the workspace, populates it with research or reports, then transfers the organization to a human. The agent keeps admin access for maintenance while the human becomes the owner.

The free agent plan includes 50GB of storage, 5,000 credits per month, and 5 workspaces with no credit card, no trial period, and no expiration. For teams running multiple OpenClaw agents, file locks prevent conflicts when agents write to the same workspace concurrently, and audit trails track every file operation so you can trace what each agent did and when.

Frequently Asked Questions

What is context engineering in OpenClaw?

Context engineering in OpenClaw is the practice of deliberately designing what information your agent sees in its system prompt. This includes choosing which workspace bootstrap files to create (AGENTS.md, SOUL.md, TOOLS.md, and others), setting truncation limits so critical instructions aren't cut, and inspecting the assembled prompt to verify the agent receives the right information. It differs from prompt engineering in that you're designing the structural container for the model's input, not just the text within individual messages.

How do I optimize my OpenClaw context window?

Start by running /context list to see which files are injected and their sizes. Check whether any files hit the 20,000-character per-file cap or the 60,000-character total cap. Trim files that carry low-value content and move detailed reference material into skill files that the agent reads on demand rather than injecting into the bootstrap. Disable tools you don't actively use, since their JSON schemas consume context that isn't visible as plain text. Use /context map after a few turns to visualize how context is distributed across components.

What goes in AGENTS.md vs SOUL.md?

AGENTS.md contains behavioral rules, domain knowledge, and workflow instructions. It defines what the agent does and how it should approach tasks. SOUL.md defines personality and communication style, covering tone guidelines, formatting preferences, and interaction patterns. Think of AGENTS.md as the job description and SOUL.md as the personality of the person doing the job. Keep SOUL.md concise because personality directives rarely need the same level of detail as operational rules.

How do I inspect what OpenClaw sends to the model?

Use /context list for a quick overview of injected files and their sizes. Use /context detail for a deeper breakdown that includes tool schemas, skill entries, and system prompt length. Use /context map to generate a visual treemap showing context allocation after at least one model call. For real-time tracking, /usage tokens adds a token consumption footer to each reply, and /status shows overall window fullness.

What are the default bootstrap truncation limits in OpenClaw?

OpenClaw truncates individual bootstrap files at 20,000 characters and caps total bootstrap injection at 60,000 characters. Both limits are configurable in your OpenClaw settings. When truncation occurs, an inline warning appears by default. You can adjust the warning frequency to show on every occurrence, only on the first, or turn it off entirely.

Do I need a custom context engine for OpenClaw?

Most projects work well with the built-in legacy context engine. Consider building a custom engine when you need cross-session memory recall during prompt assembly, custom compaction strategies that preserve specific types of information, or specialized subagent context handling. The plugin interface requires three core hooks (ingest, assemble, compact) and registers through kind context-engine in the plugin system. If a custom engine fails at runtime, OpenClaw automatically quarantines it and falls back to the legacy engine.

Related Resources

Fastio features

Store your OpenClaw agent's outputs where the next session can find them

50GB free workspace with Intelligence Mode for semantic search and MCP access at /mcp. No credit card, no trial, no expiration.