Context Is All You Need: How Self-Improving Agents Learn From Their Mistakes
Most AI agents are stateless. They solve problems, forget everything, and start fresh. Self-improving agents flip this model by accumulating context across sessions. This post explores context as the central abstraction that makes cross-session learning possible.
Most AI agents are stateless. They solve your problem, forget everything, and start fresh next time. This works for simple tasks, but it can be wasteful. The agent makes the same mistakes repeatedly, never learning from experience.
Self-improving agents flip this model. They persist what they learn between sessions, building up a memory of what works and what doesn’t. The result: an agent that gets better at your specific problems over time, without retraining the underlying model.
This post explores context as the central abstraction that makes this possible. Context isn’t just “what the agent knows right now.” It’s the accumulated wisdom from past attempts, structured so the agent can apply it to new problems. This post covers how context flows, accumulates, and persists in a working implementation.
What Is Context, Really?
In a self-improving agent, context is a structured bundle of information that shapes how the agent approaches a task. It has layers:
- Task context: The query itself, expected output format, available data
- Prior context: Examples from similar past problems, hints about common pitfalls
- Attempt context: What the agent tried, what went wrong, evaluator feedback
- Improvement context: Synthesised lessons - “don’t filter on column X, use column Y instead”
The key insight: context accumulates. Each failed attempt adds information. Each past session contributes examples. The agent facing attempt #3 has richer context than the agent facing attempt #1.
The Flow: Generate, Evaluate, Improve
The agent operates in a loop. Each iteration enriches the context for the next. Here’s the flow:
-
Load priors - Before attempting anything, the agent searches its memory for similar past problems. If it finds matches, their solutions become examples in the prompt.
-
Execute - The agent generates code based on current context, runs it in a sandbox, captures the output.
-
Evaluate - A separate evaluator compares the output to the expected answer. If wrong, it classifies the error type and generates feedback.
-
Improve - The feedback becomes new context. The agent now knows “I tried X, it failed because Y, consider Z instead.”
-
Persist - Whether pass or fail, the agent saves the attempt. Failures become cautionary examples; successes become reusable patterns.
The loop continues until the task passes or attempts are exhausted. Here’s the important part: even failures are valuable. They become context for future sessions.
Accumulation: Context Gets Richer With Each Failure
Here’s what happens to context across three attempts at the same task:
Attempt 1: The agent has only the task and any prior examples from storage. It generates code, runs it, gets the wrong answer. The evaluator says: “Output was 125,000 but expected 150,000. Error type: filter_error. The code filtered on ‘Revenue’ but should have used ‘Gross Revenue’.”
Attempt 2: Now context includes the failed code, the error classification, and the hint. The agent generates different code - but makes a new mistake. The evaluator adds: “Correct column now, but filtered Q1 instead of summing Q1-Q4.”
Attempt 3: Context now contains two failures with two distinct lessons. The agent has seen what doesn’t work. It generates code that passes.
Each attempt didn’t just retry - it retried with more information. The context grew from “here’s the task” to “here’s the task, here’s what failed, here’s why, here’s what to avoid.”
Persistence: Context Survives Between Sessions
The accumulation within a session is useful, but the real power comes from cross-session learning. When the agent finally solves a problem it saves a structured record called an episode and then moves on.
An episode captures:
- The original query
- The code that failed (and why)
- The code that worked
- Keywords extracted for similarity matching
- An effectiveness score (updated over time)
When a new session starts with a similar problem, the agent searches its episode storage. It finds matches using keyword similarity - “gross revenue Q1 2024” matches “gross revenue Q2 2023” even though the specifics differ. The working code from the old episode becomes an example in the new prompt.
This is the payoff: the agent doesn’t re-discover the same lessons. It starts attempt #1 with context that took three attempts to learn last time.
Pruning: Not All Context Is Good Context
There’s a risk: what if an episode encodes a wrong pattern? An agent might learn a “fix” that worked once by luck but fails on similar problems. Bad context is worse than no context - it confidently misleads.
The system tracks effectiveness. Each episode has a score that updates every time it’s applied:
- Applied and the task passed? Score goes up.
- Applied and the task failed? Score goes down (weighted more heavily).
- Old and rarely used? Score decays over time.
When retrieving episodes, the agent filters by effectiveness threshold. Low-scoring episodes don’t make it into context. They’re still stored for debugging, but they stop influencing new attempts.
This creates a self-correcting loop: the agent’s memory improves not just by adding episodes, but by learning which episodes actually help.
When Does This Matter?
Self-improving agents aren’t always worth the complexity. If your tasks are one-off and varied, there’s nothing to learn across sessions. If your domain changes rapidly, old episodes become misleading fast.
But for repeated, domain-specific tasks - the same kinds of queries against the same kinds of data - context accumulation compounds. The agent gets measurably better at your problems without any model fine-tuning or retraining. You’re building institutional memory into the system.
The key trade-offs:
- Storage overhead: Episodes accumulate. You need a strategy for pruning or archiving.
- Cold start: The first session has no priors. Benefits emerge over time.
- Domain drift: If your data schema changes, old episodes may hurt more than help.
For data analysis tasks - querying financial data, generating reports, answering business questions - the pattern fits well. The same column names, the same aggregation patterns, the same edge cases appear repeatedly. Context becomes a competitive advantage.
The implementation described here is available at github.com/TAJD/adaptive-agent. It’s a working demonstration of these concepts - not production-ready, but complete enough to experiment with and extend.