Self-Improving Agents: A Practical Guide to Continuous Learning

We build agents to take work off engineers’ plates. Then we give those engineers a new manual job: reading failed runs and babysitting prompts.

Agents will improve themselves automatically. We’re not there yet, but this is the future I’m betting on.

We’ve been working on this ourselves at Komodor over the past year. We know how hard it is to turn a failure into an improvement that holds up beyond a few examples. That experience has made me more convinced this is where things are going.

The model, prompt, and tools are all means to an end: the outcome. They should change whenever there is a better way to deliver it.

Here’s the full approach, with a small example you can run.

One Way to Do It: Teacher and Student

Start with an agent that already does a job well, usually running an expensive model. You want a cheaper agent to take over, but you need evidence that it can deliver the same quality.

The teacher-student approach gives you a reference to improve against. Both agents receive the same inputs and are evaluated against the same requirements. The teacher shows what your existing agent achieves; the student is the version you keep changing.

The interesting part is what happens when the student falls short. Rather than having an engineer investigate every failure, another agent reads the evaluation feedback and proposes a change. You test that change and repeat.

There are four roles:

1. Teacher: the agent you trust today, kept on a fixed configuration throughout the experiment.
2. Student: the agent you want to improve, usually a cheaper or open model, doing the same job on the same inputs.
3. Evaluator: checks both answers against the requirements, explains its judgments, and does not know which agent produced each answer.
4. Coach: reads the student’s failures and evaluation feedback, then proposes a change without seeing the answer key.

The teacher doesn’t write the student’s prompt. The coach does. And the student is judged on whether it does the job correctly, so it can produce a different answer from the teacher and still succeed.

In the simplest version, both start with the same prompt. The coach then adapts the student’s instructions based on where it struggles. After each revision, you rerun the cases, including ones the student already handled correctly. Fixing one failure is only useful if the change doesn’t introduce unacceptable regressions elsewhere.

Choose an acceptable margin around the teacher’s performance before starting. Once the student reaches it, a person reviews the results and decides whether to promote it, keeping the teacher as a fallback. Human-approved answers can also become trusted examples for future evaluations.

That person should receive a proposed change with evidence: which failures it fixed, what happened to previously successful cases, and whether anything regressed. The manual work moves from finding and writing every fix to deciding which tested changes should ship.

How it Works: Watch the Prompt Change

I built a small version in komodorio/hello-world-continual-learning. It uses four support tickets for a fictional store. The agent receives a customer message and a policy snippet, then writes a reply. The coach can only change the prompt, so the effect of each revision is easy to follow.

Below are simplified excerpts illustrating the prompt changes, rather than the full repository prompts.

Version 1: describe the job.

      Answer the customer using the supplied store policy. Include the resolution and next steps. Keep the reply under 120 words.

This tells the agent what to deliver, but leaves it to work out how to apply the policy.

In the recorded example, the student applies the wrong policy time window and rejects a valid claim. The evaluator identifies that mistake, and the coach makes the decision process more explicit.

Version 2: make the failing step explicit.

      Before deciding eligibility, identify every applicable policy window and which event starts it. Check the customer’s dates against the correct event before deciding.

The student now makes the correct decision. But the revised instructions lead to longer answers, and several exceed the word limit. The next change needs to preserve the better reasoning while fixing the new problem.

Version 3: keep the improvement and address the regression.

      Check eligibility against the correct policy window. Include the decision and all required next steps within 120 words. Remove repetition and shorten explanations. Cut wording rather than content.

That reflects the next revision in the example. The student is tested again against every ticket, checking that the replies still contain the required information and reach the right decision.

The coach also receives the earlier prompts and their score history. It has evidence about which instructions helped and which made things worse. That matters because otherwise every failure can become another paragraph in an increasingly unwieldy prompt. The supplied draft describes an earlier attempt where the prompt kept growing while performance deteriorated.

The engineer didn’t write each of those revisions. The evaluator exposed the failure, the coach proposed a change, and the next run checked its effects. The revised prompt becomes a candidate for the next version of the agent, rather than a correction that disappears when the conversation ends.

Think of it as “Gradient Descent” for the Agent

There is a useful optimization analogy here. You measure an error, use it to choose an adjustment, and check whether the adjustment reduced the error.

In this loop, the evaluation feedback supplies a proposed direction. “You used the wrong policy window” becomes “identify which event starts each window before deciding.” The coach turns that criticism into an edit, and another evaluation checks whether the edit helped.

Prompt-optimisation research such as ProTeGi describes a related mechanism as “textual gradients”: natural-language feedback about failures guides changes to a prompt. It is an analogy to gradient descent, not literal differentiation through sentences or an update to the model’s weights. (arxiv.org)

The distinction matters. A coach can give a very convincing explanation for a terrible change. Its reasoning suggests where to look; the test results determine whether the change is worth keeping.

That is also how I would approach changes beyond prompts. A proposal to switch models or replace a tool should come with a hypothesis about what will improve and evidence from testing it.

What Makes This Continuous?

Feed the loop with real work. Capture approvals, edits, and rejections alongside the original inputs; reviewed corrections become trusted examples. Trigger another cycle when failures recur, the agent changes, or its environment changes.

Test candidates through historical replay, then shadow runs with side effects disabled, and controlled A/B tests where appropriate. A person decides what ships. The goal is for improvement work to keep happening without an engineer having to initiate every round.

Where this Approach Breaks?

The hard problems remain even with a much larger dataset and a more sophisticated implementation.

1. You can optimise the wrong thing. If the evaluator rewards convincing answers more reliably than correct decisions, the loop can raise the score without improving the agent. Check its judgments against expert reviews and use separate checks for distinct requirements. A friendly answer must not compensate for a wrong policy decision.
2. It can overfit to the work it has already seen. Repeatedly revising prompts against the same scenarios can improve results in those specific cases without helping with new ones. Production-derived examples are useful, but growing the dataset is not enough. Keep fresh cases outside the coaching loop and evaluate whether improvements survive changes in the workload. The example itself has no hold-out set.
3. The teacher can preserve the wrong standard. Matching the teacher helps you decide whether a cheaper agent can take over. But the teacher can be wrong, and the student can inherit those mistakes. Keep checking both against the actual requirements. Reaching the target means the student met your comparison threshold, not that you have established correctness.
4. More iterations can make things worse. A change can fix one failure and create another. Instructions can accumulate, conflict, or make the agent overly cautious. Preserve the best-known version and the history of attempted changes, and set a search budget. Finishing without a candidate worth promoting must be an acceptable outcome.
5. The improvement process has its own cost. Teacher runs, student runs, grading, and coaching all consume resources. Shadowing adds work before a cheaper student delivers any savings, and testing actions requires safe substitutes for real side effects. Evaluate the economics of the whole process, rather than counting only the final student’s inference cost.
6. The coach may be changing the wrong component. Better instructions cannot supply missing information or repair a broken tool. A broader system needs to recognise when prompt changes have reached their limit and propose changes elsewhere. Those changes need their own tests and permissions; giving the coach more freedom also increases what can go wrong.

Building that broader improvement loop across prompts, models, and tools is what we are working on with AgentOps at Komodor.

Repo: https://github.com/komodorio/hello-world-continual-learning | From the checkout: uv sync, configure your keys, then uv run prompt-coach run; use uv run prompt-coach serve to inspect saved runs.

When your agent makes the same mistake next week, will the system do anything about it, or will another engineer rewrite the prompt? Komodor’s Agentic Operations Platform is designed to keep agents in check and ensure that they can be trusted to run reliably and efficiently in production, while continuously learning and improving. Try for yourself: https://komodor.com/free-trial/