Debugging AI agents: why the output was wrong and how to fix it

The reflex when an agent produces something wrong is to rewrite the prompt. Usually that is the wrong move, because the prompt was rarely the problem.

Four things cause almost every bad output. Work them in order. Only the last one is the model, and it is the rarest.

1. It did not have the context

By far the most common. The agent produced something reasonable given what it could see, and what it could see was incomplete.

Symptoms: it invents a function that does not exist, uses a pattern from a different part of the codebase, or solves a problem you already solved somewhere it never looked.

The tell is that the output is internally coherent but disconnected from your reality. That is not confusion. That is working from a smaller world than you assumed.

The fix is not a better instruction, it is better context. Give it the file instead of describing the file. Paste the actual error, with the stack trace, rather than characterising it. If you find yourself explaining what the code does, you should be showing it the code.

Test: could a competent engineer, given exactly what the agent was given and nothing else, have produced the right answer? If not, it is a context problem and no amount of prompt engineering fixes it.

2. The tool surface misled it

If the agent has tools, a wrong action is often a naming problem rather than a reasoning problem.

Two similarly named tools, a vague description, an error message that reports failure without saying what would succeed - each produces confident wrong choices. Watch which tool it reached for first. That first choice tells you what the name and description communicated, which is frequently not what you meant.

The fix lives in the tool definition, not the prompt. Sharper name, explicit “use this when / do not use this for”, and errors written so the caller can correct and retry.

3. The instructions contradict each other

As context accumulates - project instructions, a skill, a long conversation, your latest correction - the guidance stops being consistent. The agent is not ignoring you. It is resolving a conflict you did not know you had created, and resolving it differently than you would.

Symptoms: it follows an instruction from earlier and drops the recent one, or oscillates between two behaviours across turns.

Look for the contradiction rather than restating the instruction more firmly. “Always ask before making changes” plus “work autonomously and do not check in” cannot both hold. Delete one. Adding a third instruction to arbitrate makes it worse.

Long sessions accumulate this naturally. If behaviour degrades over a long conversation, that is usually what happened, and starting fresh with the good context is faster than arguing with the accumulated mess.

4. It genuinely cannot do it

Real, and rarer than people assume. Some tasks need judgement about consequences the model has no way to evaluate, or context that exists only in someone’s head, or verification against a system it cannot reach.

The tell: it gets close repeatedly but fails the same way each time, and each of your corrections produces a different failure rather than convergence.

At that point stop. Do the part that needs you, and hand back the part that does not. Recognising this early is a skill; grinding against it is where most wasted time goes.

Read the transcript, not the summary

The agent’s account of what it did is a reconstruction. The transcript is evidence. When something went wrong, read what actually happened: which tools were called, with what arguments, what came back.

Nine times out of ten the moment it went wrong is visible and earlier than you thought - a tool returned an empty result and it carried on regardless, or a search matched nothing and it inferred rather than asked.

Change one thing

The temptation on a bad output is to rewrite the instructions, add context, and adjust the tools all at once. Then it works and you have no idea why, so you cannot apply the lesson anywhere else.

Change one thing. See what happens. It is slower for one bug and much faster across ten, because you end up understanding your setup rather than accumulating superstitions about it.

When the agent gets it wrong: a debugging method