How do I monitor AI agents in production? Verify what each run changed.

How do I monitor AI agents in production? Record a trace for every run, track cost and latency for every step, and set alerts before you launch. Then add the part that most guides leave out. Check the path of each run and the change that it made in the world. An agent can return “success” after a wrong path. No error appears, and latency looks normal. Only a check on the tool calls and on the final state shows the fault. I build agents for a living, and this is the check that I add first.

How do I monitor AI agents in production? The short answer

What should a trace capture at each step?

The vendor pages agree on the basic model. Sentry, Mastra and other tools describe a trace as a tree of spans. The tree starts at the user input. It goes through each model call and each tool call. It ends at the output. Logs record the single events inside the tree: which tool ran, with which parameters, what it returned, and which error occurred.

I use one rule for the span design. A span must hold enough data to replay the step. For a model call, save the model id, the prompt, the reply, the input tokens, the output tokens and the latency. For a tool call, save the tool name, the arguments, the result and the error. Also save a run id and a step number on every span. Without them, you cannot put the steps back in order.

Do not write this schema by hand if you do not need to. Open-source libraries such as OpenLLMetry, OpenInference and OpenLIT add this instrumentation to the common model SDKs and agent frameworks. They send spans in the OpenTelemetry format. Thus you can change the backend later without a rewrite.

OpenTelemetry also defines attribute names for these spans. The GenAI conventions moved to their own repository. The page there has the status “Development”. Attributes such as gen_ai.operation.name, gen_ai.request.model, gen_ai.tool.name, gen_ai.tool.call.arguments and gen_ai.tool.call.result all have this status. The names can change. Check the status again each time you upgrade your instrumentation library.

Which metrics and alerts go in before launch?

Start with the metrics that every vendor lists. Track latency for each span, input tokens, output tokens, cost for each request, error rate and model id. Then attach quality scores to the outputs. The scores come from the evals that I describe below. Put the model id on every metric. A silent model upgrade by your provider is a common cause of a sudden change in behavior.

Cost needs the most care, because the total hides the cause. A GMI Cloud guide gives an example of a retry loop that calls a model 50 times instead of 5. Without cost logging for each step, you do not see it. The numbers in that example are an illustration, not a measurement. The risk is real, though. Mastra also warns that token costs can spike overnight when an agent enters a retry loop. Sentry says the same about a vector store outage.

Set three alerts before the first production deployment. The first alert fires when the error rate goes above the baseline. The second fires when p99 latency goes above a limit that you choose. The third fires when the cost of one session goes above a limit. The GMI Cloud guide recommends this set of alerts. The same source suggests a hard stop: kill a session that goes over a step count, for example 20 tool calls for a task that normally needs 5.

I add a fourth rule that is not an alert. Put a limit on repeated calls in the agent itself. If the agent calls the same tool with the same arguments again and again, stop the run. I wrote about this control in preventing tool call cycles. An alert tells you after the money is gone. A limit in the loop stops the loss.

How do I catch a run that succeeded after a wrong path?

This is the gap in most monitoring guides. Error rate and latency catch a crash and a slow run. They do not catch a run that finishes with a clean reply after it did the wrong work. The agent skipped a step. Or it called a tool that you did not expect. Or it passed a wrong argument, and the tool accepted it. The final status is “success”, and every dashboard is green.

Mastra writes that the same user query can produce different tool-call sequences. It also says that agents find tool-call sequences “that work but were never anticipated during development”. Thus a normal-looking run can hide a bad path. In multi-agent systems, Mastra describes a subagent that returns “a subtly wrong intermediate result” that the supervisor accepts. The visible error, if one appears, is often a consequence. It is not the cause. The cause is earlier in the trace.

So you check two things. First, check the trajectory. Compare the sequence of tool calls to the sequence that you expect for that task. Look for repeated calls, skipped steps and tools that should not appear. Second, check the change in state. After the run, read the system that the agent changed. Did the refund exist? Did the row change? Did the email go to the right person? The reply of the agent is a claim. The state of the system is the proof. I described this rule in delta verification: assert the change, not the absolute state.

The figure below shows why the depth of a check matters. One implementation carries six hidden defects toward the customer. You choose how many checkpoints are active.

Press Run trajectory with “unit only” and count the defects that leak. Then switch to “integration” and to “near-boundary”. Each defect is caught at the first active checkpoint that can see it. The leak count falls as the checks move closer to the customer. One defect leaks in every mode. The spec never described it. Only a human can decide that one. Production monitoring has the same limit, and I accept it.

How do online evals on sampled traffic fit in?

You cannot read every trace. You can score a sample. An online eval takes a fraction of live traces and gives each one a score. The score is a quality number that you attach to the trace, as the vendor guides recommend. Then you chart the score over time and alert on a drop, as you do for latency.

Choose the sample with a purpose. Take a small random sample of all traffic, to measure the baseline. Take all runs that are unusual: runs with more steps than the median, runs with a repeated tool call, runs with a high cost, and runs that the user rated badly. These runs find more faults than a random sample does.

There are two kinds of scorer. A rule-based scorer is cheap and exact. It checks the tool sequence against the expected sequence, or it checks that a required tool ran before a write. Mastra calls the related metric tool-call accuracy: whether the agent called the right tools with the right parameters. A model-based scorer judges the quality of the reply, for example if the reply answers the question. It costs tokens, and it can be wrong. Calibrate it against a set of traces that a person has labeled.

Do not use the model-based score as the only gate. It reads the text of the run. It cannot see that the database did not change. Use it for the questions that rules cannot answer, and keep the rule-based and state checks for the rest.

How do I probe the live system on a schedule?

Sampled traffic shows what users did. It does not show what users did not try, and it does not run when traffic is low. A scheduled probe fills this gap. A probe is a test run against the live service, with a task that you know. You know the correct path and the correct end state. Thus you can check both.

I built this idea into Elwen. A harness points an agent at the live service. The agent follows a plain-English brief and returns typed findings with steps to reproduce. In the first stage, cheap deterministic checks run, and they cost no tokens. The agent starts only after these checks pass. The same pattern works as a monitor. Run the probe after each deploy and on a timer, for example every hour. I describe the harness idea in more detail in harness as a service.

Press Run probe and watch the event pass the pre-pass, the prompt and the agent. Then watch the findings arrive one at a time in the inbox. Switch the profile from web to api to security, and see that the probe looks for different things. The figure shows the deploy-time use of the harness. For monitoring, you send the same kind of event from a scheduler.

Give each probe task a known end state. For example, the probe asks the agent to create a test order and to cancel it. After the run, a rule reads the order table and checks that the order has the status “cancelled”. Use test accounts, and delete the test data after each run. If a probe fails, send the full trace and the repro steps to your alert channel. A person must be able to replay the failure without a long search.

So, how do I monitor AI agents in production?

Record a trace for every run with enough data to replay each step. Track cost, latency and errors for each span, and set alerts for error rate, p99 latency and session cost before you launch. Add a limit on steps and repeated calls inside the agent. Then close the gap that dashboards leave open. Compare each sampled run to the expected tool path, read the state that the run changed, and run a scheduled probe against the live service with a known end state. A green status tells you that the agent finished. These checks tell you that it did the right work. If you want help to build this for your agents, see my page for an agentic AI engineer.