Tracing

Definition

Recording the sequence of model calls, tool calls, intermediate steps and actions taken by an AI system.

Put simply

It gives you a record of the path the AI took.

What boards and leaders should understand

For a simple chatbot, seeing the final answer may be enough to understand what happened.

For an agent that performs dozens of steps across several systems, the final outcome tells only part of the story.

Tracing becomes important for troubleshooting, assurance and accountability because it can help reconstruct what the agent saw, what it did and where the process went wrong.

Questions to consider

  • If an agent caused a material incident, could we reconstruct its actions quickly enough to understand and contain it?