title: "AI Agent Debugging: Tools, Techniques, and Best Practices" slug: "ai-agent-debugging-tools-and-techniques" description: "A comprehensive guide to debugging AI agents — tracing tools, prompt debugging, hallucination detection, and the observability stack that makes agent debugging systematic." category: "Engineering" publishedAt: "2026-07-22"

AI Agent Debugging: Tools, Techniques, and Best Practices

Debugging AI agents is fundamentally harder than debugging traditional software. When your code calls a function and gets unexpected results, you can trace the execution path, inspect variables, and reproduce the issue. When an AI agent produces a wrong answer, the "bug" might be in the prompt, the model, the context, the tool call, or the interaction between all three. This guide makes agent debugging systematic.

Why AI Agent Debugging Is Hard

Traditional debugging assumes deterministic systems. You set a breakpoint, inspect state, and trace execution. AI agents break these assumptions:

• Non-deterministic outputs — The same input produces different outputs across runs
• Opaque reasoning — You can't always inspect "why" the agent chose a particular path
• Distributed execution — Agents call external tools, APIs, and models across network boundaries
• Context sensitivity — Agent behavior depends on full conversation history, not just current input
• Emergent failures — Individual components work correctly in isolation but fail when combined
• Model version drift — Model updates silently change agent behavior

These challenges require new debugging approaches and specialized tooling.

The AI Agent Debugging Stack

Effective agent debugging requires four layers of observability:

Layer 1: Execution Tracing

Every agent run should produce a complete trace showing:

• Input received
• System prompt used
• Model called (name, version, parameters)
• Tools invoked (name, parameters, response)
• Intermediate reasoning steps
• Final output delivered
• Token counts and costs
• Latency per step

from dataclasses import dataclass, field
from datetime import datetime
import json

@dataclass
class AgentTrace:
    session_id: str
    steps: list = field(default_factory=list)
    
    def add_step(self, step_type, data):
        self.steps.append({
            "timestamp": datetime.utcnow().isoformat(),
            "type": step_type,  # "llm_call", "tool_call", "decision", "error"
            **data
        })
    
    def to_json(self):
        return json.dumps(self.steps, indent=2, default=str)

# Usage
trace = AgentTrace(session_id="sess_123")

trace.add_step("input", {
    "user_message": "What's the weather in Berlin?",
    "context_length": 4500
})

trace.add_step("llm_call", {
    "model": "glm-5.0",
    "input_tokens": 4520,
    "output_tokens": 85,
    "latency_ms": 1200,
    "decision": "call_tool",
    "tool_name": "get_weather"
})

trace.add_step("tool_call", {
    "tool": "get_weather",
    "params": {"location": "Berlin"},
    "result": {"temp": 22, "condition": "sunny"},
    "latency_ms": 340
})

trace.add_step("llm_call", {
    "model": "glm-5.0",
    "input_tokens": 4620,
    "output_tokens": 45,
    "final_output": "The weather in Berlin is 22°C and sunny.",
    "latency_ms": 800
})

Layer 2: Prompt Inspection

For each LLM call in the trace, capture the exact prompt sent:

trace.add_step("llm_call", {
    "model": "glm-5.0",
    "messages": [
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_message},
        {"role": "assistant", "content": previous_response},
        {"role": "user", "content": new_query}
    ],
    "temperature": 0.7,
    "max_tokens": 1000
})

This lets you reproduce the exact model call during debugging.

Layer 3: Tool Call Inspection

Capture the exact parameters and responses for every tool call:

trace.add_step("tool_call", {
    "tool": "database_query",
    "params": {"query": "SELECT * FROM users WHERE created_at > '2026-07-01'"},
    "result_rows": 342,
    "result_sample": [...first_5_rows],
    "latency_ms": 145,
    "error": None
})

Layer 4: Decision Reasoning

When an agent decides between multiple paths, capture why:

trace.add_step("decision", {
    "decision": "escalate_to_human",
    "reasoning": "Confidence score 0.42 < threshold 0.60",
    "alternatives_considered": [
        {"option": "auto_respond", "confidence": 0.42},
        {"option": "request_more_info", "confidence": 0.55},
        {"option": "escalate_to_human", "confidence": 0.88}
    ]
})

Common Bug Categories and Solutions

Bug 1: Agent Calls Wrong Tool

Symptoms: Agent calls send_email when user asked to "draft an email" Root Cause: Tool descriptions too similar, or prompt doesn't distinguish between "draft" and "send"

Debugging Steps:

1. Inspect the tool descriptions in the system prompt
2. Check if the model saw both tools and chose wrong, or only saw one
3. Improve tool description specificity:

# Before (ambiguous)
@mcp_tool(name="send_email", description="Send an email")
async def send_email(to, subject, body): ...

@mcp_tool(name="draft_email", description="Create an email draft")
async def draft_email(to, subject, body): ...

# After (specific)
@mcp_tool(name="send_email", description="Immediately send an email to recipients via SMTP. This action is irreversible.")
async def send_email(to, subject, body): ...

@mcp_tool(name="draft_email", description="Create an email draft in the drafts folder. Does not send. Returns draft ID for later review.")
async def draft_email(to, subject, body): ...

Bug 2: Agent Hallucinates Tool Parameters

Symptoms: Agent calls search(query="berlin weather today") but the tool expects search(location="Berlin", date="2026-07-22")

Debugging Steps:

1. Check the tool's parameter schema in the trace
2. Verify the model received the correct JSON schema
3. Add explicit parameter examples to the tool description

@mcp_tool(
    name="search",
    description="Search for information. Example: search(location='Berlin', date='2026-07-22')",
    schema={
        "location": {"type": "string", "description": "City name, e.g., 'Berlin', 'Munich'"},
        "date": {"type": "string", "pattern": "^\\d{4}-\\d{2}-\\d{2}$", "description": "Date in YYYY-MM-DD format"}
    }
)

Bug 3: Agent Loses Context in Long Conversations

Symptoms: After 8-10 turns, agent forgets earlier information Root Cause: Context window exceeded, or earlier messages truncated

Debugging Steps:

1. Check token count at each step in the trace
2. Identify where truncation occurs
3. Implement context summarization:

class ConversationManager:
    def __init__(self, max_context_tokens=8000):
        self.max_context_tokens = max_context_tokens
        self.messages = []
    
    def add_message(self, role, content):
        self.messages.append({"role": role, "content": content})
        self._manage_context()
    
    def _manage_context(self):
        total_tokens = sum(estimate_tokens(m["content"]) for m in self.messages)
        
        if total_tokens > self.max_context_tokens:
            # Keep system prompt + last 4 messages, summarize the rest
            system = self.messages[0]
            to_summarize = self.messages[1:-4]
            recent = self.messages[-4:]
            
            summary = llm.summarize([
                {"role": "system", "content": "Summarize this conversation concisely, preserving key facts and decisions."},
                {"role": "user", "content": json.dumps(to_summarize)}
            ])
            
            self.messages = [system, {"role": "system", "content": f"Previous conversation summary: {summary}"}, *recent]

Bug 4: Agent Produces Different Results for Same Input

Symptoms: Temperature >0 causes non-deterministic outputs Root Cause: High temperature or lack of few-shot examples

Solution:

• Set temperature=0 for tasks requiring consistency
• Use few-shot examples to anchor behavior
• Cache responses for identical inputs

Bug 5: Infinite Tool Call Loops

Symptoms: Agent repeatedly calls the same tool with the same parameters Root Cause: Tool returns data the agent can't parse, or agent's goal is never satisfied

Solution:

class LoopDetector:
    def __init__(self, max_repeated_calls=3):
        self.call_history = []
        self.max_repeated_calls = max_repeated_calls
    
    def check(self, tool_name, params):
        signature = f"{tool_name}:{json.dumps(params, sort_keys=True)}"
        recent = self.call_history[-10:]
        
        repeat_count = recent.count(signature)
        if repeat_count >= self.max_repeated_calls:
            raise LoopDetectedError(
                f"Tool '{tool_name}' called {repeat_count} times with identical params. "
                "Breaking potential infinite loop."
            )
        
        self.call_history.append(signature)

Bug 6: Agent Exceeds Budget

Symptoms: Agent burns through token budget in a single conversation Root Cause: Verbose system prompt, large tool responses, or excessive reasoning steps

Solution:

class BudgetManager:
    def __init__(self, max_tokens_per_session=50000, max_cost_eur=0.50):
        self.session_usage = {}
        self.limits = {"tokens": max_tokens_per_session, "cost": max_cost_eur}
    
    def check_and_deduct(self, session_id, tokens_used, cost):
        usage = self.session_usage.setdefault(session_id, {"tokens": 0, "cost": 0})
        usage["tokens"] += tokens_used
        usage["cost"] += cost
        
        if usage["tokens"] > self.limits["tokens"]:
            raise BudgetExceededError(f"Token budget exceeded: {usage['tokens']}/{self.limits['tokens']}")
        if usage["cost"] > self.limits["cost"]:
            raise BudgetExceededError(f"Cost budget exceeded: €{usage['cost']:.4f}/{self.limits['cost']}")

Debugging Tools for AI Agents

LangSmith

Comprehensive tracing and evaluation platform:

import langsmith

# Automatic tracing (with LangChain)
@langsmith.trace
async def my_agent(input_text):
    # All LLM calls, tool calls, and chain executions are automatically traced
    result = await agent.invoke(input_text)
    return result

# Manual tracing
with langsmith.trace_context("custom_agent_run") as run:
    run.add_metadata({"user_id": "user_123", "session": "sess_456"})
    result = await agent.process(input_text)
    run.add_output(result)

Langfuse

Open-source alternative to LangSmith:

from langfuse import Langfuse

langfuse = Langfuse()

# Create a trace
trace = langfuse.trace(name="customer_service_agent")

# Add generations
generation = trace.generation(
    name="intent_classification",
    model="glm-5.0",
    input=user_message,
    output=classification,
    usage={"input": 150, "output": 20}
)

# Add spans for tool calls
span = trace.span(name="crm_lookup", input={"customer_id": "123"})

OpenTelemetry for AI

Standard observability with AI-specific semantic conventions:

from opentelemetry import trace

tracer = trace.get_tracer("ai-agent")

async def agent_with_tracing(input_text):
    with tracer.start_as_current_span("agent_execution") as span:
        span.set_attribute("agent.name", "customer_service")
        span.set_attribute("agent.session_id", session_id)
        
        with tracer.start_as_current_span("llm_call") as llm_span:
            llm_span.set_attribute("llm.model", "glm-5.0")
            llm_span.set_attribute("llm.input_tokens", 4500)
            response = await llm.complete(input_text)
            llm_span.set_attribute("llm.output_tokens", 120)
        
        with tracer.start_as_current_span("tool_call") as tool_span:
            tool_span.set_attribute("tool.name", "crm_lookup")
            result = await crm.lookup(customer_id)

Agent-Specific Debuggers

MCP Inspector — Built-in tool for inspecting MCP server communication:

npx @modelcontextprotocol/inspector node server.js

Provides a web UI showing all tool registrations, parameter schemas, and live call traces.

Promptfoo — Debug prompt issues by running test cases:

npx promptfoo eval --prompts prompts/customer_service.txt \
  --tests tests/customer_service.yaml \
  --providers openai:gpt-4o,zai:glm-5.0

Debugging Workflow

Step 1: Reproduce

Get the exact input, context, and model version that caused the issue. Without reproduction, you're guessing.

Step 2: Isolate

Determine which component failed:

Was the input parsed correctly?     → Input validation
Was the system prompt correct?       → Prompt engineering
Did the model reason correctly?      → Model capability
Was the right tool called?           → Tool routing
Were tool params correct?            → Schema validation
Was the tool response handled?       → Response parsing
Was the final output formatted?      → Output formatting

Step 3: Inspect the Trace

Walk through each step in the execution trace. Look for:

• Unexpected tool calls
• Missing context
• Confidence scores below thresholds
• Error messages that were silently handled
• Token counts higher than expected

Step 4: Hypothesize and Test

Form a hypothesis, make the fix, and run the regression suite:

npm run test:regression -- --filter=affected_tests

Step 5: Document

Create a test case from the bug:

def test_bug_142_email_not_sent_when_drafted():
    """Regression test: Agent should not call send_email when user says 'draft'"""
    result = await agent.process("Draft an email to John about the project update")
    
    tools_called = [c.tool_name for c in result.tool_calls]
    assert "send_email" not in tools_called
    assert "draft_email" in tools_called

Production Debugging Strategies

Feature Flags for Debugging

FLAGS = {
    "verbose_logging": False,      # Enable full prompt/response logging
    "trace_all_calls": False,      # Trace every tool call
    "debug_mode": False,           # Return debug info in responses
}

# Toggle in production for specific sessions
if session_id in DEBUG_SESSIONS:
    FLAGS["verbose_logging"] = True
    FLAGS["trace_all_calls"] = True

Sampling

Don't trace every request in production (too expensive). Trace 1-5% for debugging:

import random

if random.random() < 0.02:  # 2% sample
    trace = create_full_trace()
else:
    trace = create_minimal_trace()  # Just input/output/duration

Time-Travel Debugging

Store enough state to replay any production request:

async def handle_request(request):
    # Store full request context
    context = {
        "request": request,
        "agent_state": agent.export_state(),
        "model_versions": get_model_versions(),
        "timestamp": datetime.utcnow()
    }
    
    # Store for 7 days
    await redis.setex(
        f"replay:{request.session_id}",
        604800,  # 7 days
        json.dumps(context, default=str)
    )

Conclusion

AI agent debugging requires a systematic approach built on comprehensive tracing, structured analysis, and the right tools. The investment in observability pays for itself — every hour spent building debugging infrastructure saves 5-10 hours of production firefighting.

Start with full execution tracing. Add Langfuse or LangSmith for visualization. Build a regression test suite from every bug you find. And always, always reproduce before fixing — guessing at AI bugs wastes more time than any other debugging mistake.