Tracing and Debugging AI Agent Tool Calls

0) Problem Restatement

An AI agent calls external tools (search, databases, APIs). Sometimes a call fails, but the error the user sees doesn't show where it failed:

• while preparing the request (the model produced bad arguments),
• while reaching the tool (network, DNS, auth),
• while the tool was executing (the tool's own error or timeout),
• while parsing the response (unexpected format),
• or during retries (the first error hidden by the last one).

Design tracing so engineers can see exactly which stage failed and why. TikTok asked this.

1) Requirements

• Every agent run produces a trace: model calls, tool calls, and their stages as nested spans.
• Each span records the start and end time, status, error type and message, and (safely) inputs and outputs.
• Retries are linked to the same logical tool call, with each attempt visible.
• Traces are searchable ("all failed search_docs calls in the last hour with PARSE errors").
• Sensitive data is redacted, and payload sizes are limited.
• Low overhead on the agent.

2) Trace Structure

Trace: agent_run (trace_id=abc)
 ├─ span: llm_call #1                         (tokens, latency)
 ├─ span: tool_call search_docs  (call_id=7)
 │   ├─ span: prepare_args      ok     (validated JSON schema)
 │   ├─ span: attempt 1
 │   │   ├─ span: send          ok     (DNS 2ms, connect 10ms)
 │   │   ├─ span: execute       ERROR  timeout after 5000ms   (from tool's own span)
 │   ├─ span: attempt 2
 │   │   ├─ span: send          ok
 │   │   ├─ span: execute       ok     (812ms)
 │   │   └─ span: parse_response ERROR  missing field "results"
 │   └─ result: FAILED (stage=parse, attempts=2)
 └─ span: llm_call #2 ...

Each span has trace_id, span_id, parent_span_id, name, stage, status, error.type, attributes (tool name, attempt number, HTTP status, sizes).

3) Architecture

flowchart LR
    AG["Agent runtime - SDK creates spans"] -->|"trace context header"| TOOL["Tool service - continues the trace"]
    AG --> COL["Collector - redact, sample, batch"]
    TOOL --> COL
    COL --> K[("Kafka")]
    K --> ST[("Trace store - columnar / search")]
    K --> MET["Metrics: failure rate per tool and stage"]
    MET --> AL["Alerts"]
    UI["Trace viewer UI"] --> ST

• Instrumentation SDK (built on OpenTelemetry): wraps each tool call and automatically creates stage spans. Engineers don't have to remember to add them.
• Context propagation: the SDK sends traceparent headers to tools. Tools that are instrumented add their own internal spans (e.g., the DB query inside), so "execute" failures show the tool's side too.
• Collector: redacts secrets and personal data (API keys, emails), truncates payloads (e.g., keeps the first 4 KB plus a hash of the full body), and batches exports.
• Sampling: keep 100% of traces with errors and a small percentage of successful ones (tail-based sampling: decide after the run finishes).
• Trace store: a columnar or search store for queries by tool, stage, error type and time.

4) Classifying Errors by Stage

Make the error type explicit, so dashboards can group them:

• PREPARE: schema validation failed (the model gave wrong arguments). Fix the prompt or tool description.
• NETWORK: DNS, connect, TLS, or 5xx from a proxy. An infrastructure issue.
• TOOL_ERROR: the tool returned an error or timed out. The tool owner should look.
• PARSE: the response didn't match the expected schema. Contract drift.
• RETRY_EXHAUSTED: the final summary, which links to all attempts.

5) Using the Data

• Dashboards and alerts: failure rate per tool and stage. Alert if search_docs PARSE errors jump after a deploy.
• Debugging: open a trace to see the exact arguments, timings and responses (redacted) per attempt.
• Replay: re-run a failed tool call with the same inputs in a sandbox.
• Evals: turn common failures into test cases for prompts and tools.

6) Trade-offs & Alternatives

7) Wrap-Up

Wrap every tool call in an SDK that emits a span per stage (prepare, send, execute, parse) and per retry attempt under one logical call, propagate the trace context into the tools so their own spans join the trace, and classify errors by stage. A collector redacts and truncates payloads and keeps all error traces with tail-based sampling. The trace store powers a viewer, per-tool and per-stage failure metrics with alerts, replays and eval datasets.