Skip to main content
The SDK has two observability layers:
  1. OpenTelemetry export — distributed traces, metrics, and structured logs pushed to any OTLP-compatible backend (Jaeger, Grafana, Datadog, etc.). Configured once at agent creation.
  2. Run-level telemetry — AgentTelemetry on every AgentRunResult. LLM round counts, tool breakdowns, and storage operation summaries returned inline with each result. No configuration required.

​
Enable OpenTelemetry

Pass WithObservabilityConfig to wire traces, metrics, and logs in one block: 
a, err := agent.NewAgent(
    agent.WithLLMClient(llmClient),
    agent.WithObservabilityConfig(&agent.ObservabilityConfig{
        Endpoint: "localhost:4317",        // gRPC OTLP endpoint
        Protocol: agent.OTLPProtocolGRPC,  // or OTLPProtocolHTTP
        Insecure: true,                    // dev only — omit for TLS
    }),
    agent.WithLogLevel("info"),
)
defer a.Close() // flushes all OTLP exporters — required
Use the same WithObservabilityConfig on NewAgent and NewAgentWorker. LLM calls and tool executions run on the worker process — without matching config those spans and metrics are dropped. Opt out of individual signals: 
agent.WithObservabilityConfig(&agent.ObservabilityConfig{
    Endpoint:       "localhost:4317",
    DisableTraces:  false, // default — enabled
    DisableMetrics: false, // default — enabled
    DisableLogs:    false, // default — enabled
})
See Tracing, Metrics, and Logs for per-signal details and collector configuration.

​
Run-level telemetry (AgentTelemetry)

Every completed run populates AgentTelemetry on AgentRunResult. This is run-level behavioral data returned with the result — separate from OTLP export. Use it for app-side logging, dashboards, and eval harness assertions.

​
Structure

AgentTelemetry has three sections:
SectionTracks
RunOrchestration lifecycle — timing, LLM round count, finish reason
ToolsTool invocation totals, failures, per-tool breakdown
StorageRetriever searches and memory recall/store operations
Fields are zero when the corresponding feature is not configured (no retriever → retriever counts are 0; no memory → memory counts are 0).

​
Run telemetry

FieldDescription
StartedAtRun start time
CompletedAtRun completion time
TotalLLMCallsLLM calls across all tool rounds
FinishReasoncomplete or max_iterations
max_iterations means the run hit WithMaxIterations before producing a final answer.

​
Tool telemetry

FieldDescription
TotalCallsTotal tool invocations
FailedCallsTools that returned an error — excludes approval-denied and unauthorized cases
BreakdownPer-tool invocation counts — key is tool name
FailedBreakdownPer-tool failure counts
Covers native tools, MCP tools, retriever tools, and sub-agent delegation tools.

​
Storage telemetry

FieldDescription
TotalRetrieverSearchesTotal RAG searches
FailedRetrieverSearchesFailed retriever searches
PrefetchSearchesSearches from prefetch or hybrid mode
AgenticSearchesSearches from agentic or hybrid mode (LLM-initiated)
TotalMemoryRecallsMemory load operations before the run
FailedMemoryRecallsFailed recall operations
TotalMemoryStoresMemory persist operations
FailedMemoryStoresFailed store operations
See Retrieval and Memory for when each counter increments.

​
Run

result, err := a.Run(ctx, "What's the weather in Tokyo?", nil)
if err != nil {
    return err
}

t := result.Telemetry
fmt.Printf("llm_calls=%d finish=%s\n", t.Run.TotalLLMCalls, t.Run.FinishReason)
fmt.Printf("tool_calls=%d failed=%d\n", t.Tools.TotalCalls, t.Tools.FailedCalls)
fmt.Printf("retriever=%d prefetch=%d agentic=%d\n",
    t.Storage.TotalRetrieverSearches,
    t.Storage.PrefetchSearches,
    t.Storage.AgenticSearches)
fmt.Printf("memory recall=%d store=%d\n",
    t.Storage.TotalMemoryRecalls,
    t.Storage.TotalMemoryStores)
Example: Simple Agent · Observability.

​
Stream

Telemetry is on Result.Telemetry inside the AgentEventTypeRunFinished event: 
for ev := range eventCh {
    if ev.Type() != agent.AgentEventTypeRunFinished {
        continue
    }
    finished, ok := ev.(*agent.AgentRunFinishedEvent)
    if !ok || finished.Result == nil || finished.Result.Telemetry == nil {
        continue
    }
    fmt.Println(finished.Result.Telemetry.Run.TotalLLMCalls)
}

​
RunAsync

Same as Run — telemetry is on the final AgentRunAsyncResult.Result.Telemetry when the run completes.

​
Token usage vs telemetry

DataWherePurpose
TelemetryResult.TelemetryRun behavior — LLM round count, tool breakdown, storage ops
Token usageResult.LLMUsageAggregate token counts for cost and quota tracking
See Token Usage.

​
Eval harness

Telemetry fields are designed for eval assertions — total LLM calls, tool breakdown, finish reason, and storage counts. See Eval Harness.

​
Example

Telemetry

SHOW_TELEMETRY on run results

Metrics

OTLP counters and histograms

Tracing

Distributed spans per LLM and tool call

Token Usage

LLMUsage on run results

Observability Example

Runnable demo with telemetry footer