Skip to main content
The durable_agent/temporal example is an interactive Temporal lab, not a minimal hello-world. It runs the agent client and Temporal worker in separate processes, streams events over the workflow’s event stream, and includes a REPL so you can deliberately break things — kill workers, crash processes, miss approvals — and observe how Temporal and the SDK behave. Use it after Agent Worker when you need to feel durability guarantees. For the single-process Restate lab, see Durable Agent (Restate). Source: examples/durable_agent/temporal/
For hands-on step-by-step exercises, follow examples/durable_agent/temporal/README.md.

​
Architecture

Both processes share the same agent options from durable_agent/temporal/opts/opts.go so the SDK fingerprint matches — name, LLM client, tools, approval policy, etc. The agent uses Stream. The REPL prints event types as they arrive:
  • TEXT_MESSAGE_CONTENT — token deltas
  • Tool lifecycle events
  • CUSTOM — tool and delegation approvals (respond with AgentStream.Approve in the sample)
  • RUN_FINISHED / errors
Events are delivered by the runtime as a durable ordered stream. Each event carries a monotonic offset. The example persists runID and the last event offset to /tmp/durable_agent_temporal_runstate.json on every event. Kill the agent mid-stream with pkill -SIGKILL and restart — the REPL detects the saved state, asks whether to reconnect, and resumes via GetAgentStream + WithOffset from the exact offset. See the README and Durable Execution for the full protocol.

​
Before you start

Run all commands from the examples/ directory. Complete Configuration first (.env, API key, env load order). Add to examples/.env (defaults match local Docker compose; override for Temporal Cloud or a custom cluster):
Both agent and worker read the same values. Override any TEMPORAL_* variable for your environment. Start Temporal:
With the local Docker dev server, open http://localhost:8233 (Temporal Web UI) while the worker and agent run. Inspect workflows and activities in namespace default. Helpful when stepping through the scenarios below.
Stop Temporal when finished:
Minimal smoke test (from examples/): Terminal 1 — worker:
Terminal 2 — agent REPL:
One-shot (no REPL):
Type exit, quit, or bye to leave the REPL. For hands-on scenarios, see the README.

​
Durability scenarios

Each scenario below lists what to try and what you should learn. Step-by-step instructions: README.

​
1. Baseline — worker first

Start worker, then agent, send Hello from remote agent!. Learn: End-to-end remote path works — stream completes, REPL ready for the next turn.

​
2. Agent without worker (intentional timeout)

Start agent with no worker, send a prompt. With no pollers on the task queue, the SDK checks Temporal before starting the stream (~15 seconds). Typical error:
If a worker was running recently, Temporal may still list stale pollers briefly — the stream can start, the workflow queues, and you hit the run timeout instead (3 minutes): [error] request timed out (approval expired or deadline exceeded). Then start the worker and resend the same prompt — run succeeds. Learn: Interactive mode fails fast when no worker is polling — better than hanging forever. After the worker is up, resend the prompt and the run succeeds.

​
3. Kill worker between runs

3a — Graceful stop: Complete a run, Ctrl+C worker, restart worker, send another prompt. 3b — Crash: Complete a run, kill the worker (pkill -f 'go run ./durable_agent/temporal/worker|go-build/.*/worker'), restart worker, send another prompt. Learn: Planned or crash worker shutdown between runs does not lose completed work. Temporal history already recorded finished activities; the restarted worker does not re-execute them.

​
4. Kill worker during an LLM call (mid-stream)

Send a long prompt (e.g. 7-day Japan travel plan). While tokens stream, stop the worker — use pkill mid-stream (graceful Ctrl+C may wait for the in-flight activity to finish; see README scenario 4). 4a — Worker stays down: Stream pauses silently until the run timeout (3 minutes in this example), then [error] request timed out (approval expired or deadline exceeded). Restart worker to resume normal operation. 4b — Worker restarts before timeout: Stop worker mid-stream, restart within 3 minutes. Temporal reschedules the in-flight LLM activity; stream resumes on the same agent process without resending the prompt. Learn: Core durability — the run continues when the client still lives. Caveat: on worker restart the LLM call retries from the start; the SDK discards stale tokens so subscribers receive clean delivery, but there may be a brief gap before new tokens arrive. Final conversation content is one complete result.

​
5. Agent restart or crash

5a — Graceful agent exit: Finish a run, type bye, restart agent, new prompt works against same worker. 5b — Agent crash between runs: Kill the agent (pkill -f 'go run ./durable_agent/temporal/agent|go-build/.*/agent') after a completed run; worker keeps polling; new agent process connects and runs immediately. 5c — Agent crash mid-LLM call: Kill the agent while tokens stream (pkill -f 'go run ./durable_agent/temporal/agent|go-build/.*/agent'); worker completes the run in Temporal even though the user saw nothing. Restart agent — follow-up prompt has no conversation memory in this example (no WithConversation wired). Learn: Worker survives agent death; work can finish server-side. This example does not persist chat history — a follow-up like “What was the first destination?” gets no context. For production UIs where users expect continuity after reconnect, see Agent Chat (Postgres + SSE + durable workflows).

​
6. Two workers, one queue

Run two durable_agent/temporal/worker processes on the same task queue. Send prompts; stop one worker after a reply or during a long prompt while the other keeps polling; send another prompt. Learn: Temporal load-balances across workers. Losing one worker mid-session does not drop an in-flight run if another worker polls the same queue.

​
7. Task queue mismatch

Point the worker at a different queue via env while the agent keeps the default from .env (see README — stop workers on the default queue first):
Send a prompt from the agent. Outcome depends on timing (same as scenario 2):
  • Typical: [error] failed to start stream: no workers available on task queue agent-sdk-go-durable-agent_remote-worker (~15s poller check).
  • Stale pollers on the agent queue: stream may start, then [error] request timed out (approval expired or deadline exceeded) after 3 minutes.
Learn: Misconfiguration surfaces clearly — not silent corruption. Restart both processes with matching TEMPORAL_TASKQUEUE (or remove the override). Tip: Under AgentModeAutonomous, the immediate worker check is skipped — mismatch may queue until timeout instead of failing fast. Always align task queue (and shared opts) between agent and worker before deploy.

​
Approvals in this example

agent/main.go handles CUSTOM approval events — parse with ParseCustomEventApproval / ParseCustomEventDelegation and call AgentStream.Approve with the token (same pattern as Approvals). Shared opts in opts/opts.go do not register tools by default, so simple prompts usually skip this path. If you add tools that require approval and ignore prompts, tools are skipped with a clear message rather than hanging indefinitely.

​
What this example does not cover

​
Learn more

Durable Execution

Server-side durability and client-side stream recovery

Distributed Execution

Production split-process pattern

Reconnect Example

Single-process GetAgentStream demo

Temporal Runtime

Architecture and event delivery model