AI steps

step.ai makes a model call a durable step. Like step.run, each call takes a stable id, records its result under that id, and returns the saved result on replay instead of calling the model again. So a retry after a crash never re-spends on work that already completed.

New to AI steps? The AI quickstart walks you from a first generate call to spend landing in the console.

Duraton stores the AI metadata (model, token counts, latency) as an opaque journal block. It never parses it and never stores your prompt, the response text, or your API key - those stay in your runner.

const { output } = await ctx.step.ai.generate<Triage>("classify", {
  model: "claude-opus-4-8",
  prompt: `Classify this ticket: ${subject}`,
  output: triageSchema,
});

The step.ai API

step.ai.generate

Make one model call as a durable step. The built-in provider is Anthropic; the request is validated, sent, and the result memoized under id.

const result = await ctx.step.ai.generate("draft-reply", {
  model: "claude-opus-4-8",
  prompt: `Write a one-line apology for ticket ${ticketId}.`,
});
// result.text, result.model, result.usage.inputTokens, result.usage.outputTokens

generate returns a StructuredResult<T>:

Structured output and durable re-ask

Pass output (a JSON Schema) to constrain the model and get a typed, validated value back on result.output. If the response fails to parse or validate, generate re-prompts with the validation error - each re-ask is its own memoized step, so the retry survives a crash and never repeats a committed attempt. Add validate for semantic rules the schema can't express.

const { output } = await ctx.step.ai.generate<{ category: string; priority: string }>("triage", {
  model: "claude-opus-4-8",
  prompt: `Triage: ${subject}`,
  output: {
    type: "object",
    properties: { category: { type: "string" }, priority: { type: "string" } },
    required: ["category", "priority"],
  },
  reask: 2,
  validate: (v) => (["low", "normal", "high"].includes((v as { priority: string }).priority) ? undefined : "priority out of range"),
});

Streaming

Pass stream: true to feed the model's tokens to a live viewer as they arrive. Each delta is appended to the run's durable timeline as an ai_chunk frame, so a viewer sees the text build in real time and a late or reconnecting viewer replays it from token 0. The return value is unchanged - result.text is still the complete response, memoized on replay - so streaming affects only what a viewer sees while the step runs.

const result = await ctx.step.ai.generate("summarize-thread", {
  model: "claude-opus-4-8",
  prompt: `Summarize this thread:\n\n${thread}`,
  stream: true,
});
// result.text is the full summary; deltas streamed live on the way there.

Live deltas need a live channel to Duraton, which the connect runner transport provides; over an HTTP serve runner the call falls back to a plain generate (identical result, only the final text recorded). See the Streaming concept for the timeline frames, resumability, and the useStream React hook.

Fallback chains

Pass fallback - an ordered list of backup models - to keep a call resilient when a model is rate-limited or down. The primary model is tried first; if it fails with a retryable error (429, a 5xx, or a timeout), the call advances to the next candidate, and the first one to return wins. Its result is the step's durable output, so a caller never sees the failover.

const answer = await ctx.step.ai.generate("answer", {
  model: "claude-opus-4-8",
  prompt: question,
  fallback: [{ model: "claude-sonnet-4-6" }, { model: "claude-haiku-4-5" }],
});

Each candidate is { model, provider? }; provider defaults to the call's provider, so a chain can span providers once you have more than one adapter configured. The step's journal records the outcome:

The console renders this as a chain pill on the AI step, and it rides the opaque journal so an agent reading the run over MCP sees the same chain / used / reason.

Inference cache

Set cache to reuse the result of an identical earlier call instead of paying for it again. On a hit the provider is never called, so the step commits with zero spend - the cache is the one control that reduces spend rather than capping it, and a cached call counts nothing against cap / budget / tokenThrottle. Where step memoization already makes a replay free, the cache makes an identical call in a different run free too.

const answer = await ctx.step.ai.generate("answer", {
  model: "claude-opus-4-8",
  prompt: question,
  temperature: 0,   // required: caching engages only for a deterministic call
  cache: true,      // or { ttlMs, seed }
});

The key is an exact match over the runner's app, the model, the prompt, and every output-affecting parameter, so no entry ever crosses a project boundary and a different config never returns a stale answer. The step's journal records the outcome:

The console renders this as a cache pill on the AI step, and it rides the opaque journal so an agent reading the run over MCP sees the same hit / key / ageMs.

step.ai.wrap

Makes a model call you already write yourself - through the OpenAI SDK, the Anthropic SDK, the Vercel AI SDK, or anything else - a durable step, with no other change to the call site. wrap returns your function's value unchanged; when it recognizes the response shape it enriches the journal with the model and token counts and records which library it wrapped.

import OpenAI from "openai";
const openai = new OpenAI();

const completion = await ctx.step.ai.wrap("classify", () =>
  openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: subject }],
  }),
);

Recognized shapes: the Anthropic SDK, the OpenAI SDK, and the Vercel AI SDK. An unrecognized value still becomes a durable wrap step - you just get less metadata on the journal.

step.ai.embed

Turn a list of inputs into vectors, one durable batch at a time. Anthropic has no embeddings API, so you supply the embedding call (embed); Duraton owns the batching and per-batch checkpointing. If a batch fails, only that batch re-runs on retry - committed batches are not re-embedded.

const { vectors } = await ctx.step.ai.embed("embed-kb", {
  model: "voyage-3",
  inputs: ["duplicate charge policy", "annual plan refunds", "refund SLA"],
  embed: (batch) => voyage.embed(batch),
  batchSize: 2,
});

embed returns { vectors } - one vector per input, in input order.

step.ai.loop

A durable agent loop. Each turn is your own model call (bring-your-own, normalized to tool calls or a final answer); the loop executes the tools the turn requested and feeds the results into the next turn, until the model returns a final answer, stop fires, or maxIterations is reached.

Every turn is a durable step (id:iter:N) and every tool call is a durable step (id:iter:N:tool:<callId>), so an agent that crashes mid-run resumes at the last committed turn.

Writing turn yourself is the low-level path. To declare a model, instructions and tools and have the turn composed for you, use the agent kit - it composes this loop rather than replacing it, so everything below still applies.

const agent = await ctx.step.ai.loop<{ resolution: string }>("agent", {
  prompt: `Resolve the ticket about: ${subject}`,
  maxIterations: 6,
  tools: {
    "search-kb": { handler: (input) => searchKb(input) },
    "lookup-order": { workflow: "orders.lookup", app: "orders" },
  },
  turn: (ctx, iteration) => callModel(ctx.prompt, ctx.history, iteration),
});
// agent.final, agent.iterations, agent.stopReason

Your turn returns a LoopTurn - either tool calls to run, or a final answer:

Which loop composed a turn

Every turn's journal carries a loopVersion alongside the model and the token counts. The model says what answered; this says what asked:

{ "kind": "loop", "iteration": 0, "loopVersion": "typescript.1", "model": "claude-sonnet-5" }

It is written when the turn is composed and never rewritten, so replaying a year-old run shows the version that produced each turn rather than the version replaying it. Without it a replay quietly mixes old journal entries with new loop behaviour: the run replays, the numbers look reasonable, and the conclusion drawn from them is wrong.

Each SDK versions its own loop - typescript.1, go.1, python.1 - because each implements the loop separately. The language is part of the value so an entry read later needs nothing but itself to be understood. Only turns carry it: a generate or an embed was not composed by the loop.

A tool is either a handler (a local function) or a workflow (another Duraton workflow, called as a linked child run):

Gating a tool on a human

A tool marked requiresApproval does not run until someone decides on it. The run parks in needs_attention holding no worker, exactly as step.approval does - it is the same gate, raised for you:

tools: {
  "issue-refund": { requiresApproval: true, handler: (input) => issueRefund(input) },
}

Three things follow, and each is a deliberate choice:

A tool's approval says more about that gate and can decide whether it is raised at all, and the loop's own approval is the default for every tool that has not answered for itself. Both are the same ApprovalRule: the environment a condition reads is on Approvals, and the order the two levels resolve in is on the agent kit.

maxIterations bounds how many turns a loop may run; a per-run budget cap bounds how much it may spend. When a run reaches its cap the loop halts before its next turn's model call and the run fails with a BudgetError - the committed turns stay, and the halted turn is the loop's last (failed) iteration.

ctx.history gives each turn the prior turns' toolCalls and toolResults, so your model call can see what it has already tried. loop returns:

Providers

step.ai.generate resolves its provider name to an AIProvider adapter through a port, so you can supply your own instead of the built-in registry. Pass resolveProvider to connect or serve and every generate call in that runner goes through it - the call sites are unchanged. The same resolver is also reachable directly as ctx.resolveProvider, which is what step.ai.loop (and the agent kit's agent()) resolves its own model call through, since a loop's turn only ever sees (ctx, iteration) and has no opts.provider-shaped call site of its own to inject into.

import { connect } from "@duraton/sdk";
import { type AIProvider, createAnthropicProvider, getProvider } from "@duraton/sdk/ai";

const recording: AIProvider = {
  name: "anthropic",
  generate: (req) => fixtures[req.prompt] ?? getProvider("anthropic").generate(req),
};

connect({
  app: "support-app",
  workflows,
  resolveProvider: (name) => (name === "anthropic" ? recording : getProvider(name)),
});

An AIProvider implements generate(req) and, optionally, stream(req, onDelta) (an adapter without it falls back to generate, so stream: true still returns the right text) and classifyError(err) (which decides whether a failure is retryable, and so whether a fallback chain advances - an unclassified error is treated as terminal). It may also declare capabilities - see Conversations.

Using any model provider

The aisdk provider delegates to the Vercel AI SDK, so one adapter reaches every provider the AI SDK supports - OpenAI, Google, Mistral, Bedrock, Groq and the rest - without Duraton shipping an adapter per vendor. Install ai alongside the provider package you want:

npm i ai @ai-sdk/openai

Point resolveModel at that package and wire it through resolveProvider:

import { openai } from "@ai-sdk/openai";
import { connect, createAisdkProvider, getProvider } from "@duraton/sdk";

await connect({
  workflows,
  resolveProvider: (name) =>
    name === "aisdk"
      ? createAisdkProvider({ resolveModel: (model) => openai(model) })
      : getProvider(name),
});

Then ask for it per call:

const answer = await ctx.step.ai.generate("draft", {
  provider: "aisdk",
  model: "gpt-5.1",
  prompt: "Summarise this ticket.",
});

Without resolveModel the model string is passed to the AI SDK as-is, which resolves it through its global provider - the Vercel AI Gateway, requiring AI_GATEWAY_API_KEY. Supply resolveModel whenever you want to call a provider directly rather than route through the gateway.

Two behaviours are worth knowing:

• apiKey on the call is ignored by this adapter. The AI SDK carries credentials on the model, so the key belongs to whatever resolveModel returns (openai({ apiKey })).
• Duraton still owns the loop and the retries. The adapter makes exactly one model call per step and disables the AI SDK's own retries, so a rate limit checkpoints and reschedules durably instead of blocking a worker. Tools are declared to the AI SDK without an executor, so every tool call comes back to step.ai.loop and stays a durable, approvable step.

The anthropic adapter is not deprecated by this. It imports no framework, which is what keeps GenerateRequest from drifting into any one vendor's types.

Tool calling

A GenerateRequest may carry tools: ToolDeclaration[], and a GenerateResult may answer with toolCalls: ToolCall[]. If you write your own adapter, honour both halves:

Tell a tool turn from a final answer by whether toolCalls is present - never by reading stopReason, which stays your provider's own raw string.

Conversations

A multi-turn agent has a conversation, and GenerateRequest.transcript carries it: the exchanges that have already happened, oldest first, with prompt as the task that opened them. A provider's native tool-use protocol has a shape for this - Anthropic answers an assistant tool_use block with a user tool_result block - and the built-in adapter maps the transcript onto it.

Ignoring the field would silently lose the history rather than lose a nicety, so an adapter has to say it reads it:

const recording: AIProvider = {
  name: "anthropic",
  capabilities: ["transcript"],
  generate: (req) => callMyProvider(req.prompt, req.transcript ?? []),
};

An adapter that declares nothing keeps working exactly as it did. The agent kit checks with providerSupports and renders the turns into the prompt as text for anything that has not opted in, so no adapter is broken by the field existing.

Cost

Duraton holds no model price list, so a call's cost is absent unless your runner supplies it. Pass resolveCost - the CostSource port - and each step.ai call is priced from the axes the journal already holds. Supplying it is what makes cap: { maxCost } and budget: { maxCost } bite; maxTokens needs nothing, because tokens are metered from every call.

import type { CostSource } from "@duraton/sdk/ai";

const PRICES: Record<string, { in: number; out: number }> = {
  "claude-opus-4-8": { in: 5 / 1_000_000, out: 25 / 1_000_000 },
};

const resolveCost: CostSource = ({ model, tokensIn = 0, tokensOut = 0 }) => {
  const p = model ? PRICES[model] : undefined;
  return p ? tokensIn * p.in + tokensOut * p.out : undefined;   // undefined leaves cost absent
};

connect({ app: "support-app", workflows, resolveCost });

Returning undefined leaves the cost absent - Duraton never fabricates a zero - and a call that already carries an explicit cost is left untouched.

Cache store

The inference cache is backed by the AICache port, so the store is swappable. The default is createMemoryCache(): a process-local Map with per-entry TTL and LRU eviction, bounded at 1000 entries. Pass cache to connect or serve to swap it - for a store shared across runner processes, say.

import { createMemoryCache } from "@duraton/sdk/ai";

connect({
  app: "support-app",
  workflows,
  cache: createMemoryCache({ maxEntries: 10_000 }),
});

The store is only ever consulted for a call that opted in with cache - its mere presence changes nothing. The cached completion is held runner-side: Duraton's journal records only the cache metadata (hit, key, ageMs), never the payload. A store you share across processes must seed its keys deliberately, since the default seed (the runner's app) assumes the process boundary isolates it.

Workflows as tools

Hand any workflow to a loop as a tool and the model can drive it: the tool call becomes a linked child run, the model's tool input is the child's trigger data, and the child's result is the tool result. The child is a full durable run of its own - it can retry, sleep, and call further workflows - and shows up linked to the parent in the inspector. This is how the orchestrator-workers pattern maps onto Duraton.

Durable steps and replay

Because every step.ai call is a durable step, the same replay rules as regular steps apply: keep turn, validate, and stop deterministic (a pure function of their inputs), since they run again on replay while the memoized model results do not. Model calls happen exactly once per committed step; everything around them must be replay-safe.

For the concepts behind durable AI steps and end-to-end recipes, see AI agents.