Good luck if your model takes longer to answer than your function is allowed to run. Or if something fails and you want to retry it. Or if you want to spawn a sub-agent that does its own thinking. Or if you want to close the browser and come back days later.

A stateful machine is a much better fit:

A real Linux machine. Install what you want, run any CLI, pick the CPU and RAM.

Durable compute, durable streams. No timeout on a turn. A conversation you can close and come back to days later.

Fast first turns. The first LLM call runs in your own warm server while the agent boots alongside it, so durable doesn't mean slow.

Waiting costs nothing. An agent can stop, ask a person to approve something, and sit there for days without running up a bill.

Tracing and metrics built in. Every turn is a span, and there’s an AI metrics dashboard for cost, tokens and latency.

And it uses the AI SDK you probably already use.

A real Linux machine

The computer behind a conversation is an ordinary Linux machine, not a restricted runtime. Install what you need. Shell out to ffmpeg, drive a headless browser, run a CLI that expects a real filesystem and a real process tree.

import { tool } from "ai";

import { execFile } from "node:child_process";

import { promisify } from "node:util";

import { z } from "zod";

const exec = promisify(execFile);

export const transcode = tool({

description: "Transcode a video to MP4.",

inputSchema: z.object({ input: z.string(), output: z.string() }),

execute: async ({ input, output }) => {

await exec("ffmpeg", ["-i", input, "-c:v", "libx264", output]);

return { output };

},

});

You pick how big it is. Machine presets run from micro at 0.25 vCPU upwards, each with dedicated CPU, memory and disk. Set per agent, overridable per conversation.

Durable compute, durable streams

A turn has no timeout. It runs as long as the work takes, so a slow tool, a long chain of them, or a sub-agent doing its own thinking is just work. You don't have to chop anything into chunks that fit a limit, and you don't need a queue for the slow parts. In production one agent run in twenty lasts longer than 36 minutes, well past where a request would have been cut off.

Memory carries across a sleep. Whatever you kept in a variable on turn three is still sitting there on turn twenty tomorrow, so an expensive lookup you already did stays done.

const embeddings = new Map<string, number[]>();

export const myChat = chat.agent({

id: "my-chat",

run: async ({ messages, signal }) => {

const key = messages.at(-1)!.id;

const cached = embeddings.get(key) ?? (await embed(key));

embeddings.set(key, cached);

return streamText({ model, messages, abortSignal: signal });

},

});

If the machine crashes you get a new one, and the conversation comes back with it because that part is written down. The heap doesn't: your Map starts empty again. That's the same rule as any long-running server you've operated, so put anything you can't afford to lose in a database and keep memory for speed. What's different here is that memory now lasts the whole conversation instead of a single request.

The stream is durable too. A conversation is a session, and the session outlives the process serving it: runs are the compute, the session is the identity. Refresh mid-response and the stream replays from where your browser stopped reading, without re-running the model. Close the browser and come back days later and the conversation is still there.

That makes the failure cases boring. If a run is killed or runs out of memory, the next message boots a new one with the conversation restored. Deploys don't interrupt anything: a run stays on the version it started on, and moving a conversation onto new code is a call you make yourself with chat.requestUpgrade().

Fast first turns

Durable doesn't have to mean slow. Head Start is optional, and it's there for when you want the first token as fast as a plain endpoint would give it to you. It runs the first LLM call inside your own warm server while the agent boots in parallel, then hands the conversation over mid-turn for the tool calls and everything after. The user sees one continuous response and never notices the seam.

import { chat } from "@trigger.dev/sdk/chat-server";

import { streamText } from "ai";

import { anthropic } from "@ai-sdk/anthropic";

import { headStartTools } from "@/lib/chat-tools/schemas";

export const chatHandler = chat.headStart({

agentId: "my-chat",

run: async ({ chat: helper }) =>

streamText({

...helper.toStreamTextOptions({ tools: headStartTools }),

model: anthropic("claude-sonnet-4-5"),

system: "You are a helpful assistant.",

}),

});

In our tests it roughly halved both time to first token and the length of the whole turn.

It returns a plain Web Fetch handler, so it mounts in Next.js, Hono, SvelteKit, Remix, TanStack Start and the rest with no adapter. And when a first turn is pure text with no tool calls, the agent boots and exits without ever calling a model, so you only pay for what the conversation actually needed.

Because memory is snapshotted rather than replayed, none of this puts determinism constraints on your code. There's no event history to replay, so Date.now(), Math.random() and a fetch in the middle of your loop are all just code. Nothing to make deterministic, no replay log to version.

Waiting costs nothing

A tool with no execute function ends the turn with the call still open.

export const requestApproval = tool({

description: "Ask the user to approve an irreversible action.",

inputSchema: z.object({ action: z.string() }),

// No execute. The turn ends here and the agent suspends.

});

The agent suspends, the person takes as long as they take, and their answer resumes the run. Because it's suspended, you're not charged for the wait, so an approval can sit overnight or over a weekend.

The usual chat controls are here too. Stop a generation mid-stream, steer it between tool calls, or edit, branch and regenerate your way back through an earlier message.

Tracing and metrics built in

Every turn is a span in the dashboard, so you can open a conversation and see the prompts, the responses, the tool calls and how long each one took.

An AI metrics dashboard ships with every project. Total spend, total calls, average time to first chunk and average tokens per second, then cost over time and by model, tokens over time, latency percentiles by model, finish reasons, your most expensive runs, and cost broken down by task and by provider.

Every model gets its own row too, so you can see what each one actually costs you and what prompt caching is saving.

The data behind those charts is available to TRQL, so you can ask your own questions and build your own dashboards. sessions.list gives you every conversation, enough to build an inbox.

Keep using the AI SDK

streamText on the server, useChat on the client. chat.agent slots in underneath as a transport, and the API route between them goes away.

import { useChat } from "@ai-sdk/react";

import { useTriggerChatTransport } from "@trigger.dev/sdk/chat/react";

import type { myChat } from "@/trigger/chat";

import { mintChatAccessToken, startChatSession } from "@/app/actions";

export function Chat() {

const transport = useTriggerChatTransport<typeof myChat>({

task: "my-chat",

accessToken: ({ chatId }) => mintChatAccessToken(chatId),

startSession: ({ chatId, clientData }) =>

startChatSession({ chatId, clientData }),

});

const { messages, sendMessage } = useChat({ transport });

// ... render UI

}

Only the new message goes over the wire. History accumulates on the server, so you're not re-uploading the conversation on every turn.

If you already have a chat app, the migration guide has a prompt you can hand to your coding agent.

There's more

You don't have to stay at the top level. chat.agent() manages the turn loop for you, custom agents hand it back so you can write the loop yourself, and underneath that the raw session primitives let you read and write the conversation's streams directly. Start simple, drop down when you need to.

Long conversations stay affordable: compaction summarizes history when it gets close to the context limit, and prompt caching keeps the stable prefix of every request cheap. You decide when it kicks in and what the summary looks like.

export const myChat = chat.agent({

id: "my-chat",

compaction: {

shouldCompact: ({ totalTokens }) => (totalTokens ?? 0) > 80_000,

summarize: async ({ messages }) => summariseWithHaiku(messages),

},

run: async ({ messages, signal }) => streamText({ model, messages, abortSignal: signal }),

});

Your prompts can live in the codebase too. They're versioned on every deploy, you can override the text or the model from the dashboard without redeploying, and every generation is tracked back to the version that produced it.

Browsers aren't the only client. AgentChat drives a conversation from a task, a script, or another agent, and the MCP server lets you talk to your own agents from Claude Code or Cursor without writing anything at all.

const review = new AgentChat({ agent: "my-chat", id: `review-${prNumber}` });

const summary = await (await review.sendMessage(`Review PR #${prNumber}`)).text();

await review.close();

For the frontend there's also actions, for commands that change state without spending a turn.

When things go wrong, a turn killed by an out-of-memory error retries on a bigger machine without losing the message that caused it, recovery boot restores full context after a crash or a cancel, and version upgrades move a suspended conversation onto new code when you decide it's time. You can also give an agent skills, folders of instructions and scripts it discovers and uses on demand.

Testing, and the rest of Trigger.dev

You can test an agent by driving it through real turns in a unit test. No network, no task runtime, and nothing of ours mocked out from under you.

Because an agent is a Trigger.dev task, all of the existing features work too. Trigger other tasks from a tool call and wait for them, put a chat behind a queue with its own concurrency, batch work, schedule it, or build your own evals as tasks that run agents against a fixture set on every deploy. None of that is a separate product you have to adopt.

Open source, and what it costs

Trigger.dev is Apache 2.0 and chat.agent is part of it. Read the code of how this all works and run the whole thing yourself if you'd like.

A chat agent is billed as compute time when it's actually running. A suspended conversation isn't running, which is why waiting is free.

Try it today

Read the chat.agent docs to get an agent running in three steps, or if you already have a chat app, the migration guide has a prompt you can hand to your coding agent.

I'd love to hear about your use case or ways we can make Chat agent better.

James

You are receiving this email because you opted in to receive updates from Trigger.dev

Trigger.dev, 1111B S Governors Ave STE 6433, Dover, DE 19904

Unsubscribe