Information in this post reflects publicly available sources as of November 1, 2026.
A procurement team gives an agent a job: review 40 vendor contracts, email each vendor to confirm renewal terms, and produce a comparison by morning. Two hours in, at contract 31, the server running the agent restarts for a routine deploy. The run is gone. Someone starts it again, the bill doubles, and 30 vendors receive the same email twice.
Nothing about the model failed. The agent was written as if it were answering a chat message: one process, holding everything in memory, expected to finish before anything else happened. That assumption holds for a ten-second answer. It breaks for work that runs for hours, waits for people, and changes things in the world along the way.
Agents can now do that kind of work, and builders increasingly want them to. This post covers what changes when an agent runs in the background: jobs instead of requests, checkpoints, safe side effects, waiting for people, and knowing when to stop.
The problem: agents outgrew the request
The length of task agents can complete has grown quickly. METR measures this as a time horizon: the length of task, measured by how long a skilled human takes, that an agent completes with a given reliability. Its January 2026 update estimated that this horizon on software tasks has been doubling roughly every four months since 2023. By spring 2026, the best models reached 50% success on tasks that take humans many hours, while their 80% horizons were closer to one to three hours.
That gap matters as much as the headline. Long tasks are now possible, but not reliable, so long runs need to survive failures, report progress, and stop cleanly when they go wrong.
The infrastructure most applications use for AI was built for something else. A web request is expected to finish in seconds, and servers, load balancers, and browsers enforce timeouts.
Processes restart for deploys and scaling. Users close laptops. An agent that needs an hour of work, or a day's wait for a manager's approval, cannot live inside a single request.
How it works
From request to job
The first change is structural. Instead of calling the agent and waiting, the client starts a job and gets back an identifier. A worker picks the job up and runs the agent loop. When the work finishes, the client is notified, or checks the job's status.
Model providers now support this shape directly for single long calls. OpenAI's Responses API, for example, has a background mode for long-running requests such as deep research or high-effort reasoning:
import time
from openai import OpenAI
client = OpenAI()
job = client.responses.create(
model="gpt-5.5",
input="Compare the renewal terms in the attached 40 contracts ...",
reasoning={"effort": "high"},
background=True, # return immediately with a job to poll
)
while job.status in ("queued", "in_progress"):
time.sleep(15)
job = client.responses.retrieve(job.id)
print(job.status) # completed, failed, cancelled, or incomplete
print(job.output_text)The call returns immediately with a response in a queued or in-progress state, and the client polls until it reaches a final state. Background responses can also be streamed and resumed from a cursor if the connection drops, and cancelled explicitly. That solves timeouts for one model call. A multi-step agent that calls tools, sends emails, and waits for approvals needs more: it needs its own progress to survive.
Durable execution
The core technique is durable execution: record the result of every completed step, so that after a crash the run resumes where it stopped rather than starting over. Each model call and each tool call becomes a step whose output is saved before the next one begins.
When the worker restarts, the runtime replays the run. Completed steps return their saved results instantly, without calling the model or the tool again. Only the step that was in progress at the crash runs again. The run continues as if nothing happened, except for the delay.
This also solves a subtler problem. Model calls are not deterministic: rerunning step 3 could produce a different plan than the first time, sending the rest of the run down a different path. Recording each model output as a step result means a resumed run follows the same path it was already on.
Durable execution engines, such as Temporal, Restate, Inngest, and DBOS, provide this as infrastructure, and agent frameworks such as LangGraph offer checkpointing of agent state. They differ in how they store state and what they ask of your code, but the principle is the same: progress lives in a database, not in a process's memory.
Side effects need idempotency
Durable execution has one gap, and it is the dangerous one. If a step sends an email and the process crashes before the result is recorded, the runtime cannot know the email went out. On resume, it runs the step again.
The fix is to make side effects idempotent: safe to repeat. Give every action a key derived from the run and the step, and check it before acting.
def send_vendor_email(run_id: str, vendor_id: str, body: str) -> dict:
key = f"{run_id}:renewal-email:{vendor_id}" # same step, same key, on every retry
if sent := outbox.get(key):
return sent # already sent: return the saved result
result = mailer.send(
to=vendors[vendor_id].email,
body=body,
idempotency_key=key, # many APIs deduplicate on this key too
)
outbox.save(key, result)
return resultThe key stays the same however many times the step runs, so the second attempt finds the record and returns it instead of sending again. Many payment and messaging APIs accept an idempotency key themselves, which covers the crash window between sending and saving. Design every tool a background agent can call this way, as the agents post recommended for tool design generally.
Waiting for people
Long-running agents often need a person: approval before spending money, a decision between options, or information only someone else has. In a request, that is impossible. In a durable job, it is just another step, one that waits.
The run saves its state, records what it needs, and notifies the right person. Nothing runs, and nothing costs money, while it waits. When the answer arrives, by button, message, or API call, the run resumes from the saved state.
Two details matter. Every wait needs a deadline and a fallback, such as escalating, cancelling, or proceeding with a safe default, or runs pile up waiting for answers that never come. And after a long wait, the agent must re-check the world before acting: prices change, records get updated, and another person may already have done the task.
Budgets and stop conditions
A background agent has no one watching it in real time, so its limits must be explicit. Set a maximum number of steps, a token or dollar budget, and a wall-clock deadline for every run. When any is reached, the run stops in a clear state and reports what it did and what remains.
Watch for runs that are busy but not progressing: the same tool called repeatedly with the same arguments, or the same error retried in a loop. A heartbeat that records the last meaningful step lets a supervisor process spot stuck runs and stop them, rather than discovering them on next month's bill.
Reporting back
The person who started a background run is, by definition, not watching. When they return, they need to understand quickly what happened. Good runs leave a clear trail: a short status that updates as the work proceeds, a final summary of what was done and what was not, links to the artifacts produced, and a record of every action with side effects.
Every run should end in an explicit, reported state: completed, failed, over budget, or cancelled. The worst outcome is not a failure but a silent one, a run that stopped somewhere and told no one.
When to use it, and when not
Background agents fit work that takes longer than a person wants to wait and does not need them at every step: research across many sources, changes across a codebase that end in a pull request, processing a batch of documents, reconciling records, and monitoring tasks that run on a schedule. They also fit any workflow with built-in waits, such as approvals or replies.
Do not make work long-running when it does not need to be. If a task can be done in one fast call, a job queue and checkpoints add complexity for nothing. If many independent calls need no agent loop at all, such as classifying 100,000 records, batch APIs are simpler and usually cheaper. And interactive work, where the user is waiting and steering, belongs in a conversation, not a background job.
What goes wrong
Lost progress. Runs keep state in memory, and any restart throws hours of work away. Store progress outside the process from the first step.
Duplicate actions. Retries and resumes repeat steps that had already sent an email, created a ticket, or issued a refund. Make every side effect idempotent, and test the crash-and-resume path deliberately rather than hoping it works.
Runaway runs. A loop that never finishes keeps spending tokens all weekend. Enforce budgets in the runtime, not just in the prompt, because a model told to stop after 50 steps will not always count correctly.
Context bloat. Hours of tool results and reasoning fill the context window, and quality degrades long before the hard limit. Summarize completed work, store large results outside the context, and give the agent a running notes file, as described in the context engineering and memory posts.
Acting on a stale world. An approval granted on Monday is used on Wednesday, after the facts changed. Re-validate state immediately before any action with consequences, and expire approvals that are too old.
Silent failure. A run fails at 3 a.m., and nobody finds out until someone asks where the report is. Alert on failures, on runs past their deadline, and on runs with no progress, and make sure each alert reaches a person who can act on it.
Cost, latency, and complexity
Long runs accumulate cost in a way single calls do not. Each step resends a growing context, so the cost per step rises over the run. Prompt caching cuts that sharply for the stable parts of the context, and compaction keeps the growing part in check. Track cost per run, and alert when a run exceeds what that kind of task normally costs.
Latency becomes a different question. Nobody minds whether a background run takes 40 minutes or 50; they mind not knowing. Time to a useful status update matters more than time to completion. For bulk work where nobody is waiting at all, batch APIs from OpenAI, Anthropic, and Google trade a delay of up to a day for a discount of around half the normal price.
The complexity is real but well understood. Job queues, durable workflows, idempotency, and alerting are standard backend engineering, older than AI agents. The cost of adding them is modest compared with the cost of explaining to 30 vendors why they received two emails.
The current landscape
At the model level, OpenAI's Responses API supports background mode with polling, resumable streaming, and cancellation, and the major providers offer batch APIs for asynchronous bulk work. Agent SDKs from the major providers, along with frameworks such as LangGraph, include state persistence or checkpointing to varying degrees.
For durable execution, Temporal is the most established workflow engine, while Restate, Inngest, and DBOS offer lighter approaches, from a hosted platform to a library storing workflow state in your existing Postgres database. Cloud workflow services such as AWS Step Functions fill a similar role. Several now publish guides specifically for AI agents.
Background agents are also arriving as products. Coding tools such as GitHub's Copilot coding agent, OpenAI's Codex cloud tasks, and Google's Jules take a task, work in their own environment, and return a pull request, as the coding assistants post described. Their design choices, including sandboxes, status updates, and a reviewable result at the end, are a good model for any background agent.
Summary
An agent that runs for hours is no longer a function call. It is a job, and it needs what every serious background job needs: progress that survives restarts, actions that are safe to repeat, waits that cannot hang forever, limits that are actually enforced, and an ending that someone hears about.
So treat long-running agents as backend systems first and AI second. Start them as jobs, record every step, make every side effect idempotent, put deadlines on every wait, enforce budgets in code, and report every outcome. The model decides what to do next; the infrastructure makes sure that whatever it decides survives the night.
Part of the Applied AI series: AI from the builder's side.