Information in this post reflects publicly available sources as of October 16, 2026.
It's 9:40 on a Tuesday and checkout errors have tripled since midnight. You paste the question into a chat assistant: why are checkout requests failing? You get back a tidy, confident list. Database connection limits. A bad deploy. An expired certificate. Rate limiting from an upstream provider. All plausible, and none of them checked.
Now you ask the same question to a coding agent running in your terminal. It searches the logs and finds the first error at 02:14. It lists last night's deploys and spots one at 02:11. It reads the diff, notices that a payment client timeout dropped from 30 seconds to 3, and points you at the line. Then it asks whether you want it to open a revert.
Here is the part that surprises people: both answers may have come from the same model. The difference is not intelligence. The second one was allowed to look things up, act on what it found, and keep going until it had an answer. That arrangement is what we call an agent.
A model on its own can only talk
A plain model call is a function. Text goes in, text comes out. Everything the model knows comes from its training data and whatever you put in the prompt, and nothing else.
That leaves three hard limits. The model cannot see the current state of anything: your logs, your database, today's ticket queue. It cannot act: it can describe a fix but not apply one. And it cannot check its own work, because checking requires running something and looking at the result.
Before agents, people worked around this by hand. You asked the model what to run, ran it yourself, pasted the output back, and asked what to do next. That works, but notice what you were doing. You were the loop. An agent is what you get when you automate the human in the middle of that exchange.
How it works
Step one: the model asks, your code acts
A model cannot run anything. What it can do is ask for something to be run. This capability is called tool calling, also known as function calling, and every major model family supports it.
You start by describing the tools available. Each tool gets a name, a plain-language description, and a JSON Schema for its arguments:
{
"name": "query_logs",
"description": "Search application logs for one service. Returns up to 50 matching lines, newest first. Returns an empty list if nothing matches.",
"input_schema": {
"type": "object",
"properties": {
"service": { "type": "string" },
"level": { "type": "string", "enum": ["ERROR", "WARN", "INFO"] },
"since": { "type": "string", "description": "ISO 8601 timestamp" }
},
"required": ["service"]
}
}This definition is sent to the model alongside the conversation. The exact field names vary by vendor (OpenAI calls the schema parameters, Anthropic calls it input_schema), but the shape is the same everywhere. The description matters more than it looks. It is the only thing the model reads when deciding whether and how to use the tool, so it behaves like documentation written for a very literal colleague.
When the model decides a tool would help, it does not answer in prose. It returns a structured request: the tool name and a set of arguments that match the schema. Your code receives that request, runs the real function, and sends the result back as a new message. The model reads the result and decides what to do next.
The single most important fact about tool calling is in step 4 of that diagram. The model never executes anything. Every action passes through code you control, which means every action can be logged, checked against a permission list, or held for a human to approve. When people talk about making agents safe, this is the choke point they are talking about.
Step two: put it in a loop
One tool call is useful. An agent is what happens when you let the model make as many as it needs. Strip away the frameworks and the core of every agent looks something like this:
def run_agent(goal, tools, max_steps=20):
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": goal},
]
for step in range(max_steps):
response = model.generate(messages=messages, tools=tools)
messages.append(response.message)
if not response.tool_calls:
return response.text # the model decided it is done
for call in response.tool_calls:
try:
result = TOOL_REGISTRY[call.name](**call.arguments)
except Exception as e:
result = f"Error: {e}" # errors go back to the model too
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": str(result),
})
return "Stopped: step limit reached"The model.generate call is a stand-in for whichever SDK you use, but the logic is real. Each pass sends the full message history to the model. If the response contains tool calls, the code runs them and appends the results. If it contains no tool calls, the model has decided it is finished and its text is the answer. The max_steps limit exists because the model's judgment about when to stop is not something you should rely on alone.
Notice the except branch. Errors are not swallowed; they become tool results. A model that sees Error: service 'checkout-api' not found will often try checkout next. That ability to recover from a failed step is a large part of what makes agents feel capable.
Step three: the context window is the agent's memory
The model is stateless. Between one call and the next it remembers nothing, which is why the loop resends the entire message history every time. That list of messages, held in the model's context window, is the agent's working memory. Everything it knows about the task, every tool result, every dead end, lives there.
This has two consequences that shape how agents behave in practice. First, the history grows with every step, so each call costs more and takes longer than the one before. Second, quality degrades as the window fills. A crucial detail from step 3 is easier to overlook when it sits under 40,000 tokens of log output from steps 4 through 18.
Good agent products spend a lot of effort here. They trim large tool outputs, summarize older parts of the conversation (often called compaction), have the agent write notes to files it can reread later, and hand self-contained subtasks to subagents that start with a clean context and report back a short result. Deciding what goes into the window is a discipline of its own, and it gets its own post in this series.
The harness
Everything around the model in that loop has a name: the harness. It includes the system prompt, the tool set, the loop itself, the permission checks, the context management, and the logging. Claude Code, OpenAI Codex, and Cursor's agent mode are all harnesses wrapped around a model.
This explains something people find puzzling. Two products running the same model can behave very differently, one sharp and the other hopeless. Usually the model is not the difference. The tools, the instructions, and what each harness chooses to keep in context are.
Agents, workflows, and the space in between
Not everything that calls a model several times is an agent. The useful question is a simple one: who decides the next step?
In a workflow, your code fixes the path. A support pipeline might classify an incoming ticket, route it by category, then draft a reply with a model call at each stage. The model does the work inside each step, but it never chooses which step comes next. In an agent, the model makes that choice.
Workflows are easier to test, cheaper to run, and fail in ways you can predict. Plenty of products marketed as agents are workflows underneath, and that is usually a sign of good engineering rather than a shortcut. The agent end of the spectrum earns its cost only when the path genuinely cannot be known in advance.
When to use an agent, and when not to
Agents fit tasks where the next step depends on what the last step revealed. Debugging is the classic case: you do not know which file to read until the stack trace tells you. Research, codebase-wide changes, and incident investigation have the same shape. They fit best when results can be verified, because a test suite or a query result gives the agent something concrete to check its own work against.
They are the wrong tool when the steps are already known. If every ticket goes through the same three stages, a workflow will be faster, cheaper, and more reliable. They are also a poor fit for latency-sensitive paths where a user is waiting on a sub-second response, and for irreversible, high-stakes actions where nobody reviews the plan before it runs.
A practical test: if you can draw the flowchart before the task starts, build the flowchart. If the flowchart would need a box labeled "figure it out," that box is where an agent belongs.
What goes wrong
Errors compound. If each step succeeds 95% of the time, a 20-step task succeeds end to end only about 36% of the time (0.95 raised to the 20th power). Real agents recover from some mistakes, so the true number is better, but the shape holds: long tasks fail in ways short ones do not. Keeping tasks shorter, and giving the agent ways to verify progress, matters more than squeezing out another point of model accuracy.
Agents loop. A model that hits the same error will sometimes retry the same call with trivial variations until it runs out of steps. Step limits stop the bleeding. Better tool errors prevent it, because an error that says what to do differently gives the model somewhere new to go.
Tool results get misread. If query_logs returns an empty string on a typo in the service name, the agent may conclude there were no errors. Tools should say exactly what happened: "0 lines matched for service chekout; known services are checkout, cart, payments." Tool design is where most agent reliability is won or lost.
Tool results can carry instructions. Everything a tool returns lands in the context window, where the model reads it with the same attention it gives your instructions. A web page, an issue comment, or a log line can contain text written to steer the agent. This is prompt injection, and it is the defining security problem for agents. The defense starts with permissions: an agent that reads untrusted content should not also hold credentials to do damage without review.
Agents overreach. An agent told to "clean up the test environment" may decide that includes deleting a database it should not touch. Read-only tools, scoped credentials, and approval prompts for writes are not optional extras. They are the harness doing its job.
Cost and latency
Because each step resends the full history, cost grows faster than step count. Take a modest task: a 5,000-token starting context (system prompt, tool definitions, and the goal), with each step adding about 2,000 tokens of tool calls and results.
By step 20 each call is sending 43,000 tokens, and the run as a whole has sent about 480,000 input tokens. At an illustrative $3 per million input tokens, that is roughly $1.44 for one task before counting output. Run that on every support ticket in a busy queue and the numbers get serious quickly.
Latency compounds the same way. Each step is a full model round trip, often several seconds, plus however long the tool takes. Twenty steps means minutes, not seconds, which is fine for a background code fix and unacceptable for a chat reply.
The main levers are well established. Prompt caching, which most major providers now offer, discounts the repeated prefix of each call, and an agent's prefix repeats constantly. Trimming tool output keeps the history from ballooning. Routing simple subtasks to a smaller, cheaper model saves the large one for decisions that need it. And step and token budgets put a ceiling on the worst case.
Where you will meet agents today
The most mature agents are coding agents: Claude Code, OpenAI Codex, GitHub Copilot's coding agent, and Cursor's agent mode. Code is a near-ideal environment for agents because almost everything is verifiable. The agent can run the tests, read the failure, and try again.
Beyond code, the research modes in most chat assistants are agents that run dozens of searches before writing a report. Computer use and browser agents operate interfaces built for humans by taking screenshots and clicking. And a growing number of support, operations, and analytics products run agents behind the scenes, usually well fenced in by workflow-style guardrails.
If you are building one, the vendor SDKs are the usual starting point: the OpenAI Agents SDK, Anthropic's Claude Agent SDK (renamed from the Claude Code SDK), and Google's Agent Development Kit. Model-agnostic frameworks such as LangGraph, CrewAI, and Pydantic AI cover cases where you need explicit state machines or want to switch providers, and Microsoft's Agent Framework merges its earlier AutoGen and Semantic Kernel projects. Framework choice matters less than it first appears. The loop is twenty lines; the tools and the context strategy are where the real work is.
Two open standards sit underneath much of this. The Model Context Protocol (MCP) standardizes how agents connect to tools and data sources, so a tool written once works in any compatible client; its latest stable revision, 2026-07-28, removed protocol-level sessions to make servers easier to scale. The Agent2Agent protocol (A2A) covers agents calling other agents across frameworks. Both are coming up in their own posts in this series.
Summary
An agent is not a smarter model. It is an ordinary model given three things: tools it can ask to use, a loop that feeds results back, and a rule for when to stop. The model contributes judgment about what to do next. Everything else, including what it can touch, what it remembers, and when it is forced to halt, belongs to the harness you control.
That reframes how to approach agents in practice. When one misbehaves, the first questions are not about the model. What did the tools return? What was sitting in the context window? What was it permitted to do without asking? Most agent failures are answered there, and most agent improvements are made there too.
Part of the Applied AI series: AI from the builder's side.