Agents and tools

The agent loop

An agent is a program with three parts: a model that decides, a set of tools (functions), and a loop that connects them. On each round:

1. the model receives the full context (system prompt, tool descriptions, task, and history) and produces either a call (a name and data) or the final answer;
2. the program validates the call, applies the permissions, and runs the function;
3. the result is added to the history as one more message and we go back to step 1, until the model answers or a step limit runs out.

That is the idea behind ReAct (Yao et al., 2022): interleave reasoning and acting so that each observation corrects the next reasoning step. In their tests, with only one or two examples in the prompt, ReAct beat imitation and reinforcement-learning methods by 34 and 10 points of success rate in two interactive environments (ALFWorld and WebShop), and on questions answered with a Wikipedia API it reduced hallucination compared with chain of thought alone. Since 2023, providers’ interfaces (for example, the “function calling” OpenAI launched on June 13, 2023) make the model emit the call in a structured format. Before that, Toolformer (Schick et al., 2023) had trained a model to decide on its own when to call an API and with which arguments.

A function call, inside

A tool is described by a name, a description, and a schema of its input data (JSON Schema). Field names vary from provider to provider; the idea is the same:

// what the model receives (as text) at the front of its context
{ "name": "buy_tickets",
  "description": "Buys tickets for a showtime. Charges the saved card. Cannot be undone.",
  "input_schema": { "type": "object",
    "properties": { "showtime": { "type": "string" }, "quantity": { "type": "integer", "minimum": 1 } },
    "required": ["showtime", "quantity"] } }

// what the model writes: a request, not an execution
{ "name": "buy_tickets", "input": { "showtime": "9:30 pm", "quantity": 2 } }

The model runs nothing: it produces the text of the call. The program must treat that data as untrusted input: validate it, bound it, and log it. The tool description (for example, “cannot be undone”) helps the model decide, but it does not protect: the spending limit lives in the code that executes.

MCP

The Model Context Protocol (Anthropic, November 25, 2024) standardizes that connection. An application (the client) connects to an MCP server that offers tools; messages are JSON-RPC 2.0: tools/list to discover the tools and tools/call to invoke one, and the result comes back with an isError flag. The specification (version 2025-06-18) says that, for trust and safety, there should always be a human in the loop with the ability to deny an invocation, and that the client should ask for confirmation on sensitive operations, show the inputs before calling the server, and log tool usage for audit purposes. On December 9, 2025, the Linux Foundation announced the Agentic AI Foundation, with MCP, contributed by Anthropic, among its founding projects.

Reliability of a chain

If each step goes right with probability $p$, independently, and one error is enough to ruin the task, an $n$-step task goes right with probability

With $p = 0.95$: $P(2) \approx 0.90$, $P(10)\approx 0.60$, $P(14)\approx 0.49$, $P(20)\approx 0.36$, and $P(40)\approx 0.13$. The number of steps at which success falls to one half is

This is a simplified model: it assumes independent errors and that any error ruins the attempt. In practice errors correlate (a bad plan drags down the steps after it), and sometimes the agent recovers on its own by reading an error message. That is why end-to-end success is measured, not computed. The lesson’s simulator does exactly this calculation with twenty attempts (fixed seed): the sample gives 12 of 20 with ten steps, 9 with fourteen, 7 with twenty, and 3 with forty.

Review in stretches

A stretch of $m$ steps goes right with $A = p^{m}$. At the end of the stretch, a person reviews it and catches an error with probability $q$; if caught, the stretch is redone. One attempt at the stretch is accepted with probability $A + (1-A)(1-q)$, so

Total success is the product of the $s$ of all stretches. With $p = 0.95$, $n = 20$, and $q = 0.9$ (the lesson’s assumptions): no review, 35.8%; reviewing every 5 steps, 89.1%, with 5.0 reviews and 25.1 agent steps per attempt; every 10, 87.8% (3.1 reviews, 31.3 steps); only at the end, 84.8% (2.4 reviews, but 47.3 steps: everything is redone). The ceiling is set by $q$: even reviewing every step, success stays at 90.0%, because one error in ten slips through.

Measuring consistency

τ-bench (Yao, Shinn, Razavi, and Narasimhan, 2024) puts an agent with tools and business rules in front of a customer simulated by another model, and compares the final state of the database with the expected one. It introduces $\text{pass}^k$: the probability that the same task goes right in all $k$ attempts. With function calling, GPT-4o solved 61.2% of the store’s tasks and 35.2% of the airline’s on the first try (48.2% on average), and in the store its $\text{pass}^8$ fell below 25%.

Cost

If each query rereads the instructions and the definitions ($B$ tokens) plus the whole history, and each step adds $g$ tokens, the total read over $n$ steps is

It grows like $n^2$, like the conversations in lesson 30. With illustrative assumptions ($B = 2{,}000$, $g = 800$): 10 steps is 56,000 tokens; 20, 192,000; 40, 704,000. And a failed attempt is paid for all the same: the cost per completed task is the cost per attempt divided by the probability of success. With $p = 0.95$ that comes to about 94,000, 536,000, and 5.5 million tokens per completed task.

Levels of autonomy

Feng, McDonald, and Zhang (2025) propose five levels according to the person’s role: operator, collaborator, consultant, approver, and observer, and argue that autonomy is a design decision, separate from the model’s capability. The lesson uses a shorter scale (suggests, acts with approval, acts up to a limit, acts alone) with the same idea: more autonomy means more speed and less control, and it is granted action by action according to reversibility. Anthropic (2024) recommends starting with the simplest solution: if the steps are known in advance, a fixed workflow is usually enough; an agent is justified when the path depends on what is observed, and its autonomy brings higher costs and compounding errors, so it should be tested in a sandbox with guardrails.

In code

SHOWTIMES = {"8:00 pm": {"seats": 0, "price": 95}, "9:30 pm": {"seats": 14, "price": 95}}
LIMIT = 300   # permission: above this, the program asks for approval

def find_showtimes(date):
    return {time: dict(s) for time, s in SHOWTIMES.items()}

def buy_tickets(showtime, quantity):
    s = SHOWTIMES[showtime]
    if quantity > s["seats"]:
        raise ValueError("not enough seats")
    s["seats"] -= quantity
    return {"showtime": showtime, "quantity": quantity, "total": quantity * s["price"]}

TOOLS = {"find_showtimes": find_showtimes, "buy_tickets": buy_tickets}

def model(context):
    """A script instead of an LLM: it decides from the last thing in the context."""
    last = context[-1]
    if last["role"] == "user":
        return {"tool": "find_showtimes", "args": {"date": "today"}}
    if last["tool"] == "find_showtimes":
        time = next(t for t, s in last["result"].items() if s["seats"] >= 2)
        return {"tool": "buy_tickets", "args": {"showtime": time, "quantity": 20}}  # mistake: one zero too many
    if last["result"] == "rejected by the person":
        return {"tool": "buy_tickets", "args": {"showtime": "9:30 pm", "quantity": 2}}
    return {"answer": f"Done: {last['result']}"}

def approves(request):   # the human in the loop (here, a rule that mimics their decision)
    print("Approve?", request["args"])
    return request["args"]["quantity"] <= 4

def agent(task, max_steps=6):
    context = [{"role": "user", "text": task}]
    for step in range(1, max_steps + 1):
        decision = model(context)                            # 1. think
        if "answer" in decision:
            return decision["answer"], step
        name, args = decision["tool"], decision["args"]
        if name == "buy_tickets":                            # permission: a rule in the program
            total = args["quantity"] * SHOWTIMES[args["showtime"]]["price"]
            if total > LIMIT and not approves(decision):
                context.append({"role": "result", "tool": name, "result": "rejected by the person"})
                continue
        try:
            result = TOOLS[name](**args)                     # 2. act: the program runs it
        except Exception as e:
            result = f"error: {e}"                           # an error is also an observation
        context.append({"role": "result", "tool": name, "result": result})  # 3. observe
    return "I did not finish.", max_steps

print(agent("Buy two tickets for tonight, for under 300 pesos."))

# Reliability of a chain: each step goes right with p = 0.95
for n in (1, 2, 10, 14, 20, 40):
    print(n, round(0.95 ** n, 2))

Output:

Approve? {'showtime': '9:30 pm', 'quantity': 20}
("Done: {'showtime': '9:30 pm', 'quantity': 2, 'total': 190}", 4)
1 0.95
2 0.9
10 0.6
14 0.49
20 0.36
40 0.13

Swap model for a call to an LLM and you have the skeleton of a real agent. What a serious product adds is what is only sketched here: input validation, per-tool permissions, step and spending limits, a log of every call, and error handling.

What comes next

An agent that reads and acts is wrong with confidence: lesson 33 covers hallucinations. Lesson 34 teaches how to measure a system with your own tasks, and lesson 35 covers the risks that tool access opens (for example, instructions hidden in what the agent reads) and how to govern them.