What Should an AI Agent Escalate to a Person?
An AI agent should escalate when a wrong action would cost money, breach a rule, or exceed its confidence. This article shows how to turn that sentence into a specification a developer can price.
An AI agent should escalate to a person when the next action would create a financial, legal or safety consequence that the agent cannot reverse, or when its confidence falls below a threshold you set in advance. To specify this for AI agent development, write three things: the exact triggers for handover, the list of tools the agent may call without approval, and the record it must leave when it stops. A developer can price those three items; a request to "use AI to handle enquiries" cannot be priced.
What is an AI agent and how is it different from a chatbot?
An AI agent takes actions in your systems, while a chatbot returns text to a person who then takes the action. That difference decides the escalation design, because every action the agent can take is a point where it may need to stop and ask.
A chatbot that answers "your order ships Tuesday" leaves the shipping update to a human. An agent with tool access can call the order system, change the address and send the confirmation. The second version needs a rule for when it may not do that.
For a Los Angeles retailer, the practical example is a customer asking to reroute a delivery to a different address. The agent can read the order and propose the change, but a reroute after dispatch affects the carrier contract and the customer's payment protection. If an action changes money, a contract or a regulated record, the agent proposes and a person approves. That single rule removes most of the risk from the first deployment.
The distinction also changes what you buy. A chatbot project is priced by conversation design and content. An agent project is priced by the number of tools, the permission model around each tool, and the escalation paths. Our AI agent development work starts by listing those tools, because that list is the scope.
How do you turn a vague business problem into a priced specification?
You turn a vague problem into a priced specification by writing the agent's job as a sequence of decisions with a named owner for each outcome. The developer prices the decisions, not the ambition.
Start with one workflow that has a visible cost today, such as routing inbound enquiries or updating a record after a phone call. Write the steps a competent employee takes, including the checks they perform before acting. That written sequence becomes the agent's task list and the developer's estimate.
Then separate what the delivery team controls from what you control. The team controls the agent logic, the tool integrations and the test cases. You control the policy: which actions are allowed, who approves them, and what the agent must do when it is unsure. If you cannot name the person who approves a refund, the agent cannot be specified, because the escalation has no destination.
A useful test is to write ten real examples of the workflow, including two that went wrong. If the agent would handle the two wrong ones differently from your staff, you have found a rule that belongs in the specification. This is the same discipline we apply in AI customer service development, where the enquiry types decide the build.
Single agent versus multi-agent: which fits your escalation rules?
A single agent with a small set of tools is easier to specify and cheaper to test than a multi-agent system, and it should be the default until one agent's tool list becomes too broad to govern.
In a single-agent design, one model holds the conversation and calls tools such as order lookup or calendar booking. The escalation rule is one list, and the test suite covers one decision path. For most enquiry routing and record updating, this is enough.
A multi-agent design splits the work between a router agent and specialist agents, for example one for billing and one for scheduling. It helps when the specialist knowledge is large or when different teams own different approvals. It costs more because each handover between agents is another place where context can be lost, and each agent needs its own guardrails.
| Decision | Single agent | Multi-agent |
|---|---|---|
| Tool list size | One list, easy to review | Separate lists per agent, more review work |
| Escalation rule | One trigger list | Triggers per agent plus handover rules between them |
| Testing effort | One decision path | Each agent and each handover path |
| Best fit | Enquiry routing, record updates | Distinct teams or large specialist knowledge |
| Cost driver | Number of tools | Number of agents and handovers |
For a Los Angeles property manager handling maintenance requests, a single agent can log the request and book a vendor. A multi-agent design only becomes justified when the same system must also handle lease questions under California rental rules, where the approval chain differs by property. Add an agent only when a separate approval owner exists, not because the workflow feels complex.
How do you keep an AI agent from taking wrong actions?
You keep an agent from taking wrong actions by limiting what it can do without approval, not by asking it to be careful. Agent guardrails are permission rules enforced outside the model.
Agent tool access should be granted per action, not per system. An agent that can read orders does not automatically need to cancel them. Write each tool with the narrowest permission that completes the task, and log every call with the input, the output and the conversation that triggered it.
Then set thresholds the developer can implement:
- Money: any refund, credit or price change above a figure you set requires approval.
- Records: any change to a regulated record, such as a lease or a payment instruction, is proposed and confirmed.
- Confidence: if the agent cannot match the request to a known intent, it stops and asks.
- Repetition: if the same customer asks twice for the same exception, the agent routes to a person.
The delivery team can build these rules and test them against your ten examples. What it cannot do is decide your risk appetite. Set the money threshold before development starts, because it changes the approval step and therefore the price. This is the part of scoping that most often stalls a project.
If your agent also touches physical operations, such as a restaurant's order flow, the same rule applies to the point-of-sale record. Our notes on restaurant POS system requirements show how record changes are specified there.
When should an AI agent hand over to a human?
An agent should hand over to a human when the request falls outside its written scope, when the next action needs approval, or when the customer asks for a person. Conversational AI handover is a designed step, not a failure.
Write the handover as a state with a destination. The agent should pass the conversation summary, the actions already taken and the reason for stopping to a named queue. A handover that drops the context forces the customer to repeat themselves, which is worse than no agent at all.
For readers in Los Angeles, customer expectation adds a local detail: many support conversations happen in English and Spanish, and a handover to a person should route to someone who can continue in the language the customer used. If the agent cannot continue in the customer's language, the handover should happen at the first turn, not after three failed attempts.
The delivery team controls the routing logic and the summary format. You control the queue and the staffed hours. If no one is available at 2 a.m., the agent should say so and offer a callback rather than promise a person who is not there.
How much does it cost to build an AI agent, and what drives agent cost per conversation?
Build cost depends on the number of tools, the number of escalation paths and the quality of your written examples; running cost depends on conversation length, model choice and how often the agent calls a tool.
Agent cost per conversation is driven by tokens, tool calls and handovers. A short enquiry that ends in a lookup is cheap. A long conversation that calls three systems and ends in a handover costs more, and the handover itself consumes staff time. Budget for the handover rate as a percentage of conversations, because it is the cost line that grows quietly.
A custom AI agents for business project is usually priced in stages: specification, build, test against your examples, and a pilot period. Ask any AI agent development company to quote the specification separately, so you can see whether the scope is understood before the build begins. Our Los Angeles engagements follow that sequence, and the FAQ page covers the commercial questions buyers ask first.
Frequently asked questions
What is an AI agent and how is it different from a chatbot?
An AI agent takes actions in your systems, such as updating a record or booking a slot, while a chatbot returns text that a person acts on. The difference matters because every action creates a point where the agent may need approval or escalation.
How much does it cost to build an AI agent?
Build cost depends on the number of tools the agent can call, the number of escalation paths and how well your examples are written. Running cost depends on conversation length, model choice and how often the agent hands over to staff. Ask for the specification to be quoted separately from the build.
How do you keep an AI agent from taking wrong actions?
Grant tool access per action rather than per system, so the agent can read an order without being able to cancel it. Set money thresholds and require approval above them, and log every tool call with its input and output. These guardrails are enforced outside the model.
When should an AI agent hand over to a human?
Hand over when the request is outside the written scope, when the next action needs approval, or when the customer asks for a person. The handover should pass a summary, the actions already taken and the reason for stopping to a named queue. If no staff are available, the agent should say so rather than promise a callback.
What drives agent cost per conversation?
Cost per conversation is driven by tokens, the number of tool calls and the handover rate. A short enquiry that ends in a lookup is cheap, while a long conversation that calls several systems and ends in a handover costs more. The handover rate is the line that grows quietly, so track it from the pilot.
Questions we are asked
What is an AI agent and how is it different from a chatbot?
An AI agent takes actions in your systems, such as updating a record or booking a slot, while a chatbot returns text that a person acts on. The difference matters because every action creates a point where the agent may need approval or escalation.
How much does it cost to build an AI agent?
Build cost depends on the number of tools the agent can call, the number of escalation paths and how well your examples are written. Running cost depends on conversation length, model choice and how often the agent hands over to staff. Ask for the specification to be quoted separately from the build.
How do you keep an AI agent from taking wrong actions?
Grant tool access per action rather than per system, so the agent can read an order without being able to cancel it. Set money thresholds and require approval above them, and log every tool call with its input and output. These guardrails are enforced outside the model.
When should an AI agent hand over to a human?
Hand over when the request is outside the written scope, when the next action needs approval, or when the customer asks for a person. The handover should pass a summary, the actions already taken and the reason for stopping to a named queue. If no staff are available, the agent should say so rather than promise a callback.
What drives agent cost per conversation?
Cost per conversation is driven by tokens, the number of tool calls and the handover rate. A short enquiry that ends in a lookup is cheap, while a long conversation that calls several systems and ends in a handover costs more. The handover rate is the line that grows quietly, so track it from the pilot.
