AI Agent Routing: How Systems Send Work to the Right Specialists

A password reset and a billing dispute can land in the same support inbox in the same minute. Route both through one model with one prompt, and something has to give: either the easy one costs as much to answer as the hard one, or the hard one gets the same shallow pass the easy one needed.

A router solves that with one job, done before any real work starts: classify what the request is, then dispatch it to whoever handles that kind of request. Nothing else. That classification has to be cheap, or routing costs more than it saves.

What does a router do?

If you've already decided a job needs more than one AI agent instead of a single prompt, the next question is who gets that job once it exists.

That's routing: read a request, match it to the specialist built to handle it, before anyone does the actual work. If that first decision isn't settled yet, start with how to decide when a job needs an agent instead of one prompt.

Uber's support platform runs a version of this at production scale. Its Customer Obsession Ticket Assistant, COTA, reads an incoming support ticket and sends it toward a resolution path, live across more than 400 cities [1].

What Uber measured	The number
Markets covered	400+ cities
Tickets handled	90%+ of inbound support tickets
Faster resolution	about 10% less handling time per ticket
What Uber says it saves	"tens of millions of dollars every year"

Uber's own writeup doesn't publish COTA's classifier or the threshold that sends a ticket to a person instead of an automated path. What it does publish is one internal comparison: two ways of ranking candidate solutions once a ticket has already been read.

A model comparing candidates by similarity beat a model assigning one fixed topic by 25%, on that ranking step alone [1]. That's a real number about one modeling choice inside COTA, not a claim about the whole system's accuracy.

What makes a preference-trained router different?

A router built the way COTA's ranking step works still needs someone to write the categories by hand. RouteLLM, from the team behind the LMSYS Chatbot Arena, trains its classification step on preference data instead: pairs of responses, one from a cheap model and one from an expensive one, scored by which a person liked better [2].

That's the sharper mechanism. Fix the categories by hand, and a router can only sort a request into one someone already wrote down.

Train it on preference data instead, and it learns something narrower: which prompts the cheap model already answers just as well, whatever category they fall into. That's what lets it generalize to a request type nobody labeled in advance, because it was never learning categories in the first place.

On MT-Bench, RouteLLM's routers cut cost by more than 85% while keeping 95% of the stronger model's quality, sending only the harder slice of requests to the expensive model [2]. The team published the code, the training data, and the results, so a technical reader can check the number against MT-Bench's own leaderboard rather than take it on faith [3].

When does a router get it wrong?

Routing can also fail in the other direction: sending too much down the cheap path.

In February 2024, Klarna announced that its AI assistant had handled 2.3 million customer service conversations in its first month, doing the work of about 700 full-time agents and covering two-thirds of the company's chats [4].

Read as a routing decision, that's one threshold set very wide: most incoming conversations went straight to the automated path, with less held back for a person.

By May 2025, Klarna's CEO said the focus on cost had produced lower quality, and the company started hiring customer-service staff again so a person stays reachable [5]. Klarna didn't say the assistant stopped working. It said the line for what stayed automated had been drawn too far in one direction once real volume, and real edge cases, hit it.

That's the same mechanism this whole guide is about, run backwards. Uber's ranking model works because a person can check it against a labeled outcome.

Klarna's launch numbers were real, and the walk-back belongs in the same case, not as an asterisk on it: a routing decision that looks like a clean win in month one can still need a wider escalation path once it runs for a year.

How do I route work in my own AI office?

My own AI office runs a small version of the same idea, one fan-out example, not a production system with its own measured savings.

Before any of my specialists start on a request, one step reads it and works out who should see it. The simplest version is a keyword match:

# orchestrator/router.py, route_request() -- classify by keyword, dispatch by role
words = set(re.findall(r"[a-z]+", request.lower()))
selected = [
    role
    for role, keywords in _ROUTING_KEYWORDS.items()
    if any(word == kw or word.startswith(kw) for kw in keywords for word in words)
]
return selected or ["strategy", "product"]

That match never returns nothing. A request that matches no keyword still lands with two generalist roles, because a routing step that dispatches to nobody has done no work at all.

For a request the keyword table would misread, a second layer replaces it with one small model call instead of a full agent turn:

# orchestrator/router.py, CosPlanner() -- one small model call, not a full agent turn
"""
One small model call on the CoS's own model -- no Company Brain, no MCP
servers, no memory tools. Staffing is a classification over a fixed
roster, so it does not need the machinery a full agent turn needs, and
paying for that machinery on every request would make routing the slowest
step in the graph.
"""

Some requests need two specialists in order, not one: Growth rewrites a page, then Engineering lands the change, and the second role runs only once the first one's output exists to hand over.

My own plan stops at three specialists deep for exactly that reason. A handoff chain longer than that has stopped being a routing decision and turned into a project.

I asked my own office a question once without naming who should answer it, and it reached an engineer in four minutes. That run is written up in the guide where it first appeared; I'm keeping it there rather than telling it twice.

What should you check in your own AI tools this week?

You probably already run more than one AI tool without calling any of it a router: a support chatbot, something that drafts your marketing copy, a research assistant you paste questions into.

Most small businesses never need to build a custom router. What's worth an hour of your time is checking whether one of those tools is doing a job it's too expensive, or too shallow, for.

The job	The tool doing it now	The question to ask
Answering "where's my order"	A general chatbot, same prompt every time	Would a lookup against your order system answer this without a model at all?
Drafting a reply to an angry refund request	The same chatbot	Does this need a person before it goes out, the way Klarna's did?
Summarizing last week's reviews	A person, by hand, every Monday	Would a cheap, fixed prompt do this the same way every time?

Pick one tool you already pay for and ask which of its jobs is the wrong size for it: too expensive for what it's answering, or too shallow for what it's being asked to judge. That's the whole routing decision, before anyone builds anything to automate it.

Quick recap

• A router makes one decision before real work starts: classify the request, then dispatch it. Nothing else.
• Uber's COTA runs this across 400+ cities, handles 90%+ of inbound tickets, and Uber says it saves tens of millions of dollars a year [1].
• RouteLLM trains the classification step on which response a person preferred, not on fixed task labels, and that's what lets it generalize to a request type nobody labeled [2].
• Klarna's 2024 launch and 2025 walk-back are the same mechanism, run in the direction that sends too much down the cheap path [4], [5].
• A routing step that dispatches to nobody has done no work. Mine falls back to two generalist roles rather than returning nothing.
• Check one AI tool you already pay for: is it doing a job that's the wrong size for it?

Start Here

The intake at daisyguti.ai/work-with-me is about nine questions and takes a few minutes, held as a live conversation with an AI. It turns your answers into a brief with what you're trying to solve, what you've already tried, and what a working version looks like to you.

I read that and reply with whether a routing rule between your existing tools, or a custom workflow automation, would fit.

Sources

1. Uber, "COTA: Improving Uber Customer Care with NLP & Machine Learning" (January 3, 2018) - https://www.uber.com/blog/cota/ - 400+ cities, 90%+ of inbound tickets, about 10% faster handling, "tens of millions of dollars every year," and the 25% relative accuracy gain on the ranking-model comparison specifically.
2. LMSYS Org, "RouteLLM: An Open-Source Framework for Cost-Effective LLM Routing" (July 1, 2024) - https://www.lmsys.org/blog/2024-07-01-routellm/ - over 85% cost reduction on MT-Bench at 95% of GPT-4 quality, trained on preference data rather than task labels.
3. LMSYS Org, "Chatbot Arena Leaderboard Week 8: Introducing MT-Bench and Vicuna-33B" (June 22, 2023) - https://www.lmsys.org/blog/2023-06-22-leaderboard/ - where MT-Bench itself is documented and scored.
4. Klarna, "Klarna AI assistant handles two-thirds of customer service chats in its first month" (February 27, 2024) - https://www.klarna.com/international/press/klarna-ai-assistant-handles-two-thirds-of-customer-service-chats-in-its-first-month/ - 2.3 million conversations in the first month, equivalent to 700 full-time agents.
5. Entrepreneur, "Klarna Is Hiring Customer Service Agents After AI Couldn't Cut It on Calls, According to the Company's CEO" (May 9, 2025) - https://www.entrepreneur.com/business-news/klarna-ceo-reverses-course-by-hiring-more-humans-not-ai/491396 - Sebastian Siemiatkowski on cost-driven lower quality and rehiring human agents.