Quick Facts
- ACRouter cost $13.21 on a full in-distribution test run versus $34.02 for always defaulting to Claude Opus, a 2.6x savings.
- The system uses a Context-Action-Feedback loop and a sub-billion parameter orchestrator to route tasks dynamically at runtime.
- On out-of-distribution agentic tasks, ACRouter reached 62.50% average performance, ahead of Always-Opus at 57.14%.
A research team has built an open-source AI router that learns which model to call based on past performance, slashing costs without sacrificing accuracy. The system, called ACRouter, was introduced in a paper first submitted to arXiv on June 22, 2026, by Pengfei Zhou and 10 co-authors.
Most enterprise AI deployments route every prompt to the same frontier model, regardless of task complexity. That approach is expensive and often unnecessary. ACRouter routes each task to the most capable model for that specific job, then logs the outcome and adjusts future decisions accordingly.
How It Works
ACRouter is built on a framework the researchers call Agent-as-a-Router. The system has three components: an Orchestrator, a Verifier, and a Memory module. Together they run a Context-Action-Feedback loop on every task.
The Memory module is a vector store keyed by task embeddings. It logs the chosen model, observed performance, cost, and the Verifier’s trace for each completed task. Retrieval uses cosine nearest-neighbor search with a similarity threshold of 0.5 and returns up to 10 past records. The store holds up to 20,000 entries on a first-in, first-out basis.
The Orchestrator reads the incoming prompt alongside retrieved memory and selects a model from the available pool. It runs on a fine-tuned Qwen 3.5 adapter with 0.8 billion parameters, small enough to self-host. The Verifier then evaluates the chosen model’s output and produces a success or failure signal, which feeds back into Memory for the next decision.
The practical effect: if a model hallucinates a column name on a SQL task and the query fails to compile, the Verifier logs that failure. The next time a similar query arrives, the router sends it to a stronger model instead.
Benchmark Results
The team tested ACRouter on CodeRouterBench, an evaluation environment of roughly 10,000 tasks with verified scores across eight frontier models, including Claude Opus 4.6, GPT-5.4, Qwen3-Max, and GLM-5. Tests split between in-distribution coding tasks and out-of-distribution agentic programming tasks requiring multi-step planning, file navigation, and iterative debugging.
On the in-distribution test, ACRouter reached the highest average performance at 49.98% and the lowest cumulative regret at 205.5, beating the next-best static router by 2.48 percentage points. Its performance-per-dollar ratio was 3.79, compared to 1.29 for Always-Opus.
On out-of-distribution tasks, where static routers break down as conditions shift, ACRouter reached 62.50% average performance. Always-Opus finished at 57.14%. A fine-tuned Qwen3.5-0.8B baseline scored 55.36%. The performance-per-dollar ratio for ACRouter was 1.18 on OOD tasks versus 0.64 for Always-Opus.
Why Static Routers Fail
The researchers identified three failure modes in existing routing systems. Static routers cannot accumulate new execution feedback after deployment. They break down when enterprise data or user behavior shifts away from their training distribution. And they become obsolete when new models enter the market, since their classifiers were trained on a fixed model pool.
ACRouter addresses all three by treating the router as an agent that learns continuously from verified outcomes rather than a classifier trained once and frozen.
Business Implications
For software teams running high-volume AI workloads, the cost gap is material. A 2.6x reduction in model spend on coding tasks compounds quickly at scale. The Orchestrator’s small parameter count also means the routing layer itself adds minimal overhead and does not require dedicated cloud infrastructure to operate.
The open-source release means engineering teams can adopt and extend the framework without licensing costs. Enterprises dealing with frequent model updates will also benefit from the system’s ability to adapt as new models enter the pool, without retraining the router from scratch.
Read more: ACRouter picks the smartest AI model per task, beating Opus-only setups by 2.6x on cost