Alibaba Research Cuts AI Agent Token Use by 99% With New Routing Framework

Quick Facts

• SkillWeaver reduces token consumption per query from approximately 884,000 to about 1,160, a greater than 99% reduction.
• The framework’s feedback loop raised decomposition accuracy from 51% to 67.7% on a 7B model, and to 92% on Alibaba’s Qwen-Max model.
• Source code has not been released, but researchers published prompt templates compatible with LangChain and LlamaIndex.

Researchers at Alibaba have published a framework called SkillWeaver that routes AI agent tasks without loading an entire tool library into context. The result is a 99% drop in token use per query, directly cutting API costs and latency for enterprise AI systems.

The problem SkillWeaver addresses is common in production AI deployments. Agents handling complex, multi-step workflows must coordinate across hundreds of tools. Exposing the full tool library to the model on every call consumes hundreds of thousands of tokens and can exceed context limits entirely.

How the Framework Works

SkillWeaver runs a three-stage pipeline: Decompose, Retrieve, and Compose. An LLM first breaks a complex user query into atomic subtasks. A bi-encoder retriever with FAISS indexing then finds candidate tools for each subtask from the skill library. A dependency-aware planner assembles the results into a Directed Acyclic Graph, mapping which tools must run sequentially and which can run in parallel.

The key mechanism is a feedback loop called Iterative Skill-Aware Decomposition, or SAD. Standard LLM decomposition produces generic step descriptions that often fail to match the specific vocabulary of real tools in a library. SAD fixes this by running a preliminary search, feeding the loosely matched tools back to the LLM as hints, and letting the model rewrite its plan to match the actual tools available.

Performance Numbers

Without SAD, a 7-billion parameter Qwen2.5 model achieved 51% decomposition accuracy. With SAD active, accuracy reached 67.7%. On hard tasks requiring four to five distinct skills, SAD improved accuracy by 50%. Competing methods performed poorly: the direct LLM approach hit 21.1% accuracy in tool retrieval, while ReAct-style agents reached 0%.

One counterintuitive finding: a 14-billion parameter model performed worse than the 7B model in the baseline setup. The larger model over-decomposed tasks into unnecessary steps. Once SAD introduced tool hints as anchors, accuracy recovered. The researchers conclude that aligning an agent to the vocabulary of specific tools can matter more than using a larger, more expensive model.

The Benchmark

To evaluate the framework, the team built CompSkillBench, a set of 300 compositional queries drawn from 2,209 real Model Context Protocol server skills across 24 functional categories. Transfer experiments showed the framework generalized even when target skill categories were absent from the retrieval pool, with a 35.6% relative gain in decomposition accuracy.

Business Implications

Token volume is the primary cost driver for any enterprise using frontier model APIs. A workflow that previously consumed 100,000 tokens per execution could fall below 1,000 with this approach. At scale, that gap determines whether complex AI automation is economically viable or prohibitively expensive.

SkillWeaver is not production-ready out of the box. The source code has not been released. The framework also lacks error recovery: if one step in a multi-tool chain fails, the entire execution fails. Alibaba’s team did publish prompt templates, and the retrieval index can be built quickly using existing libraries, so teams can begin testing the approach without waiting for an official release.

Read more: New Alibaba AI framework skips loading every tool, cutting agent token use 99%