Meta and UIUC Train an 8B Model to Match Claude Opus 4.5 at a Fraction of the Cost

Quick Facts

• A Qwen3-8B model trained with EvoHarness-RL hit a 96.9% success rate, matching Claude Opus 4.5’s 96.4% on the ALFWorld benchmark.
• The framework uses a two-stage training process combining supervised fine-tuning with cost-aware reinforcement learning.
• Equipping frontier models like GPT-5 with the BPE harness alone boosted their success rate by 25.7 percentage points without any additional training.

Researchers from Meta AI and the University of Illinois Urbana-Champaign have trained an 8-billion-parameter model to match the task performance of Anthropic’s Claude Opus 4.5, one of the most capable closed models available. The method, called EvoHarness-RL, gives AI agents a structured way to manage memory, track progress, and reuse past experience across long, multi-step tasks.

The trained Qwen3-8B model achieved a 96.9% average success rate on the ALFWorld benchmark, a 49.0 percentage point jump over its baseline ReAct configuration. Claude Opus 4.5 scored 96.4% out of the box. For software teams running high-volume agent workloads, that gap in model size and cost is significant.

The Problem With Manual Agent Design

Long-horizon AI agents need external support to stay on track across complex tasks. They rely on tools to maintain state, invoke actions, verify outcomes, and recall past attempts. Most teams build this support by hand, through prompts, heuristics, and domain-specific code.

Xuying Ning, a co-author of the paper and UIUC researcher, told VentureBeat that this approach breaks down quickly. “The optimal harness often changes with the model,” Ning said. “Different models may need different prompts, memory designs, permissions, or sandbox configurations. If all of this logic is manually coded, every model upgrade can lead to another long cycle of tuning and debugging.”

Ning also flagged a flaw in how most agents handle memory. “Append-only memory assumes that more context is always helpful, which is not necessarily true,” she said. “Over a long task, the memory may contain outdated conclusions, failed attempts, or information that is no longer relevant.”

How EvoHarness-RL Works

The framework centers on a unified interface the researchers call the Belief, Progress, and Experience system, or BPE. It organizes an agent’s external needs into three categories: Belief tracks the current state of the environment, Progress manages completed and pending subgoals, and Experience stores historical knowledge for reuse. Agents interact with this system using four compact actions: track, commit, recall, and note.

Training happens in two stages. First, supervised fine-tuning teaches the base model how to extract and structure useful information from messy interaction logs into the BPE format. Second, cost-aware reinforcement learning trains the agent to decide when querying external memory is worth the computational cost. The result is a model that treats tool use as a learned behavior rather than a hardcoded rule.

Emergent Behaviors Cut Costs Further

Two behaviors emerged during training that have direct implications for production deployments. The first, which the researchers call harness annealing, describes a pattern where the agent initially queries its memory and progress trackers frequently, then steadily reduces those calls as it masters routine actions. By the end of training, the agent averaged roughly one harness call per episode. Fewer calls mean lower latency and reduced token costs at runtime.

The second behavior, harness evolution, shows the agent dynamically adjusting its memory strategy based on task complexity. Simple tasks get minimal scaffolding. Complex tasks get richer state tracking. The agent learns to match its tool use to the situation.

Results Beyond the Headline

EvoHarness-RL outperformed other trainable agent frameworks tested in the study. SkillRL scored 89.9% and SkillOS reached 80.2%, both below the trained 8B model’s 96.9%.

The BPE harness also delivered gains when applied to frontier models without any reinforcement learning. GPT-4.1’s success rate improved by 22.1 percentage points and GPT-5 by 25.7 points when equipped with the BPE prompt-time interface alone. That means teams using closed APIs could see meaningful performance gains without retraining anything.

The research team includes contributors from both institutions: Xuying Ning, Tianxin Wei, Yuanchen Bei, and others from UIUC, alongside Dongqi Fu, Hanqing Zeng, Qifan Wang, and additional researchers from Meta AI.

For technology leaders evaluating AI agent infrastructure, EvoHarness-RL points toward a concrete alternative to frontier model dependence. A small, trained model running efficient harness calls can now deliver results that previously required paying for the most expensive closed APIs available.

Read more: Meta researchers taught an 8B AI model to match Claude Opus 4.5 — without the frontier price tag