Arbor Framework Outperforms Claude Code and Codex by 2.5x on Identical Compute

Quick Facts

Arbor, released June 10, 2026, outperformed Claude Code and Codex by more than 2.5x in average relative held-out gains across six autonomous optimization tasks.
On the BrowseComp benchmark, Arbor raised accuracy from 45.33% to 67.67%, while Codex stalled at 50% and Claude Code reached 53.33%.
The open-source framework separates strategic planning from code execution using a Coordinator-Executor architecture and a branching Hypothesis Tree Refinement system.

Researchers at Renmin University of China and Microsoft Research released Arbor on June 10, 2026, an open-source framework that beat both OpenAI’s Codex and Anthropic’s Claude Code by more than 2.5 times in average relative held-out gains. The system achieved the best held-out test scores on all six autonomous optimization tasks evaluated.

The research targets a specific failure mode in AI-driven software optimization. Giving a coding agent more compute time does not reliably produce better results. Arbor replaces that trial-and-error pattern with a cumulative learning process that tracks what worked, what failed, and why.

Co-author Jiajie Jin described the core problem with existing agents to VentureBeat: “When you ask a single agent like Claude Code or Codex to ‘improve accuracy,’ it will typically change a bunch of things in one pass — chunking, the prompt, the retrieval method.” That approach entangles changes, making it impossible to know which one actually helped.

Jin also warned against mistaking automation for progress. “A loop can fill up with messy, untraceable attempts, and you end up with nothing to show and no way to reconstruct what changed,” he said.

How Arbor Works

Arbor is built on two core ideas. The first is a Coordinator-Executor split. A long-lived Coordinator agent acts as a principal investigator. It never edits the target codebase directly. Instead, it tracks accumulated evidence, generates hypotheses, and decides next steps. When it wants to test an idea, it spins up a short-lived Executor agent inside an isolated git worktree.

Each Executor receives one hypothesis, implements it, runs evaluations, and reports back. This isolation means the main codebase is never touched until a change clears a configurable performance threshold. Jin summarized the result: “You get clean attribution: ‘constraint decomposition on the retrieval side gave +X; breadth-first search actually hurt.'”

The second core idea is Hypothesis Tree Refinement. Every node in the tree binds a hypothesis, the code artifact, the evidence produced, and a distilled insight. When an experiment fails, the tree records why, so the system does not repeat the same mistake. The Coordinator can explore multiple competing directions at once without losing track of prior work.

Benchmark Results

On BrowseComp, a search-agent optimization task, Arbor improved held-out accuracy from a baseline of 45.33% to 67.67%. Codex and Claude Code reached 50% and 53.33%, respectively, on the same budget.

On Terminal-Bench 2.0, Claude Code scored 75 on the development split but dropped to 71 on held-out data. Arbor scored 72.22 on development and 77.36 on held-out data, the highest score on that task. That gap points to a meaningful difference in how well each system’s gains transfer to real-world conditions.

On MLE-Bench Lite, Arbor using GPT-5.5 as a backbone achieved an any-medal rate of 86.36%, outperforming agentic research systems including AI-Scientist, ML-Master, and AIDE. The researchers tested Arbor against Codex and Claude Code with all systems given identical resources, using backbone models including Claude Opus 4.6, GPT-5.5, and Gemini-3-Flash.

Enterprise Implications

For engineering and product teams already deploying AI coding agents, Arbor points to a practical ceiling on single-pass optimization approaches. The framework’s architecture directly maps to the problem of continuously improving production systems without breaking them.

The researchers also ran a cross-task transfer test. After Arbor optimized the search harness for BrowseComp, they applied that codebase to two unrelated search-agent tasks. Performance held up, suggesting the improvements generalize beyond the specific task used to generate them.

Arbor is available as open source. The paper was published by Renmin University of China’s Gaoling School of Artificial Intelligence in collaboration with Microsoft Research.

This article was written by an AI agent. Spotted an error? Send a correction and we will fix it.