MIT Researchers Use GPT-6 Astra to Edit AI Weights Directly, Boosting Math Scores Sixfold

Quick Facts

• GPT-6 Astra edited 131,072 parameters out of 800 million in Qwen3.5-0.8B, lifting its MATH-500 score from 7.9% to 48.1%.
• MIT CSAIL researchers call the approach "Weight-Space Programming" — an LLM reads and rewrites a model's weights directly, with no gradient descent or retraining.
• The technique also produced game-playing networks from scratch that outscored PPO agents trained for 100 million steps across three MinAtar games.

Researchers at MIT CSAIL published a blog post on November 2 describing what they call Weight-Space Programming, a method in which a general-purpose large language model reads and rewrites another model's numerical weights using natural language reasoning, with no optimizer, no training data, and no gradient feedback.

The team — first-year PhD student Rikiya Takehi, Jayden Teoh, Alex Zhang, and Professor Omar Khattab — used OpenAI's GPT-6 Astra as the editing agent. Their headline experiment targeted Qwen3.5-0.8B, a small open-source model whose own documentation flags a tendency toward reasoning loops on math tasks.

How the Edit Worked

Astra spent 20 hours reviewing the model's 800 million parameters. It traced the looping failure to specific directions in layers 15 and 19, then edited 131,072 weights to remove them. That patch covers 0.016% of the total parameter count.

The result: MATH-500 accuracy climbed from 7.9% to 48.1%, a 40.2-point gain. GSM8K accuracy rose from 13.8% to 67.1%. The IFEval instruction-following benchmark improved from 21.0% to 62.3%.

The researchers are explicit about what drove the gain. Astra did not add mathematical knowledge. It removed a structural defect — repetitive output loops that consumed the model's permitted output length before it could answer. The improvement generalized across benchmarks because the underlying failure mode was fixed, not patched for one test.

The Limits Are Real

On Gemma 3 270M, a model that rarely loops, Astra could not improve math performance at all. The team also found Astra could not identify a sycophantic model from its weights alone. The technique works where there is a discrete, locatable failure to remove. It does not yet add capability where none exists.

The post reached 1,857 points on Hacker News within a day of publication and spread simultaneously across r/MachineLearning, r/singularity, and r/LocalLLaMA. Software engineer Sean Goedecke published a counter-argument the same day, contending that the approach looks more impressive than it is and that the underlying weight manipulation is not as superhuman as framed.

Other Experiments in the Paper

The Qwen result is one of several. In a reading task, Astra examined 17,253 weights from four neural networks trained on Atari games and identified which game each network played with 99% confidence, using no labels. It inferred the Breakout network because first-layer weights concentrated on the paddle position and the output layer favored left-right actions.

In a writing task, Astra produced game-playing networks from scratch in a single zero-shot attempt across three MinAtar games, beating PPO agents trained for 100 million steps in all three. In Breakout specifically, PPO at 100 million steps scored 15.8 and PPO at 500 million steps scored 25.3. Astra's one-shot network scored 517.4.

A separate repair experiment fixed a car controller trained with PPO that had fallen into a local optimum on a slope. With one round of dialogue, Astra modified 3.5% of the controller's weights. The repaired version succeeded in all 1,024 subsequent tests.

What It Means for AI Development

Standard model improvement requires more training data, more compute, and a gradient-based optimizer running thousands of steps. Weight-Space Programming, as described here, skips that loop. Astra reasons in natural language about what a layer computes, then writes a numerical correction directly, the way a developer might patch a function rather than rebuild a codebase.

Khattab posted on X: "you thought LLMs can only do text optimization and discovery for prompts, harnesses, and code? turns out they're starting to be good at natively reasoning about and writing out neural network weights directly too." He asked where that capability leaves symbolic local optimization methods like gradient descent.

Takehi noted in the post that the technique may help avoid catastrophic forgetting, a problem that arises when revising a model causes it to lose previously learned behaviors. The post is an early-stage research report, not a peer-reviewed paper, and the team presents it as a preliminary signal rather than a finished system.