MIT Demonstrates Weight-Space Programming as GPT-6 Astra Directly Reads and Writes Neural Network Weights

A research team from MIT CSAIL unveiled experimental results for Weight-Space Programming, an approach enabling a general LLM—GPT-6 Astra—to analyze and directly modify the numerical weights of other models without relying on traditional training or gradient updates. In the Qwen3.5-0.8B experiment, Astra inspected weights in a Python environment and analyzed up to 50 batches of inference feedback from GSM8K and MATH training sets. Astra pinpointed repetitive looping behavior to multi-layer perceptrons (MLP) in layers 15 and 19, editing roughly 130,000 weights (about 0.016% of the total). Following this modification, Qwen's accuracy on MATH-500 surged from 7.9% to 48.1%, GSM8K rose from 13.8% to 67.1%, and IFEval instruction-following jumped from 21.0% to 62.3%. Researchers also demonstrated Astra's ability to read and write weights for small neural networks from scratch based on game rules and source code, outperforming a PPO model trained for 100 million steps across three games, including Breakout. However, Astra failed to significantly improve math benchmarks on Gemma 3 270M, which exhibited fewer repetition loops. The team has made experimental records and modified weights publicly available.

Key takeaways

• MIT CSAIL used GPT-6 Astra to edit about 0.016% of Qwen3.5-0.8B weights without retraining, resolving repetitive looping issues.
• Post-modification benchmark scores improved sharply: MATH-500 from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval from 21.0% to 62.3%.
• Astra synthesized weights from scratch using game rules, outperforming a PPO reinforcement learning model trained for 100 million steps on games like Breakout.
• The method showed no clear math improvements on Gemma 3 270M, which had fewer loop issues; experimental records and weights have been open-sourced.