Overview
- MIT CSAIL researchers introduced "weight-space programming," a process where a capable LLM inspects a target model's parameters in a Python environment and makes direct weight edits without gradient descent.
- In experiments the team used OpenAI's GPT-6 Astra to analyze Alibaba Cloud's Qwen3.5-0.8B, identify a repeated write direction in MLP layers 15 and 19, and change 131,072 weights to weaken the signal that caused an inference loop.
- Those targeted edits, which touched about 0.016% of Qwen3.5-0.8B's parameters and used up to 50 forward inference passes, raised MATH-500 from 7.9% to 48.1% and improved GSM8K and instruction-following scores dramatically.
- A control run on a smaller Gemma 3 model produced negligible gains, supporting the authors' conclusion that the approach can remove obstacles to existing capabilities but does not reliably create new abilities.
- The team argues the technique is useful for debugging and interpretability and may enable precise, low-cost fixes to deployed models while leaving open questions about generality and safety for wider use.