AI rewrites AI: GPT-6 Astra patches Qwen weights and lifts math scores sharply

What it's about

In a blog post dated November 2, 2026, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab describe how current frontier language models are surprisingly good at reading and writing neural network weights directly. The tool was OpenAI's GPT-6 Astra. The authors call this "weight-space programming": a general-purpose model reads a network's weights, decides from a natural-language description what needs to change, and writes new values. It explicitly does not use gradient descent or a training loop. These are early results, and there has been no peer review. Cryptobriefing lists Takehi as affiliated with MIT.

Reading weights

In the first experiments, Astra received raw weights as a prompt.

• Identifying the game: Four networks with identical architecture, trained with PPO (a common reinforcement learning method). Astra named the correct game for all four without knowing the options. Clues included where the input weights are concentrated (in Breakout, at the bottom edge, where the paddle is).
• Diffusion model: Astra worked out whether a DDPM generates circles, triangles or squares, and drew them as outlines, even though the models produce filled shapes. The authors say this suggests it read the geometry of the weights rather than computing anything.
• Middle layers only: For three small language models (7.2M parameters) trained on Python, Romaji and English, Astra got only the four middle transformer blocks. In both trials it named the right language and found all induction heads (attention heads that repeat earlier text) without naming any false ones. It used a method from Elhage et al. that it hadn't been told about.

Writing and repairing weights

• Car at the ramp: A PPO network drove into a local minimum and got stuck at a ramp. In one trial without tools, Astra changed 3.5% of the weights, after which the car completed all 1,024 episodes. It added a small side path that first makes the car back up.
• From scratch: In three MinAtar games, Astra wrote networks with no data and no rollouts:

• Go: Astra wrote a network with 1.5M parameters (243 → 512 × 6 → 82) in a Python sandbox. With no forward passes, it won more than half its games against two smaller PPO networks. With 50 passes, it beat a PPO network of the same size trained on 20M games.

Qwen3.5-0.8B repaired

The key result: Qwen3.5-0.8B often falls into endless loops, which its own model card warns about. 85 of the first 96 practice answers ran to the limit of 32,768 tokens. Astra was allowed 10, 25 or 50 forward passes on prompts from GSM8K and MATH. It traced the loops to a direction written by the MLPs of layers 15 and 19, and subtracted it from 64 output columns each (131,072 weights, 0.016%).

The authors stress that this removes an obstacle but adds no capability. On Gemma 3 270M, which hits the limit in only 8.5% of cases, Astra found ordinary arithmetic errors with no single cause. MATH-500 moved only from 6.8 to at most 7.7%, which Astra itself says is not conclusive.

Limits and outlook

One experiment failed: Astra couldn't tell two Gemma 3 270M copies apart from the weights alone, when one had been trained on sycophancy (flattery, agreeing with the user). Only with attention patterns on self-written texts did it succeed, in all three pairs. There, middle heads paid more attention to the user's suggestion. The authors see bottlenecks in context length and the harness. They expect audits without evaluation (e.g. for backdoors), cheaper optimization, adding or removing capabilities without data, recursive self-improvement and model merging across architectures. A benchmark is in the works, and all logs are on GitHub.

Cryptobriefing covered the work under a "up to sixfold" headline and lists open questions: replication, transfer to larger models and families, and control and traceability of such interventions. According to the report, Qwen3.5-0.8B was released in March 2026 under Apache 2.0, and GPT-6 Astra on September 3, 2026.

Our take

The work is a proof of concept. That a language model understands weights semantically would matter for interpretability and model maintenance. But the big math jump is mainly a loop fix on a small model known for that problem, not a general math upgrade. It also depends on 50 forward passes; at 10 and 25, the effect was small. It's a single blog post, the results haven't been independently reproduced, and Astra is only available through OpenAI.