One Open Model, Three Jobs: Sakana's MASS Teaches Itself
Sakana AI and UC Berkeley let a single 27B open-weights model design its own agent teams, judge the results and train on the winners. Two laps in, it gets more done per token, and the weak link is not the judge.
Most self-improving AI systems quietly lean on a stronger model somewhere in the loop: a frontier grader, a human rater, a larger teacher. A paper posted to arXiv on October 8 by researchers at Sakana AI and UC Berkeley removes that crutch. One open-weights model does every job, and it still gets better.
The method is called MASS, short for Multi-Agent Self-Supervision. The model is Qwen3.6-27B. The code is public on GitHub under the SakanaAI organisation, and the paper, "Recursive Self-Improvement through Multi-Agent Self-Supervision," lists Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Somayeh Sojoudi, Matei Zaharia and Yujin Tang as authors.
One model, three hats
The repository's own summary is the cleanest description: "MASS alternates between improving workflows and learning from them."
Each cycle has two halves. First, with the weights frozen, the model searches for better multi-agent workflows. It acts as the executor that runs them, the optimizer that proposes edits to them and the in-loop evaluator that decides which output is better. An evolutionary search refines the workflows under structural constraints.
Second, the system collects trajectories from the workflows it selected, ranks them and fine-tunes on its own preferred runs. The README describes a pipeline of search, collect, rank, select, render, train and export, with the lowest-validation-loss checkpoint kept. The next cycle starts from the updated model. The team ran two cycles, the second training a fresh LoRA adapter on the merged first-cycle model.
External judges appear only for reporting results, not inside the training loop. That is the point. The paper targets open-ended work where human experts cannot reliably grade outputs, so the loop has to supervise itself.
What two cycles bought
The paper's headline claim is 1.2x to 1.6x higher score per output token on four open-ended research benchmarks: ScienceAgentBench, MLR-Bench, DSBench and AstaBench. Per-token efficiency is the right lens for agent systems, where cost scales with how much a model writes.
The raw scores in the README show where the gains land. On ScienceAgentBench, success rose from 28.8% to 31.7% across the two cycles. On MLR-Bench, the score went from 1.71 to 2.25, and the share of runs that actually delivered a paper jumped from 57% to 76%. AstaBench (E2E-Bench-Hard) moved from 0.053 to 0.067, with delivery rising from 50% to 76%.
That delivery number may matter more than the scores. A research agent that finishes three runs in four instead of one in two is a different tool.
The README is also candid about the misses. DSBench is roughly flat. Terminal-Bench 2.0 slipped from 0.375 to 0.352, and SWE-bench Verified from 0.626 to 0.611. Those drops sit within or near the reported error bars, but the direction is clear enough: training on self-chosen research trajectories did not make the model a better software engineer. Self-improvement here is aimed, not general.
The bottleneck is the optimizer
The most useful finding sits in the analysis. A natural assumption is that a weak model's weakest job is judging: if it cannot tell good work from bad, the loop collapses.
The authors argue the opposite. When they swapped only the evaluator for GPT-5.5, performance improved less than when they swapped both the evaluator and the optimizer. Their conclusion is that optimizer capacity, not evaluator capacity, is what limits the speed of self-improvement.
They add two details that sharpen it. The all-base-model loop eventually reached unanimous wins on 11 of 12 synthetic tasks, so the 27B model can find good workflows; it is just slow. And the size of a textual edit to a workflow barely predicted whether it helped. Proposing the right change, not recognising a good result, is the hard part.
For anyone building agent frameworks, that is a practical pointer. If you have a budget for one stronger model, spend it on whoever rewrites the plan, not whoever grades it.
Teams make better teachers
A second result supports the multi-agent premise. A student trained on first-cycle multi-agent traces reached a 68.3% win rate against the base model using about 28.8 million supervised tokens. A student trained on single-agent traces reached 64.0% with about 42 million, roughly 1.4x more data. Multi-agent runs produced about 2.4x more tokens on average, yet external judges preferred them 85.1% of the time. The authors frame this as a result from their runs, not a general law.
The first-cycle fine-tune itself was modest: about 19 hours on six Nvidia H100s, a figure that excludes workflow search, trajectory collection and evaluation.
What it does not show
The limits are stated plainly. There are no formal guarantees; the retained workflow is the best under the model's own sequential comparisons, not a certified optimum. Two cycles is a short run, and the paper names faster per-cycle gains as an open problem. The README notes the released driver differs slightly from the paper's algorithm and does not include every ablation.
Still, the shape of the result is new. A mid-sized open model, with no stronger teacher in the loop, built better teams for itself, learned from them and came out more efficient on research tasks. The next question is the one the ablation raises: give the optimizer seat a bigger brain, and does the loop speed up or just get more expensive? The code to find out is already public.