When a model starts evaluating its own work, the usual failure mode is subtle: it gets better at agreeing with itself. Sakana AI’s new paper, Multi-Agent Self-Supervision (MASS), published October 12, 2026, attacks exactly that problem — and the mechanism they chose is one product builders should pay attention to, because it treats agent orchestration as a training-data strategy rather than a runtime trick.
The core idea: learn from the team, not just the task
MASS uses one shared model that solves tasks through a team of virtual subagents. The system searches for better multi-agent workflows, the model judges which executions are best, and then training on those best executions distills the team’s collective experience back into the single shared model.
The reasoning behind it comes from a practical observation the Sakana AI team makes in the announcement: as AI takes on more open-ended research problems, human experts may struggle to assess results and guide progress. The model itself may be the best available optimizer and evaluator. The hard part is getting useful self-supervision inside that loop without the model simply reinforcing its own blind spots. A team of agents with different roles creates more diverse signals than a single model talking to itself.
What the two cycles actually showed
The experiments were run on a 27B open-weights model over synthetic open-ended research tasks, for two cycles. After the second cycle, score per output token reached 1.2-1.6x the base model’s level across four research benchmarks.
Three results from the paper stand out for anyone building agent systems:
- Training only on task-solving data from the optimizee improved both the optimizer and the evaluator — what they call role generalization within a recursive self-improvement loop.
- Multi-agent trajectories turned out to be more efficient training data than single-agent trajectories for RSI.
- Optimizer capacity, not evaluator capacity, can be the bottleneck in how fast improvement happens.
That last point is counterintuitive. A lot of current effort goes into making evaluators more reliable. If the optimizer is the real bottleneck, that changes where you’d invest.
The safety caveat they put front and center
Sakana AI is explicit that homogeneous RSI — a model serving as its own evaluator — deserves careful community scrutiny. When the same model generates and judges the work, shared blind spots could let mistakes get accepted as progress and then reinforced through training. Their stated position is that improving beyond human expertise safely requires understanding these failure modes and keeping ways to detect and intervene in them.
For builders, this maps onto an evaluation question you probably already face: who grades your agent’s output? If it’s the same model family doing both jobs, you’re running a small-scale version of the homogeneous loop. We’ve touched on this gap before when looking at how ATLAS scores search agents that genuinely depend on search — independent benchmarks matter precisely because self-reported quality can drift.
What to take from it
The paper and code are public — the code lives on the SakanaAI GitHub and the paper is on arXiv (2610.12176) — so the findings are reproducible rather than a product pitch. The work came from Hyunin Lee, Jinglue Xu, Jeffrey Seely, Yujin Tang, and collaborators at UC Berkeley including Somayeh Sojoudi, Matei Zaharia, and Donghyun Lee.
The practical takeaway: if multi-agent trajectories are more efficient training data than single-agent ones, the orchestration patterns you’re building today aren’t just runtime plumbing — they’re a candidate dataset format. Worth keeping your subagent trajectories around. The limitation to hold onto: two cycles on synthetic tasks is early evidence, and the safety questions the authors raise about self-evaluation are still open.