Summary: A paper published in October 2026 by researchers at Johns Hopkins University and Apple ML Research proposes SelfCompact, a scaffold that lets a language model agent decide for itself when to compress its own accumulated context. Instead of summarizing on a fixed token schedule, the agent calls a compaction tool guided by a short rubric — and matches or beats fixed-interval compaction at a fraction of the token cost, with no fine-tuning.
The problem with summarizing on a timer
Long agent runs accumulate junk: failed tool calls, dead-end chains of thought, partial results from searches that went nowhere. Left alone, this stale content anchors later generations and eventually blows past the context window. The common fix is a compaction trigger tied to a token threshold — summarize every N tokens, no questions asked.
The SelfCompact authors point out the flaw in that approach: the trigger ignores trajectory structure. Compact at the wrong moment and you discard a half-finished derivation or an in-progress search. The schedule is blind to what the agent is actually doing.
A tool plus a rubric, not a fine-tune
SelfCompact pairs two inference-time pieces. First, a compaction tool the model can invoke to summarize accumulated context. Second, a lightweight rubric that tells the model when firing makes sense — a sub-task has resolved, or the trajectory is converging — and when to suppress it, such as mid-derivation or when stuck.
The interesting finding is that both halves are necessary. According to the paper, the tool alone gets used unevenly across open-weight models — invoked at unhelpful moments or skipped entirely. The rubric alone has no way to act. Together they produce effective adaptive compaction with zero training or external supervision. That’s the part worth stealing: a short written policy around a tool changed model behavior that no amount of fine-tuning was applied to achieve.
What the numbers say
The evaluation covers six benchmarks across competitive math and agentic search, run against seven models. Reported results:
- Up to 18.1 points better than a no-summarization baseline on math tasks.
- 5–9 points better on agentic search.
- 30–70% lower per-question token cost than fixed-interval summarization, at equal or better quality.
The cost angle matters if you’re already thinking about long-context economics — the KV cache sizing tradeoffs we looked at in what a 1/8-size KV cache means for your agent bill push in the same direction: tokens you never carry are tokens you never pay for.
The meta-cognitive gap
The sharpest insight in the paper is what the authors call a meta-cognitive gap: unprompted models cannot reliably tell when their own context is rotting. They don’t notice the stale tool output poisoning the next step. But a lightweight rubric closes that gap — meaning “when to compact” is a capability a scaffold can supply rather than something the model must learn.
For anyone building agent harnesses, that reframing is practical. You don’t need to train a model to have good judgment about its own memory; you need to write the judgment down and hand it over as a rule. The same pattern shows up wherever agent behavior depends on timing decisions — when to delegate, when to stop, when to persist state. The lesson generalizes: externalize the policy, give the model the action, skip the training run.
The honest caveat is scope. These are benchmark results on math and search tasks with seven models; how the rubric transfers to messier production workflows — long customer-support sessions, multi-day coding agents — isn’t demonstrated in the paper. If you’re building an agent with compaction today, the cheap experiment is obvious: replace your fixed token threshold with a compaction tool plus a few lines of when-to-fire guidance, and watch both your token bill and your mid-derivation failure rate.