The Same Brain, Another Pass

Think about editing an essay.

One way is to hand it to twelve different editors. Each reads it once and passes it on. That is a normal language model. Twelve layers, each with its own learned numbers, and your text goes through once.

The other way is to hand it to one editor who reads it twelve times. Same editor, same skills, every pass. The draft can get better. The editor never changes.

That second one is a looped transformer. Here is the thesis in a sentence: looping is the same brain, allowed another pass.

If it holds up, a model can think harder on demand without getting bigger. That changes what you buy and how you serve it.

I went looking into this because of my friend Dmitri. He is an AI researcher with a great habit: he drops a new concept or a new word into the AIMUG Discord and walks away, and it sends me off on a week of reading every time. This one landed in the same week that Jeremy Kahn at Fortune reported, citing The Information, that OpenAI’s Astra model uses recurrent depth. No loop count, no block size, no benchmark is public, so I am treating that as journalism rather than a technical source. But it made me want to understand the architecture properly.

It also connects to something I wrote last week. In The Router Was Right, a small router decides whether a model thinks before it answers. Looping is a different knob for the same decision, and this one lives inside the model.

What looping actually is

A standard transformer stores twelve layers of weights and runs your text through each one once.

A looped transformer stores one layer and runs your text through it twelve times, feeding the output back as the input. Both do twelve layers of work. One stores twelve layers. The other stores one.

That asymmetry is the whole idea.

There are three ways to make a model think longer.

1. Add layers. Costs weights and memory.
2. Loop the same layers. Costs compute, but no new weights.
3. Write out chain of thought. The model produces reasoning as text and reads it back.

The difference between the last two matters. Chain of thought is also a loop, but its written reasoning has to pass through the vocabulary and become words, and those words do not show every internal computation. Looping happens in latent space, in internal vectors that never become words. Geiping and colleagues argue in their abstract that this “can capture types of reasoning that are not easily represented in words.”

They are not either/or. You can do both, and some models ship a variant that does.

A worked example makes the storage point concrete. Huginn stores eight layers: two at the start, a four-layer block in the middle, two at the end. Run the middle block 32 times and it executes 2 + (4 x 32) + 2 = 132 layer passes. A standard model doing that much work would have to store all 132 layers.

The win: depth without parameters

The cleanest demonstration comes from Saunshi and colleagues at Google Research. They trained tiny models on i-GSM, a simplified, symbolic version of grade-school math problems.

Read it left to right. A one-layer model, run once: 24.5 percent. A one-layer model trained to loop eight times: 73.2 percent. A model with eight separate layers and eight times the parameters: also 73.2 percent. These are separately trained models, not one checkpoint run longer, so this is a result about training with recurrence, not about turning a knob at answer time.

Their Table 1 shows the same shape on adding 32 numbers. One layer once gets 0.0 percent. A one-layer model trained to loop twelve times gets 99.6. Twelve distinct layers get 100.0.

The second win: turn the knob at answer time

Huginn is a 3.5B-parameter model trained on 800B tokens (a token is a small unit of text, often a word or part of one). During training the loop count was drawn at random each step, averaging 32. That detail matters: it was not trained at 32 loops, it was trained across a range centered there.

Every point is the same checkpoint with the same weights. Going from 4 loops to 32 adds about 21 points on ARC-Easy, a grade-school science benchmark. The others gain between 4 and 22 points, most of them 8 to 14. On most benchmarks nearly all of it arrives by 16 loops and the curves flatten after that; WinoGrande is the exception, with about 40 percent of its gain landing after 16. SciQ actually slips from 93.90 to 93.50.

That is published evidence of diminishing returns in a real model, which is rarer than it should be.

On competition math, Huginn at 32 loops, without a system prompt, scores 12.58 on Minerva MATH against 5.62 for OLMo-7B-0724, a 7B model trained on 2.75T tokens. Half the parameters, under a third of the data, roughly twice the score. OLMo-2, trained on 4T tokens, still beats it at 19.08. And this is a score-per-token observation across models with different training mixes (Huginn’s leans toward math and code), not a controlled experiment, so read it as suggestive.

The twist: parameters store facts, depth manipulates them

This is the chart that changed how I think about the whole idea.

Saunshi and colleagues trained 1B-scale models where every looped variant has 24 effective layers. The dashed line is a normal 24-layer model.

On the left, perplexity, which measures how well a model predicts ordinary text. Looping helps, but never catches up. It closes only 34 to 48 percent of the gap. The baseline sits at 7.40 and the looped configurations run 7.90 to 8.79.

On the right, reasoning primitives, the paper’s suite of small synthetic reasoning tasks. Every looped model beats the 24-layer baseline. The four-layer version looped six times scores 56.9 against 47.5, with one sixth of the parameters.

The paper calls this “an interesting dichotomy between reasoning and memorization.”

What convinced me it is real is that a different lab found it independently a year later. The Ouro authors write that their advantage “stems not from increased knowledge capacity, but from superior knowledge manipulation capabilities.”

So here is my working rule, and it is a tendency, not a law. If your workload is recall-heavy, trivia and facts and memorized knowledge, looping tends to be the weaker trade. If it is manipulation-heavy, multi-step math and code and chains of logic, it tends to be the stronger one. Looping still helps recall against a shallow model with the same parameters. It just does not replace a deeper one.

Parameters store facts. Depth manipulates them. They are different resources, and in these results you buy them separately.

The limits: looping past your trained depth is not free

Ouro is a 1.4B and 2.6B open-weight model trained on 7.7T tokens. Its training started at 8 loops, hit loss spikes, and was cut to 4 loops for the rest of training, so 4 is the depth it finished at. Its tables run the same checkpoint from 1 to 8 loops.

Accuracy on MMLU, a 57-subject exam, climbs steeply to loop 4, the depth it was trained at, then drifts down.

Compare that to Huginn, trained with a loop count drawn from a heavy-tailed distribution averaging 32, so it occasionally saw much deeper passes. Its model card says benchmarks keep improving to about 64 loops, given enough context to reason about. Ouro finished training at 4 and turns over at 5.

The difference looks like training-time depth rather than architecture. And extrapolation is possible: Kohli and colleagues get harder problems solved by adding loops at inference in their setting. The question is when it is reliable at language-model scale.

On Ouro-2.6B-Thinking, the variant tuned for reasoning, on these benchmarks, the turnover is not a gentle drift. It is a cliff.

OlympiadBench, olympiad-level math and science problems, goes 76.44 at four loops to 39.26 at eight. AIME 2024, a hard high-school math contest, goes 70.33 at three loops to 39.00 at eight. Roughly half the score, from nothing but extra thinking.

Kohli and colleagues name this “overthinking, where excessive recurrence degrades predictions.”

So “more loops means more thinking” is a useful picture, not a law.

So the open question is whether overthinking is built into the architecture or is an artifact of how these models were trained. The Huginn and Ouro pattern points at training depth. But the controlled experiment, one architecture trained at several depths at scale, has not been run, so that is a hypothesis.

The fair fight, and how to read any claim

Here is the thing that took me longest to see.

Most looped-transformer papers compare at fixed parameter count. That flatters looping, because the looped model is doing more compute to get its result. It answers a memory-constrained question, what you get from a fixed number of weights. It does not answer the compute question. Both are legitimate. They are different.

Three 2026 papers ask the harder question.

Schwethelm and colleagues fit a scaling law across loop counts and measure what they call a recurrence-equivalence exponent of 0.46. In plain terms, looping a block four times is worth more than nothing and less than four new blocks. Their example: a 410M model looped four times performs like a 580M model, but costs like a 1B model to train. For dense models on plain language-modeling loss, that is a loss.

The Loopie authors built looped models at 20B total parameters with 2B active. Their abstract states the historical problem plainly: “given an N times increase in pre-training compute, increasing the parameter count by a factor of N usually outperforms looping a model N times.” The next sentence claims Loopie “substantially outperforms vanilla Transformer baselines trained with the same compute budget.” I have not verified that table, so I am quoting the problem statement, not endorsing the solution.

I still find the problem statement persuasive. The people building these models say the default has been to lose, and that is the bar every result in this space has to clear.

SMELT is the strictest study I found. It approximately matches per-token compute, total parameters, and cache size, within a few percent, and scales to 54B total parameters. Its fitted saving in training compute, at equal validation loss, is 6.8 to 18.0 percent, with bootstrap intervals as wide as 4 to 28 percent. Real, careful, and modest, and only after redesigning the whole architecture around the loop.

What this means depends on where you sit.

If you serve models, looping is a per-request knob for how hard to think, with no model swap. It trades weight memory for serial compute: more loops means more time per token, and the attention cache grows with loop count unless it is shared.

If you train models, dense looping loses to adding parameters at fixed compute, on language-modeling loss, in the ranges tested so far. The wins come from redesigning around the loop, and they come with stability work that the 2026 papers are still sorting out.

If you are reading a hype thread, find the baseline. “Beats a 12B model” with different training data is not an architecture result. “Same checkpoint, more loops” isolates one thing: extra compute at answer time.

So the rule I now use on any claim in this space:

Ask what was held constant. Fixed parameters flatters looping. Fixed compute is the fair comparison. Fixed wall-clock latency tends to disfavor it against a same-size shallow model, because the loops add serial work. If a paper does not tell you which one it held fixed, that is the first thing to go find.

Three questions are genuinely open.

1. Is overthinking built in, or a training artifact?
2. Does looping beat parameter scaling at fixed compute at frontier scale? Dense, on language-modeling loss in the tested range, says no. Redesigned mixture-of-experts, where each word uses only a few of many sub-networks, says a modest yes, and nobody has shown it past 54B total parameters.
3. Does reasoning in latent vectors help or hurt safety monitoring? The Ouro authors argue looped traces track the final answer more faithfully than written chain of thought. The researchers quoted by Fortune worry that humans cannot read them. Both can be true, and there is no shared benchmark between them.

Looping is not a bigger brain. It is the same brain, allowed another pass. That is worth a lot. It is not worth everything.