I ran a local AI model through 179 rounds of self-refine. It collapsed. One sampler setting fixed it.

An unguarded local model collapsed into repetition after 179 rounds of self-refine. Adding one sampler setting and rerunning the identical task produced 201 clean cycles instead.

Ben Rees - 11 August 2026

My Mac Mini has hours of spare overnight compute and nothing to lose by using it, so I ran the obvious experiment: let the local model I wrote about running a 35-billion-parameter model on it critique and rewrite its own output, over and over, and see whether iteration compensates for reasoning it doesn't have.

It didn't, the first time. By the end of the first run, the model had stopped attempting the task and was reciting its own instructions back at me. I found the specific, boring reason why, fixed it, and ran the identical experiment again. The second run held for the entire six and a half hours.

The setup

The task was brainstorming content angles, a breadth problem: many candidate ideas, not one correct answer, exactly what recursion is supposed to help with. I asked the same 35B model from the Mac Mini post to generate 15-20 ideas, critique its own list, produce an improved version, and repeat, each cycle's output feeding into the next cycle's prompt as "the current list to improve."

Self-refine is a real technique. It's just not free.

The method I ran is called Self-Refine: generate, get feedback from the same model, revise, repeat, no extra training required. Madaan et al. (2023) tested it across seven tasks on GPT-3.5, ChatGPT, and GPT-4, and found it improved output by roughly 20% on average. It works.

It worked on frontier models with strong single-pass reasoning to begin with. Mine is a 35B open-weight model, ternary-quantized to fit in 16GB of memory on a $599 machine, running through a script I wrote in an afternoon. I assumed the gap between those two setups would show up as diminishing returns. It showed up as active collapse instead.

What the first run actually looked like

Cycle 1 (97 seconds, no critique yet) was clean: 15 distinct, sensible ideas, no repetition. It was also the best output the model produced all run.

By cycle 40, items 1 through 5 were reappearing word for word as items 6 through 10, then again as 11 through 15. By cycle 80, one idea ended with the phrase "It cannot be defended" repeated eight times in a row. By cycle 179, the final output, the model wasn't attempting the brief at all. It opened with: "I will generate a list of 15-20 distinct content angle ideas. I will ensure each one is a specific, opinionated hook..." reciting its own system prompt back as if it were the answer.

There was no point after cycle 1 where the output was clearly better than cycle 1. The technique that adds 20% on GPT-4 subtracted, monotonically, from a 35B local model with no guardrails around it.

Why, specifically

Two things, neither of them "the model is too small":

I never configured a repetition penalty. Holtzman et al. (2020) showed that likelihood-based decoding on neural language models tends toward bland, repetitive text unless something in the sampling process actively discourages it, which is why their paper proposed nucleus sampling in the first place. I had nucleus sampling on. I didn't have a repetition penalty on. Once the model repeated a phrase, nothing pushed it away from repeating it again.

And every cycle fed the previous cycle's raw output straight back in as the thing to critique. Once that output contained a repetition loop, the model was refining text that already had the failure mode baked in.

The fix, and the rerun

Both of those are configuration choices, not facts about the model, so I changed both and ran the identical task again: same brief, same budget, same six and a half hours.

First, a repetition penalty (1.15, over the last 512 tokens), the setting I'd simply never set. Second, the loop no longer advances to a collapsed cycle, it retries from the last clean state instead of building on a broken one, with a hard stop after six bad cycles in a row.

201 cycles ran. Nothing stalled. Across the whole run, only two cycles ever crossed into repetition, and both recovered on the very next one without needing the retry logic to do anything.

v1 (copper) against v2 (teal), same task, same axes. v1 mean repetition score: 0.42. v2 mean: 0.019.

The honest part

I'd expected both fixes to matter roughly equally. They didn't. With only two brief wobbles in 201 cycles, the repetition penalty alone looks like it did nearly all the work, the more careful mechanism built to track and protect the best cycle barely fired. The expensive-looking fix was mostly decoration; the one-line setting was the actual fix.

Worth being equally honest about the ceiling here: "no more repetition collapse" isn't "flawless." A few outputs in the second run still ran on or read slightly garbled mid-sentence. The fix solved the specific failure mode I found, not every way a small model's reasoning can go wrong.

The actual lesson

The Mac Mini post's finding was that a local model confidently doesn't know things. This adds a second: when a local model breaks under its own recursion, check the boring defaults before blaming the hardware or the technique. Here it was one missing sampler setting, not a fundamental limit.

What's still unresolved is the more interesting question: 201 clean, distinct-looking cycles is not the same as 201 cycles worth reading. Whether that local breadth is actually worth anything next to one cloud pass on the same brief is a quality question, not a stability one, and stability was as far as this test went.

I wrote this up properly too, with the full methodology and all six stages of both runs, as a whitepaper.

More on this

Self-Refine Without a Repetition Penalty

A controlled case study: the same recursive self-improvement loop, on the same local model, six and a half hours each. One run collapses into repetition within forty cycles. The other holds clean for all two hundred and one. The difference is a single sampler setting.