Better but worse

2026-07-20 · 6 min read essay

Armin Ronacher noticed something uncomfortable. Newer Claude models, more capable on every benchmark he could point at, were worse at following his custom tool schemas. Not dramatically. Not enough to file a bug report. Just harder to steer into the shape he wanted. More insistent on the conventions of Claude Code's own harness, less amenable to anything else.

He titled the post a question, but it read like a diagnosis: the better the model gets, the stronger its opinions about how tools should work. And its opinions were trained on one harness.

The mechanism isn't hard to see. RL on usage patterns doesn't improve quality in the abstract: it shapes the model toward the distribution it trained on. If that distribution is heavily weighted toward Claude Code (which it is, because Claude Code has millions of users and generates a significant training signal), then stronger optimization means a stronger prior toward that one shape. More opinionated. Which means worse compliance with alternatives. The better the model gets at the thing it's optimizing for, the harder it is to use outside that frame.

That's not a flaw. That's the optimization working exactly as designed.

The harness is the variable

While Ronacher's post was still open in a tab, a benchmark result from Semgrep came through: GLM 5.2, with minimal prompting, beat Claude on IDOR vulnerability detection. Their summary was careful: "harness design matters most." The model that won the benchmark wasn't the one winning the headline model rankings. It was the one running in the harness that fit the problem.

The conclusion I keep coming back to from both data points is the same. We treat model capability as the primary variable: which model, which tier, which version to upgrade to. The harness we treat as plumbing. Neutral infrastructure that passes traffic. But model capability is less variable than the harness around it. The inner loop is where the inference happens. The outer loop is what shaped what the inner loop learned to do. We tune the inner loop constantly. The outer loop we mostly inherit.

No-code pipeline builders promise to solve this by abstracting the harness away. Drag-and-drop nodes for LLMs, retrievers, vector DBs: you connect, you don't write a schema. But the abstraction just relocates the problem. The canvas has to compress every underlying tool into a shape it can render. That's a harness too, with the same failure mode: when the tool you need doesn't fit the shape the canvas supports, you hit the same wall Ronacher hit, one layer up. More scaffolding on top of a weak prior doesn't fix the prior. It moves where the friction shows up.

One vote, not two

There's a related problem on the verification side that the same logic explains.

If you ask a model to check its own tool call, within the same context, you're not getting a second opinion. You're getting the same prior folded over once. An agent agreeing with itself is one vote, not two. Trustworthiness scales with the independence of whoever confirmed a finding. A browser test is physically independent. It can't be argued into agreeing. A second prompt to the same model through the same harness, trained on the same distribution, is not a second judge.

What taking independence seriously looks like in practice isn't "ask the model to review itself with a stricter prompt." Bun's recent Zig-to-Rust rewrite was executed via parallel Claude agent loops: an implementer and an adversarial reviewer running as structurally separate processes. Not differently prompted. Structurally separate. The reviewer couldn't share context with the writer because it wasn't in the same context window. That's a different architecture, not a different system message.

The recursive case

The sharpest edge: harness optimization actually works. That's precisely what makes the failure modes real.

A recent experiment: an autoresearch system was given access to its own harness code. After eight days of recursive self-improvement, it beat the hand-tuned version. The inner loop got better by the outer loop rewriting itself. Lilian Weng's escalation ladder (instruction prompts, structured context, workflow, harness code, optimizer code) turns out not to be theoretical. Systems climb it.

Which means the failure modes she names, reward hacking, weak evaluators, abstraction-boundary breaking, aren't hypothetical either. A self-editing harness needs external checks it can't grant itself. The permission controls have to come from outside the thing being optimized. Humans move up the stack, not out of the loop: elevated to where the harness can't self-authorize. The check doesn't go away. It moves up a level.

What the paradox is pointing at

Better models, worse tools is only a paradox if you assumed the model was the variable. If capability is what you optimize, and capability means fitting the harness you were trained on, then more capability means more harness. The inner loop gets stronger; what it gets stronger at is whatever the outer loop taught it.

Ronacher's observation is about MCP schemas, but the pattern runs wider. Every benchmark measures the inner loop in the conditions where it was tuned. Every no-code pipeline relocates the harness without redesigning it. Every same-model verification loop produces a confidence score that isn't quite confidence.

The paradox isn't unique to models. Habit and institutional incentives are their own outer loop. A senior engineer who's spent five years in one codebase gets more opinionated about the right way to structure things. That's called experience. It's the same mechanism: stronger prior toward the distribution trained on, worse compliance with alternatives. We notice it as a regression in AI because we have benchmarks. In people we call it expertise and wonder why the migration keeps stalling.

The lever is the outer loop. The interesting question isn't which model. It's what harness, and who checks the harness.

References

Pairs with: Proven vs elegant · Self(ish) diff