Writing code is cheap now. Verifying it is the bottleneck.

In July, Jarred Sumner announced he had ported Bun from Zig to Rust in about eleven days, using a fleet of Claude agents running in parallel. The port added over a million lines of Rust and burned roughly $165,000 in tokens at API list prices: 5.9 billion uncached input tokens against a TypeScript test suite acting as the conformance oracle. Whatever you think of the outcome, the economics are the story. A rewrite that would have taken a team years of effort now costs about one senior engineer-year of salary and less than two weeks of wall clock.

Zig’s creator had a different read. He called the result “unreviewed slop”, and whether or not you agree, the adjective doing the work in that phrase is not “slop.” It is “unreviewed.” A million lines appeared in eleven days, and there is no version of human code review that absorbs a million lines in eleven days. The tests pass. The benchmarks look fine. And the honest answer to “has anyone verified this?” is: the same class of model that wrote it.

That gap is the trend I keep running into everywhere, not just in headline rewrites. AI is not reducing the need for engineers. It is relocating the engineering effort. Writing software is becoming dramatically cheaper. Verifying software is becoming dramatically more expensive, because the volume of plausible, test-passing, unreviewed code is exploding.

Every PR looks fine. The system gets worse.

For contrast, the entire 1993 Doom engine that id Software open-sourced is about 57,000 lines of C. One coherent architecture, small enough that a single person can hold the whole design in their head. That was not just discipline; it was a constraint imposed by the cost of writing code. When every line is expensive, you are forced into elegance.

When generation is nearly free, you get what I have started calling architectural diffusion. Instead of one coherent design you get thousands of individually reasonable changes. Every function looks fine in isolation. Every PR passes CI. But nobody is holding the whole design, so the system quietly accumulates duplicated logic, three different retry implementations, inconsistent auth patterns, and the occasional endpoint that checks the session but not the tenant. No single diff is wrong. The sum is.

The failure mode is invisible at review time by construction. A reviewer approves the diff in front of them, and the diff in front of them is locally correct. The defects live in the relationships between changes: the helper that duplicates a policy check slightly differently, the new route that skips the middleware everyone assumed was global. Diffusion is a property of the system, and nothing in the PR workflow looks at the system.

Model collapse comes for your codebase

There is a second-order effect stacked on top, and it has a name in the research literature. When models train on the output of previous models, the resulting distribution degrades in a specific, measurable way.

The Nature paper describes two phases. Early collapse: statistical errors compound and the model drifts from the true distribution. Late collapse: low-frequency events vanish entirely. Now map that onto code. AI-generated repositories are already flowing back into training corpora. The unusual-but-brilliant designs, the rare idioms, the carefully considered security patterns: those are the tails. What gets statistically reinforced, generation after generation, is the middle of the distribution. Play with the figure below and watch where the mass goes.

The output of a collapsed distribution is not bad. That would almost be easier to deal with. It is average. Average architecture, average abstractions, average security practices, the same mainstream stack reproduced with the same mistakes propagated through every step of the training loop. And small mistakes do not disappear when they are averaged; they get normalized. A questionable pattern that appears in enough training data stops looking questionable to the next generation of tools.

Average code at scale is a verification problem

This is why I think the biggest opportunity of the next few years is not another coding assistant. Generation is served. The unserved questions are the ones that come after the code exists, and they are all verification questions:

Notice these are questions about the running system, not about the diff. Static review already failed us in the diffusion section, and it fails here for the same reason: the authorization bug is not in any single function. We have written before about what verification has to look like in this world - asserting on the delta, not the state, and probing the live deploy instead of trusting fixtures. The June field report covered the industry version of the same realization: the demo stopped counting. What counts is whether the thing survives contact with an attacker who has a browser console.

Generation keeps getting faster. Verification is becoming the scarce resource. Every week of AI-assisted shipping widens that gap, and the tooling on the checking side is years behind the tooling on the writing side.

Smoke test the output before it ships

This idea shaped what we built. VibeEval is our answer to the narrow, urgent slice of the problem: a security smoke test for AI-generated apps. You point it at your running application, and an autonomous agent maps every route, endpoint, parameter, and API it can reach, then fires over 310 real probes at it: auth bypass, IDOR, Supabase row-level security, exposed keys, SSRF, CORS misconfiguration, role escalation. First triage lands in under a minute; a full scan takes three to eight. It tests the live app the way an attacker would, which is the only vantage point where diffusion-style bugs are visible at all.

Two design choices matter here. Every finding ships with a captured exploit: the request, the response, a reproducible proof of concept, verified by a security engineer before it reaches you, so you are never triaging a linter’s guess. And it is built for the workflow that created the problem: apps generated with Lovable, Bolt, Cursor, Replit, v0, and Claude Code, with paste-ready fix prompts that go straight back into the tool that wrote the bug. The pattern in the scan data matches the collapse thesis, by the way: missing row-level security is the most common critical finding, and exposed secrets in client code turn up in roughly one in four apps scanned. Average security practices, at scale, exactly as predicted.

There is a 14-day free trial, no card required, and plans start at $49 a month, which is cheap insurance against the class of bug that turns into a disclosure email. Whether you use VibeEval or something else, the direction seems clear to me: the interesting tools of the next few years will not generate more software. They will tell you whether the software you already generated is safe to ship.

I am curious whether people working with AI coding tools daily are seeing the same pattern. Has verification become a bigger share of your workflow than it was a year ago? Because from where I sit, writing the code was never the hard part. Proving it does what you think it does - that was always the job, and now it is nearly the whole job.