Agent Code Decay: I Went Looking for It in My Own Repos

Dex Horthy published 'Why Software Factories Fail' yesterday, and the part I can't stop thinking about is how honest he is: he says models degrade codebase quality over time, then admits he can't prove it, and neither can you, because nothing measures this. Fair enough. I have three repos of my own, so I went looking. This is what I've found and what I think it means.

Horthy's Argument

Dex Horthy's Why Software Factories Fail is the best version I've read of the case against agent-built software, he goes all the way down to the training loop to explain why it happens. Coding models are scored by benchmarks that ask one question, did the tests pass. Everybody is quoting this line today, and it deserves it:

If the tests pass, we win, but there is no penalty for eroding codebase maintainability.

According to him, the reason nobody fixed this yet is timing. Tests give a verdict in seconds, and that's what makes millions of training loops possible. Bad architecture sends the bill much later, the first time somebody opens a file for a one-line change and can't make it in one line.

And he's not talking from theory. HumanLayer went fully lights-off in July 2025, agents writing and shipping, nobody reading the code. It broke in production, the model couldn't find the cause no matter how he prompted it, and he ended up digging through a codebase he had stopped reading months before. After the third round of that, his cofounder rewrote the patterns by hand, two weeks in VS Code. In an aside about brownfield code he puts a number on the general case:

it feels like an agent-built codebase starts to struggle after maybe three to six months, you start to slow down, and the way you approach adding new things has to change

I'm not writing this to debunk him, I think the training argument is correct. I've watched agents write code that passes every check and is still structurally awful, and the try/catch-around-everything screenshot in his post is something I've seen in my own diffs. What caught my attention is how honest he is about the evidence. He cites an industry report showing review quality going down and incidents going up, and then he says himself that it shows correlation and nothing more. And about the thing he actually wants to know:

I can't prove this. You can't prove it either. There are no good benchmarks for a model's ability to maintain codebase quality.

He's right about the benchmarks. But for months I've been running an OpenUP-derived process for coding agents. OpenUP is a lightweight descendant of the Rational Unified Process, the heavyweight methodology from the late nineties, and my version replaces the human policing with machine-enforced boundaries. So I have repositories where agents did most of the work under a fairly heavy process. If agent-built code starts to struggle at three to six months, mine had the time to show it. So I went looking, and the rest of this post is what those three repos actually did.

What I Looked At

Three codebases, all mine. Two are private products, so let's call them App A and App B. The third is the framework itself, which is public.

App A is close to a pure agent artifact: around 97% of the application commits are agent-authored, and the few human commits are ops work, secrets, DNS, Kamal config. Nobody hand-wrote a model or a controller in that app. App B is more interesting, thirteen months old with two eras inside the same codebase, same product, same stack, same team: seven months built by humans, then six months where agents did roughly half the commits.

When does the clock start? Depends, and the test depends on it. Horthy is talking about the agent-built codebase, so the clock starts the day the agents take over, and his own timeline agrees, lights-off in July, wall around October. For App B that means six agent months, the far edge of his window, still inside it. On the other hand, App B has the harder problem: the agents didn't start from an empty directory, they inherited seven months of human code with almost no tests.

Two of these repos are private, so you can't rerun anything against them. I'm giving you the numbers that carry the argument and leaving out the decimals nobody can verify anyway, which makes this a practitioner reporting on his own work and not an independent evaluation of anything.

What I Found

The most direct way to test "entropy accumulates" is to stop inferring it from commit behavior and just rebuild the codebase at the end of every month and look at it. That's what I did with App B (reviewing thirteen monthly snapshots by eye is super boring, that's why there are scripts). Thirteen months, fourteen times growth:

month end	files	lines	median file	p90 file	over 400 lines
2025-06	69	7,926	45	382	8.7%
2025-09	143	20,073	58	393	9.1%
2025-12	200	28,069	70	392	9.5%
2026-02	753	104,512	77	333	7.6%
2026-07	813	110,464	75	315	7.4%

Big files didn't pile up. God objects are the classic symptom everybody predicts, and measured on the tree as it stood each month, they got proportionally rarer. Careful with that word, "proportionally": a lot of small new files can improve a percentile without any existing file getting shorter. The check that settles it is tracking only the files that existed at the start, I haven't done that one yet.

App A doesn't move at all. Median and p90 file length end within a line or two of where they started, and its p90 is way under half of what App B had during the human months. The most agent-built codebase I own has the smallest files. I wasn't expecting to write that sentence.

The second prediction is shotgun surgery, Fowler's term, also Horthy's pick: it gets hard to change one part of the system without breaking another. That should show up as changes touching more files in more places. I plotted files per commit for App B across the thirteen months and it's flat. It moves inside a band, never picks a direction, and the agent months sit in the same band as the human months. The framework repo shows the same, measured per task instead of per commit.

There's an objection here and it's my own tooling. My process caps commit width on purpose: a lane declares which paths a change can touch, and a fence blocks the commit if the diff escapes them. With that in place, shotgun surgery wouldn't appear as fat commits, it would appear as more tasks per feature. I mostly trust the file-size numbers, this one less, because this metric is partly measuring my own fence.

The third metric taught me something, not the thing I expected to learn. Co-change coupling was my favourite candidate to catch a problem: if the architecture is degrading, files from different modules should move together more and more. It didn't climb. Then I looked at which pairs were the most coupled, and App A's top pair, by a lot, was this:

config/locales/en.yml  ~  config/locales/es.yml

Of course they move together, it's a bilingual app, the English and Spanish translations have to change together. Super obvious in retrospect. The detector's strongest signal was the app doing its job, and the next pairs were the same story, locales with routes, model with schema, controller with routes, the normal anatomy of a Rails feature.

I was planning to add a coupling threshold gate to my process. Not anymore. The first thing that gate would have flagged is the i18n invariant that must never break. A coupling alarm in a conventional web framework mostly detects the framework.

The Process Is Doing the Work (I Think)

The finding I didn't go looking for is the one I keep thinking about. I tracked App B's test-to-source ratio month by month, and the two eras don't look like the same project: the seven human months sit basically at zero, the six agent months sit around 0.5. Seven months of human code with almost no tests, then the agents arrived and the test suite arrived with them, and App A was born with a test suite and kept it the whole way.

Nobody should read that as agents caring about coverage (they don't care about anything). The process makes them, and it makes them do a few more things:

One task per run. A run completes exactly one roadmap task and stops. "Keeping going" counts as a process violation and the run ends there.
The spec goes before the code. If the behaviour changes, the spec changes first and the code follows. Sounds bureaucratic, and it's the rule that gives me the most.
Each change declares what it can touch, and a machine checks it. That's the lane and the write-fence from before.
Nothing needed to continue the work lives in a conversation. Roadmap, status, run logs, decisions, everything is in the repo, so the context doesn't evaporate when a session ends.

Horthy rejects this whole family of answers in advance, and I don't want to route around that. His thesis is that no amount of harness engineering fixes what is a model-training problem, and he has a section aimed at the people who reply "you're holding it wrong". I'm not claiming that. Lanes and fences don't make the model write better code, they can't. He describes review agents as raising the floor without moving the ceiling, and I agree with that too. My claim is smaller: with the surface bounded and the floor raised, I never needed the ceiling to move. The model writes whatever it writes, what changes is how far that can spread before a human signs off.

He could answer that I just described a more expensive harness and that the code rots inside my lanes anyway. Maybe. Six agent months doesn't settle that. What I have is a methodology from the 90s, buried around 2005, with a write-fence bolted on, and the irony is that the paperwork that killed RUP is exactly what makes it work with agents (I know how that sounds).

Where Horthy Is Still Right About Me

A few things I can't answer.

My sample is three repositories, one owner, one stack, one agent vendor. That settles nothing in general. On top of that, I'm grading a process I designed, with a script I wrote, on code I own, nobody should confuse this with an independent evaluation.

Single-owner repos maybe can't show the failure at all. HumanLayer is a company, with a cofounder and customers. If the real mechanism is coordination, several people and several agents building incompatible mental models of the same system, then my repos are structurally unable to exhibit it. This is the strongest argument against this post and I have nothing to answer it with.

My metrics are blind to the decay mode agents get accused of most, duplication, and I'm not gonna pretend my charts answer that. File size and change spread say nothing about dead code, or about the same helper existing in three slightly different versions. A codebase can keep its file sizes stable while quietly tripling its logic, and growing fourteen times in thirteen months is the kind of number that should make anybody ask if the product needs all of it.

And one comparison turned out to be unavailable. I tried to measure how many of the lines each era wrote still survive today. Agent code looked slightly worse, until I've noticed the whole result depended on one commit with about 25,000 lines of generated tests, surviving almost untouched because nobody edits generated tests again. That commit has no Co-Authored-By trailer, so it counts as human, and it's almost for sure agent-written. There's a lot of bulk-generated work sitting in git history counted as human contribution, so I can't trust survival numbers until I find a way to clean that up.

Everything Here Has a Short Shelf Life

Whatever you take from this post, check it again in three months. That's more or less the half-life of my own practices, and I'm supposedly the guy running a stable process.

The framework started in January as a small converter for 961 Eclipse OpenUP HTML files, it's not that anymore. In April I shipped the hardest enforcement I ever built, blocking hooks with no bypass, and I softened it to warnings the same afternoon because it made real work impossible. Later I moved the mechanical writing to a cheap Haiku scribe, paying the expensive model for bookkeeping was silly. And in this same post I killed a coupling gate I had already designed.

Every quarter something lands that makes a piece of my process unnecessary, subagents, skills, longer context, better tool use. Half of what I tuned last quarter is solved by the harness now. Horthy is in the same position and to his credit he wrote it down, one of the posts he links is called "Everything We Got Wrong About RPI". The people furthest ahead on this are revising their own advice every few months.

My Bet

An opinion that can't lose isn't worth much, so here's mine, in a form a single repository can break:

An agent-built codebase with almost no tests will show its largest files getting longer and its modules getting more tangled within six active months of agent work.

All three of my repositories have a real test suite, and none of them show either symptom. If you have an agent-built repo with real tests that decayed anyway, my bet is wrong and I really want to see it. If you have one without tests that decayed, then we agree, and we found the variable that matters.

This is why I think the disagreement is narrower than it looks. Horthy's part II is called "Turning the lights back on", so my guess is we end up close to each other: the discipline has to come from outside the model. We'll differ on the how. I've bet on written specs and machine-checked lanes, and his title sounds like a bet on putting humans back in front of the code at the moments that matter.

One number is still pending, though. App B has around seventeen thousand lines of JavaScript with zero JavaScript unit tests. It's exercised end to end by system tests driving a real browser, so calling it "untested" isn't fair. But a system test tells me the page works, it doesn't fail when a controller quietly grows a second responsibility, and that's exactly the decay I was looking for. That directory is the first place I'd check, and I haven't pointed a single measurement at it yet.

Horthy's post: Why Software Factories Fail, there's also a talk version on YouTube. My process and the measurement scripts are in open-up-for-ai-agents, the framework repo is public so you can reproduce that part. The two applications are private, so on those you have my word and my scripts.

TL;DR