Evaluate Against What?

If AI writes the code and the work becomes reviewing it, half the question is missing. How I audit generated code without reading it line by line, and why the criteria have to exist beforehand.

A while back I wrote that "it runs on a miracle" is an insult: if a system works, it isn't a miracle — it's that someone, at some point, thought through the design behind it.

Recently I came across an argument that arrives somewhere similar from a different direction: AI isn't degrading engineering, it's exposing the engineering we already had. You used to be able to be bad but slow; now you can be bad but fast.

The argument is Fco Javier Vives Berenguer's, on LinkedIn. I agree with it, and this post is what stuck with me afterward.

I agree with the diagnosis. But it leaves a question open, and that's the one I care about: if the work shifted from writing to evaluating, evaluating against what?

Because "review what it generates" isn't an instruction. It's a responsibility with no criteria attached.

#Audit, don't read

The first practical change is that I don't read generated code line by line.

It isn't a stance, it's economics: if the AI writes in two minutes what used to take me two hours, and then I read it with the same depth I'd have written it with, I gained nothing. I just moved the time around.

So I treat it the way I treat a colleague's PR. I audit decisions, not characters.

With one difference from reviewing a person: I intensify the use-case testing. That's where I make up for what I didn't read closely. A dev with context on the system fails differently than a model without it — a person omits what they take as given, while a model produces something plausible that may not correspond to this particular system. Against plausible-but-wrong, running real cases pays off far more than reading.

#What I look at, in order

The intent. What is this code trying to do, and how many different things is it trying to do at once. It's the same question I use to decide whether a DTO is being reused too broadly: how many intentions are actually in play here? A model tends to produce something that serves everything, because it has no reason to separate what you would separate.

The coupling. What ended up tied to what. Business logic that leaked into the controller, infrastructure that slipped into the domain, a dependency crossing a boundary the project had been respecting.

The failure points. What happens when something is missing, doesn't respond, or arrives in a different shape. Generated code tends to be excellent on the happy path — that's what it has the most examples to learn from.

The tests. Not whether they exist: whether they prove anything. A test that always passes isn't coverage, it's decoration. And watch for the ones that actually verify the mock instead of the behavior: those produce the most dangerous feeling of all, which is the feeling of being covered.

The modularization. Whether what was generated respects the shape the system already has, or invents a new way to do something the project already solved differently. Two ways of doing the same thing coexisting is newborn technical debt.

#The result is still mine

This is the part that keeps my head straight: the tool is new, so I take precautions — but the result doesn't stop being my responsibility.

"The AI generated it" won't work as an explanation. It doesn't work today and it won't work in the review six months from now, for the same reason "they told me verbally" doesn't work: what remains on record is the code, and you signed it.

If something goes wrong, the model didn't go wrong. The decision to accept it did.

#The calculator didn't end mathematicians

The idea of an AI doing the heavy lifting genuinely appeals to me. And I don't think it kills the developer — I think it amplifies them, and it amplifies the ones who work well even more.

The calculator didn't end mathematics: it moved the work up a level. Nobody misses computing roots by hand, and nobody suggests that made mathematicians dumber.

The typewriter is a less comfortable example, which is exactly why it's more useful: there the role really did change shape. Typing pools disappeared. What didn't disappear was the need for someone to decide what got written and whether it was written well. The tool absorbed the execution; the judgment moved, it didn't evaporate.

This is the same. The part being automated is the writing. What's left is deciding whether what was written is any good — which was always the hard part, just disguised inside those two hours of typing.

#The criteria have to exist beforehand

And here's the part I think the original diagnosis doesn't quite close.

If AI exposes how much engineering there really was, what it exposes concretely is how much of your criteria was written down and how much lived in someone's head. A team with explicit conventions, defined boundaries, and tests that prove behavior reviews generated code without breaking a sweat: it already knows what to compare it against. A team where "we know how things are done here" was never written down has nothing to evaluate against — and at the speed code now arrives, implicit criteria don't keep up.

That's the real audit. It isn't about AI. It's about whether you ever put in writing how things are done.

(A good part of this site is, at bottom, that exercise: writing the criteria down before needing them.)

#ai #code-review #quality #team-practices