Code Reviews in the Age of AI

· danielfm.me ·

5 min read Original article ↗

It’s sad to see how code reviews, yesterday’s best knowledge and context sharing tool, is becoming just another source of bureaucracy and friction.

Chances are you already witnessed the following scene more often than you’d like:

  1. Bob submits a PR – most likely AI-generated
  2. GitHub sends a notification to one or more CODEOWNERS for review/approval
  3. Alice (or one of her agents) receives the notification and decides to act on it
  4. Alice’s agents review the PR and submit code change requests as comments
  5. Bob (or one of his… well, you guessed it) sees the comments, and his agents respond to the comments by submitting further changes, breaking other stuff in the process, which will trigger further comments from Alice
  6. Repeat ad infinitum

It’s bad enough when this flow is orchestrated by humans – poking their agents at every new comment or request; but things get really ugly when agents battle each other autonomously. These things don’t know when to stop!

When I see one of these exchanges, my instinct is to just turn off the computer and do some gardening, wash my car, anything but watch language models beat themselves to death.

Does it mean that we can’t learn anything useful from AI-written comments? Not necessarily; but sometimes these little knowledge gems are buried under such a pile of horse shit that basically makes these almost impossible for humans to spot them, let alone learn from them.

This is my modest attempt at establishing some code review “etiquette” for today’s reality and, hopefully, bringing back the joy to what I think can be one of the most valuable learning tools a software engineer can have.

Note: I’m no AI hater by any means, and use it extensively to deliver code both as a professional and hobbyist. However, I do think we are using it wrong.

Let’s dive into it.

There are many tools we can use for sharing knowledge: technical deep dives, maintain documentation, 1:1s, etc. And there’s code reviews.

Code reviews was a great learning tool for me as an engineer. Documentation is nice and all, but there’s nothing better than to change a system – and observe how the system itself and its upstream/downstream dependencies react – to truly understand it.

Code Review Etiquette – Revisited

The idea is anchored by the notion of effort.

When responding to a request, you shall not invest more effort than the requester. Similarly, you shall not expect someone to respond to your request with more effort than you invested yourself.

As a staff engineer, I receive dozens of code review requests every day. GitHub email notifications in the hundreds. If I treat every request as a high-effort one, I would not have time to do anything else besides reviewing other people’s changes.

This prevents me from being bottlenecked by low-effort requests – thus enabling me to spend more energy on answering higher effort requests more carefully – as well as calibrate my expectations when asking (and how I ask) other people.

Everybody wins.

Common scenarios and how to handle those according to this guideline:

AI-Generated PRs Get AI-Generated Reviews

If I receive an automated request/notification for reviewing AI-generated code, with an AI-generated pull request description, without any personal interaction made by the requester, you will get a similarly 100% AI-generated review.

Similarly, if I submit a 100% AI-generated pull request, I expect your review to be 100% AI-generated as well. For most changes, this should be enough anyway.

If I wasted time investigating and then asked you something with my own words, please do not forward me untouched AI-generated text. Don’t be a meat proxy.

If I spent grey matter writing to you, feel free to use AI to assist you if you think would help you provide a better answer, but at least validate the response yourself and forward the relevant bits only. I don’t want to read a slop essay every time I ask something. Preferably rewrite the AI output in your own words.

Now, if you just screenshot your Claude terminal and paste the image back, know that hell has a special place for you. At least I hope so.

When you take some giant IA output, digest it, and explain it in your own words, you may actually learn something. Remember your school time, when you took notes and copied with pen and paper what your teacher was writing at the black board, like a caveman? That’s how we used to learn.

Say What You Want From Me

Big PRs are the new norm, so don’t submit a 5 KLOC change and expect me to look at everything. I won’t. Not anymore.

If you think only a subset of these changes need a more careful look from me, why not break that beast of a PR into smaller pieces, and then ask me to just look at the relevant one?

Let Copilot or whatever handle the rest.

Ask Me Directly In Your Own Words

Want me to take a look at something? Ping me directly, in your own words, linking the PR, and why do you need it, not what it does – Claude can help me with the what, not the why.

Feel free to use whatever AI tools you need to help you understand the problem, navigate the search space and help you identify the probable problems/solutions, but please: if you want to know what I think about something, let me hear it in your own words.

Closing Thoughts

Let’s be mindful of the workload our questions and requests may inflict on others, especially in times like this where generating stuff is easy. Let’s put our agents to work on the boring tasks and keep in touch with the more complex and interesting ones.

The idea is not to stop using AI for generating and reviewing code, but to use it in a way that results in a fair use of everybody’s time.