Self(ish) diff
The first thing I tried, when I started reviewing pull requests with an AI in the loop, was the obvious move: ask it to review the PR. Paste the diff, ask for issues, read the list. It felt productive. Six bullet points came back. I'd send a few of them as comments. Done in fifteen minutes.
What I noticed, after a few weeks of doing this, is that I couldn't really remember the changes I'd reviewed. I had read the AI's summary of them, not the changes themselves. When I later had to debug something in code I had nominally reviewed, it was unfamiliar. The review had happened; the reading hadn't.
The other thing I noticed is that the bullets weren't always real. Sometimes the AI flagged an issue that wasn't an issue, because it didn't know the surrounding code. Sometimes it missed something obvious because the obvious thing was three files away in context the diff didn't show. I was approving comments without verifying them, then sending them as if they were mine. That felt off. I think technology makes that kind of sloppy easy: it gives you a fast way to ship the appearance of the work, and you don't notice you've crossed over.
Exploring the differences
For the last few weeks I've been doing it more creatively. Slower, but it feels more like the thing I thought I was doing the whole time.
I read the diff myself. Line by line, byte by byte if it's small enough. While I read, I narrate to the AI: what I'm seeing, what I'm uncertain about, what assumption I'd want to check. The AI's job is not to review. The AI's job is to validate or push back on the specific thing I just said, fill in context I'm missing, surface a related file I might want to look at next.
The conversation is different. Instead of "here are the issues," it's "I'm looking at this function and I'm not sure why it's catching this exception silently. Is there a caller that depends on the silent fail?" The AI either tells me ("yes, file X relies on it") or tells me it's not sure ("I don't see one in the diff, but I haven't seen the rest of the package"). Either answer keeps me in the seat. The first one helps me; the second one prompts me to go look myself.
Running it in context
The other piece is asking the AI to run small snippets against the change, inside the context the change actually lives in. The test suite runs in CI; that's not what this is for. This is about seeing the specific behavior the diff describes happen in real conditions: a handful of lines that exercise the path I'm uncertain about. If the diff says X happens when Y, I want to see X actually happen when Y. The snippet runs, the AI shows me the output, I match it against what I expected.
Sometimes the output matches and the review surfaced something worth a comment anyway. Sometimes it diverges in a way I wouldn't have predicted from reading the diff alone, and the review I'd send without running anything would have missed it entirely. Both outcomes feel like they belong in a review, and only one of them is reachable through reading.
Notes that compound
The third thing I'm testing is asking the AI to take notes of the comments I'm making as I go. At the end of the week I sit with that collection alongside whatever else I've been jotting down through the days. Most of the notes are local to the PR they came from and don't generalize. The ones that recur are the interesting ones.
The point isn't to log every observation. It's to keep the ones that compound. A review I do today might surface a pattern I only recognize three weeks later, when I see the same shape somewhere else. If the comment lives only in the PR, the recognition doesn't happen. If it lives in a notes layer I'm rereading weekly, it does. There's something I've been using to hold this kind of capture; a post on it is coming soon. Any structured notes layer would probably do the same job in the meantime.
What I'm finding
I don't know if this is the right way to review code. I'm not sure that question has an answer. What I can say is that for me, right now, this is more rewarding. I'm reading the code instead of reading a summary of it. The AI conversation goes deeper because it's responding to specific things I'm noticing, not generating a generic review I have to filter. And when I send comments, they're actually mine, with the kind of context that makes them land.
It's slower. A review that took fifteen minutes the old way takes forty-five this way. The trade-off feels worth it for the kinds of reviews I do, which are usually small to medium changes in code I half-know. I don't know how it scales to a 200-file PR or to a codebase you've never seen. Probably it doesn't, or probably it changes shape.
The thing that pushed me to try this was a small observation: I was getting through more reviews per week and remembering fewer of them. That's not a metric I want to optimize. So this is what I'm testing. If you're feeling something similar, it might be worth trying.