AI code review is an automated check of changes by a model before a human sees them. It works only if there are few comments and almost all of them are valid. Two cases in October 2026 showed this: the Claude Code team at Anthropic reduced the number of review rounds by filtering out dubious comments and limiting repeat passes, while the release of PostgreSQL 19 was pushed to November because AI reviewers found more defects than the developers could process. You need to tune not the model’s “intelligence,” but the threshold, the number of rounds, risk levels, and metrics.
Prepared by AIrassvet October 17, 2026 based on Anthropic’s engineering blog post from October 15, 2026, about how the Claude Code team reviews code, and The Register’s October 16, 2026 article about the PostgreSQL 19 delay. It is intended for CTOs, engineering leaders, and team leads who are bringing AI into pull request review. We did not test the tools described or measure their accuracy: the rollout sequence and templates below are the editorial team’s methodology. AIrassvet is not affiliated with Anthropic or the PostgreSQL project; the section at the end of the article describes the publisher’s services.
Contents
- At a Glance
- Why AI Code Review Became a Topic in October 2026
- How Review Works on the Claude Code Team
- Noise: Why Extra Comments Are More Dangerous Than Missed Ones
- Endless Rounds and How to Stop Them
- The PostgreSQL 19 Lesson: Humans Have Limited Capacity
- Where the Decision Still Belongs to a Human
- How to Implement AI Code Review: 8 Steps
- Reviewer Rules File Template
- What Metrics to Track
- Common Mistakes
- FAQ
- Original Sources and Verification Method
At a Glance
- The bottleneck is human attention, not the model. The Claude Code team’s goal is for most changes to pass with at most one human in the loop, without increasing rollbacks or incidents.
- A comment is published only if it has been confirmed. Several agents look for bugs, a separate group votes on each finding, and purely stylistic comments are discarded.
- Rounds must be limited. After convergence measures, the share of internal pull requests that passed review in the first round rose from 65% to 78%, while the average number of passes fell from 2.08 to 1.32.
- The threshold depends on the stage. In PostgreSQL 19, AI review found a long tail of minor defects in functions that had already been written; there was not enough time to process them before release, so 8–10 major functions were moved to the next version.
- A human decides, not just rereads. Humans keep for themselves the questions of “should we do this?”, risky code, architecture, proof review, and responsibility for the outcome.
Why AI Code Review Became a Topic in October 2026
Automated code review is a change check performed by a model that reads the diff in the context of the entire repository and leaves comments before a human or instead of part of the manual review. Such checks existed before, but in 2026 the volume of code written by agents made manual line-by-line review difficult to scale.
According to Anthropic, the number of changes per day in its monorepo grew from about 2,500 in May to more than 30,000 in September 2026. The Claude Code team itself says that Claude writes about 90% of the product’s code. From June to September, the number of merged pull requests in the Claude Code repository more than doubled, while the rollback rate fell by more than 40% — company data on its own process, not an independent measurement.
The next day, The Register reported the flip side: the release of PostgreSQL 19, originally planned for late September or early October, is now expected around November 19, and 8–10 significant features were pushed to the next version. One of the project’s lead committers, Peter Eisentraut partly attributed this to code review using language models.
Taken together, these stories point to a practical conclusion: AI code review is not an on/off switch, but a configurable process with throughput limits.
How Review Works on the Claude Code Team
Anthropic describes not a single check, but layers. Each one catches part of the problems, and anything that gets through must be found and rolled back quickly.
| Layer | What happens | Who decides |
|---|---|---|
| Before pull request | An agent runs tests, builds, and linting; a second agent reviews the diff in a fresh context | Automation |
| Required CI | Linting, types, cross-platform tests, build; a red status blocks merging | Automation |
| AI review | Agents search for bugs in parallel, reviewers vote, and findings are sorted by importance | Automation |
| Security review | A separate pass; an unresolved finding blocks merging | Automation |
| Risk-based approval | A classifier determines the risk level: zero, one, or two people | Human by risk level |
| Merge queue and flags | Checks are repeated on the current branch, features are rolled out behind a flag and first internally | Automation |
| Fast rollback | On-call and the author monitor the release, and the flag is turned off without deployment | Human |
Source: Anthropic blog post from 10/15/2026, section on the change lifecycle.
Three rules from this setup are worth adopting for almost any team:
- Every comment requires a response. The author either fixes it or closes the discussion with a reason. This is also how false positives get cleared out.
- Risk level is determined by the change, not the author. Documentation fixes and auto-update changes follow different paths; flows like CI and payment logic always require a human.
- Agent approval does not replace human approval. Automation can approve a low-risk change, but it cannot take the place of a required human reviewer. Approval applies to a specific commit: after a new push, it must be granted again.
According to Anthropic, after the research version of AI review launched, the share of internal pull requests with substantive comments rose from 16% to 54%. Large changes longer than 1,000 lines most often receive findings, while small ones more often return without comments.
Noise: Why Extra Comments Are More Dangerous Than Missed Ones
AI review noise — the share of comments that do not lead to a fix: false positives, trivial issues, and style nitpicks. It’s easy to catch more bugs if you’re not afraid to flag a lot of non-bugs. That’s why the Claude Code team specifically checks the other side.
How Anthropic measures noise:
- By people’s reaction. Each comment is assigned to one of four groups: fixed, judged incorrect, intentional behavior, or deemed unimportant.
- On clearly clean changes. The reviewer is run on pull requests that are known to be correct, and the team measures how often it reports a problem that would have prompted a harmful change.
- By recorded priority. First — catch real bugs; then — reduce the number of review rounds.
Important detail: noise is more expensive when the author is an agent. A person brushes off a nitpick in seconds. An agent fixes it, which means a new commit and a new round of review. If an agent writes the code and another agent reviews it, they can end up bouncing off each other forever.
That leads to a practical rule: the trigger threshold is the key setting for AI code review. A reviewer who comments on everything quickly teaches the team to close comments without reading them. Then important issues get missed too.
| Type of comment | What to do | Why |
|---|---|---|
| A bug that will break behavior | Publish; block merge until there is a response | This is the main value of review |
| A security issue | Publish in a separate pass; block | The cost of missing it is high |
| A bug that existed before the change | Publish with the tag “already existed” | The author should not be blamed for old code |
| A minor issue, style, “could be cleaner” | Limit the number or do not publish | Trains people to ignore comments |
| A repeat of a finding that was already published | Do not publish | Provides no new information |
Anthropic uses the labels “important,” “minor,” and “already existed” in its product; the action-by-row breakdown in the table is the editors’ methodology.
Endless rounds and how to stop them
Anthropic gives an example: one pull request went through more than 50 rounds of review and fixes overnight before the author stopped the process. Analysis of the change history showed that about 59% of what the reviewer flagged again was code that had already existed on the first pass. The reviewer was barely finding new bugs — each round, they were just finding a few more old ones.
The team introduced four measures:
- round limit — after three rounds, the reviewer asks the author to stop and rethink the approach;
- review only the new diff after the first passes;
- specify the bug class, so the author fixes all instances of it, not just one line;
- ban repeats of findings that were already published.
Result on internal pull requests, according to the company:
| Metric | Before the measures | After the measures |
|---|---|---|
| Passed after one round of review and fixes | 65% | 78% |
| Required five or more rounds | 8.9% | 1.5% |
| Average number of review passes before success | 2.08 | 1.32 |
Source: Anthropic blog post dated 10/15/2026, Figure D. The measurement is internal; the sampling methodology is not disclosed in the post.
PostgreSQL Lesson 19: people have limited bandwidth
The PostgreSQL case shows the same problem from another angle. According to Peter Eisentraut, some reverted features had serious architectural flaws. But AI review also found the long tail of relatively harmless edge-case issues that would otherwise have been discovered years later and fixed after release.
He describes the situation like this: “in front of you is a list of forty defects… even if each one takes, on average, a three-line fix, there is fixed overhead for each one, and time runs out.” The key point is that the effective quality bar had already gone up after the features were written and accepted.
What this means for a company:
- It’s better to add AI review early in the cycle, not right before release. A reviewer turned on two weeks before launch will spit out a queue of findings across all accumulated code.
- Review legacy code in a separate stream. At Anthropic, review agents go through existing code overnight, and findings in old code are tagged separately from the new change.
- Sort findings by importance and limit the queue. Forty small comments on one function are not forty tasks for today; they’re a prioritized backlog.
- Plan time for people to triage them. Every finding needs a decision: fix it or justify not fixing it.
Eisentraut separately emphasized that the rollbacks were not related to vibe coding: most of the features were written before that approach became available.
Where the decision stays with a person
Anthropic identifies five areas where a human decides:
- whether it should be done at all — direction, taste, and scope of the work;
- risky code — payments, auto-updates, CI workflows;
- architecturethat sets the standard for others;
- reading evidence — test output, executed command, screenshot, reviewer findings;
- accountability for the outcome — one person is responsible for every released change, no matter who wrote the code.
A separate rule applies to agents: in a pull request written by an agent, the person who managed it is treated as one of the required reviewers — just as if they had written the code themselves.
How to Implement AI Code Review: 8 Steps
The sequence below is an editorial methodology assembled from Anthropic’s recommendations and PostgreSQL 19 lessons. It is designed for a team starting with a single repository.
- Choose one repository and establish a baseline. For the last 4–8 weeks, record: the number of pull requests, median review wait time, number of human comments, rollback rate, and incidents.
- Give the agent verifiable commands. Tests, build, and linter should run with a single command defined in a short project rules file. The agent should show output, not say “done.”
- Write a rules file for the reviewer. What counts as a failure in this part of the code, which paths to skip, how many minor issues are acceptable, and the requirement to cite the file and line. A template is below.
- Turn on AI review in “comments only” mode. For the first few weeks, the reviewer blocks nothing, and the team labels every comment: fixed / false positive / intentional / not important.
- Set the threshold based on the labels. If most comments are “not important,” reduce nitpicks and raise the bar for merging. Move repeated comments into rules, the linter, or tests.
- Add a required response and a round limit. Each finding is closed with a fix or a reason; after three rounds, reconsider the approach, then review only the new diff.
- Define risk levels. Which paths always require a human, which changes can pass with one reviewer, and where automated checks are enough. Start strict and loosen based on data.
- Build a task set from incidents. Changes that led to rollbacks and outages are the best test for the reviewer. Run it after every model, prompt, or tooling change.
Step 8 repeats Anthropic’s approach: every incident adds a case to the set, and any reviewer change is evaluated against the full set. More on building your own task set is in the article “Your Own Benchmark for Choosing an LLM”.
Reviewer rules file template
Anthropic recommends keeping such a file short. Good versions describe what failure looks like in this repository, limit nitpicks, list skipped paths, require a file and line reference instead of guessing from the function name, and ask for the summary to start with the number of findings. Example structure:
# Review Rules: Orders Service
## What counts as a failure
- Money: incorrect amount, currency, or rounding in an order.
- Data: writing to someone else’s order, losing status, double payment.
- API: changing the response format without versioning.
## What not to comment on
- Formatting (the linter checks it).
- Naming style, if it does not violate project conventions.
- The migrations/archive and generated/ directories.
## How to write a finding
- Cite the file and line, not a guess based on the function name.
- No more than 3 minor comments per pull request.
- Mark errors that existed before the change as “already existed.”
## Summary
- Start with the number of findings by severity.
- If there is nothing to fix, say so.This is a hypothetical example: build your own points from real incidents and repeated human comments.
Which metrics to track
Anthropic measures human time saved, and uses rollbacks and incidents as a guardrail: if they increase, human time has not been saved but shifted to fixing the fallout.
| Metric | What it shows | Warning sign |
|---|---|---|
| Human approvals on pull requests (median and 95th percentile) | How much attention a change requires | Increases after AI review is turned on |
| Human comments | How much manual work is left | Does not decrease for several weeks |
| Human wait time | Flow speed | Rises because of reviewing findings |
| Rollback and incident rate | Output quality — the guardrail | Rises when manual review is reduced |
| Rounds to review pass | Process convergence | The share of long cycles grows |
| Reaction to comments | Reviewer noise | Many “not important” and “false positive” labels |
| Findings on changes that are clearly clean | False alarms | Pushes toward harmful edits |
The first four rows are metrics Anthropic names; the last three also come from the post, grouped by the editors.
Common mistakes
- Turn on the reviewer across all repositories at once. Without rule tuning, noise will hit every team equally hard. Anthropic recommends starting with one repository and tuning the rules for several weeks.
- Treating the number of findings as success. More comments does not mean better: what matters is how many were fixed and how many errors reached users.
- Let the agent approve on behalf of a human. That removes accountability from the process.
- Not limiting the rounds. Agent-author and agent-reviewer without a limit spend time and budget on old findings.
- Enable deep review right before release. Lesson from PostgreSQL 19: the queue of findings can be larger than the team can handle.
- Send code to an external model without rules. If the code contains trade secrets or personal data, first decide where the model runs. Options are in the article “AI Agent for Development in an Isolated Environment”.
A separate check is needed if the application was built mostly with AI — the process is described in the article “Code Audit After Vibe Coding”. And whether to trust the “confident” label the model assigns to its answer is discussed in the article “Can You Trust AI Confidence”.
Frequently Asked Questions
What Is AI Code Review?
It is an automated review of code changes by a language model before a human review or instead of part of the manual review. The model reads the diff in the context of the repository, looks for bugs, policy violations, and vulnerabilities, and leaves comments in the pull request.
Can AI Fully Replace Human Code Review?
No, not when risky changes are involved. Even at Anthropic, agent approval does not count as human approval, and payments, auto-updates, and CI always require people. Automation can be trusted with low-risk changes when checks are green.
Why Do Developers Stop Reading AI Comments?
Because of noise: when a reviewer comments on everything, people get used to closing comments without looking. That is why you should publish only confirmed findings, limit minor issues, and avoid repeating what has already been published.
How Many Rounds of AI Review Are Needed?
Usually not many. The Claude Code team limited the number of rounds and after three asks the author to rethink the approach; after that, the average number of passes dropped from 2.08 to 1.32.
Why Was PostgreSQL 19 Delayed Because of AI Review?
Language models found many defects in already written functions, including minor edge cases. There was not enough time to sort them out before release, and 8-10 major functions were moved to PostgreSQL 20.
Where Should You Start When Implementing AI Code Review?
With one repository: establish baseline metrics, enable the reviewer in comment mode, tag the team’s response to comments, and set up a rules file. Introduce blocking and risk levels after that.
Source Materials and Review Method
Method: On October 17, 2026, we read the full Anthropic post, extracted the review layers, rules, metrics, and numbers before and after the changes, then verified the PostgreSQL 19 history through The Register and Peter Eisentraut’s blog. AIrassvet did not run its own AI review measurements; the eight-step implementation plan, the action table by comment type, and the rules file template are the editorial team’s methodology.
- Anthropic, 10/15/2026: “How the Claude Code team does code review” — review layers, noise, rounds, risk levels, metrics.
- The Register, 10/16/2026: “Postgres 19 delayed after AI reviewers find more bugs than humans can fix” — timeline, number of rolled-back functions, Eisentraut quotes.
- Peter Eisentraut’s blog — a post about functions being rolled back from the PostgreSQL 19 branch.
Limitations: Anthropic’s numbers are the company’s own data about its process, without independent verification; the company sells the review tool described. The experience of a large team with its own model cannot be directly transferred to a company where every run is paid for from the team budget.
How AIrassvet Helps Implement AI Code Review
AI code review brings value when it is built into the development process with rules, metrics, and access boundaries. On the AIrassvet website the following directions are described that relate directly to this task:
- development process audit: where reviewers’ time is currently going, which code paths are risky, and which metrics to collect before launch;
- integration of AI agents with the repository and CI: pull request checks, a reviewer rules file, mandatory responses to findings, and a limit on rounds;
- deployment of the language model on company servers if the code cannot be sent to external services;
- testing the reviewer on a set of past incidents and training the team to work with comments.
The first step is to choose one repository, describe the current review process, sources of data on pull requests and incidents, code security constraints, and the success criterion, for example fewer human comments without an increase in rollbacks.
Bottom line: Tune the humans’ attention, not just the model
The Claude Code team’s experience and the PostgreSQL 19 delay describe the same problem from two sides. An AI reviewer can find a lot of bugs, but each finding costs human time. If there are too many comments, people stop reading them or do not have time to work through them.
That is why implementing AI code review is about thresholds, rounds, risk levels, and metrics. Start with one repository, measure the response to comments, and use rollbacks and incidents as the limiting factor. Then automation will handle the routine, and people will focus on decisions that machines should not make.