An AI Code Review Policy That Matches Review Depth to Risk

# An AI Code Review Policy That Matches Review Depth to Risk

AI code review can make routine pull requests easier to navigate. It can also create a false sense that every change has received the same scrutiny. A mature policy makes the difference visible: small documentation updates do not need the same review depth as authentication logic, data migrations, or cross-service changes.

The policy should not be “always run the agent” or “never trust the agent.” It should define what kind of review is appropriate for each risk level, what evidence the agent contributes, and which decisions remain human responsibilities.

GitHub made this idea explicit in its August 7, 2026 release of selectable code-review effort levels, distinguishing lighter feedback for straightforward changes from deeper analysis for more complex or sensitive work. [The release note](https://github.blog/changelog/2026-08-07-copilot-code-review-effort-levels-are-generally-available/) is a product feature, not a complete policy. Its durable value is the risk-proportional model: review depth should be a deliberate engineering choice.

Define what the assistant is for

Before configuring a tool, write down its role. Good initial roles include summarizing a diff, identifying likely missing tests, spotting inconsistent error handling, locating affected documentation, and suggesting questions a reviewer should ask.

Avoid assigning authority by implication. An agent’s comment does not approve a pull request, waive a required owner review, establish compliance, or prove that a change is safe. The accountable reviewer should be named in the repository rules, not inferred from an automated comment.

Use a simple risk model

Risk models should be easy enough to use during normal work. Start with three tiers.

| Tier | Typical change | AI review role | Human requirement |
| --- | --- | --- | --- |
| Routine | Documentation, copy, isolated tests | Summarize and flag obvious omissions | One informed reviewer |
| Standard | Feature logic, API behavior, common dependencies | Analyze diff and test gaps | Code owner or peer review |
| High | Auth, permissions, payments, migrations, security, production config | Assist investigation, never substitute assurance | Domain owner, tests, and explicit approval |

The risk comes from impact and uncertainty, not from line count. A five-line permission change can be high risk; a large formatting migration may be routine.

Make the tier visible in the pull request

The author should declare a provisional tier using a short checklist: data affected, external interfaces, authentication or authorization changes, destructive behavior, migrations, and rollback plan. The reviewer can raise the tier if the declaration misses something.

When the tier is visible, the AI reviewer can be configured intentionally. Routine changes may receive a quick pass that focuses on consistency. Standard changes receive a broader request for test and edge-case analysis. High-risk changes receive targeted prompts tied to a threat model, but still need domain-specific human review.

Keep prompts out of the approval record when they contain sensitive code or data. The durable record should be the risk classification, review results, decisions, and links to tests.

Ask the agent for evidence, not a verdict

“Looks good” is not useful review output. Better questions force traceable reasoning:

• Which behavior changes for an existing caller?
• Which branches have no test coverage in this diff?
• What assumptions does this change make about input, state, or ordering?
• Which files indicate a contract or schema that may also need updating?
• What could make this rollback difficult?

An answer that points to concrete lines and test cases gives a human reviewer something to evaluate. A generic assurance adds noise and may conceal uncertainty.

Protect the review channel from automation noise

AI review becomes counterproductive when it posts a long list of low-confidence comments. Set a quality bar. Ask the tool to prioritize findings that are specific, explainable, and linked to the changed code. Group minor style observations into a summary or leave them to a formatter.

Measure signal quality through sampled review outcomes: how often did a comment lead to a corrected defect, a useful test, a rejected false positive, or no action? Do not reduce the measurement to comment count. More comments can mean a worse review experience.

Keep sensitive changes on a separate path

For high-risk changes, require controls the agent cannot satisfy alone:

• A domain owner reviews the change.
• Tests cover the relevant failure paths.
• The pull request identifies the rollout and rollback plan.
• Credentials or permissions affected by the change are reviewed by their owner.
• The change is deployed through the normal controlled pipeline.

The agent can accelerate preparation by tracing call sites or generating a test matrix. It should not be treated as a security sign-off or a substitute for production validation.

Establish a feedback loop

Every month, sample a small set of pull requests from each tier. Compare the declared risk with the eventual impact, inspect ignored comments, and look for categories where the assistant helps or distracts.

Use the findings to tune routing. Perhaps database changes need a schema-focused prompt; perhaps generated files should be excluded; perhaps documentation changes need no automated pass at all. Policies improve when they respond to the team’s actual failure modes.

A lightweight policy template

1. Authors declare a routine, standard, or high-risk tier.
2. The agent receives a tier-appropriate review request.
3. Automated comments must cite changed code, a test gap, or a concrete assumption.
4. Human approval requirements come from the tier and code-owner rules.
5. High-risk changes require a rollout and rollback note.
6. Teams sample outcomes monthly and adjust the policy.

The result is not a faster version of the old review process. It is a clearer process: automation surfaces evidence, while people retain responsibility for the decision.

Handle generated code and generated reviews separately

The code author may have used an AI assistant, and the reviewer may use another one. Those are separate sources of uncertainty. A policy should still require the author to understand the submitted change, explain its behavior, and identify what was tested. Review automation can then look for gaps, but it cannot repair the absence of author ownership.

For generated changes, raise the review tier when the author cannot readily explain a dependency, security-sensitive branch, data transformation, or failure mode. This is not a penalty for using a tool. It is the same standard applied to code copied from a library, adapted from a prior project, or written under time pressure: the team must understand what it will operate.

Review the policy after an incident

When a defect escapes, avoid asking only whether the agent noticed it. Trace the whole decision path: was the risk tier correct, was the relevant context available, did a human dismiss a useful finding, and would a different test or owner review have caught the issue? Record the policy change that follows. Over time, these small corrections turn automated review from a novelty into a dependable part of engineering practice.

Make exceptions explicit

Urgent fixes sometimes need an accelerated review path. Define it ahead of time: who can declare an exception, which minimum tests still run, who reviews after deployment, and when the decision record is completed. An emergency path that is explicit and auditable is safer than an informal shortcut created during an incident.

The same discipline helps when a repository is temporarily outside normal automation coverage. State the reason, the compensating review, and the deadline for returning to the standard policy. Exceptions should create a visible follow-up, not a permanent undocumented mode of working.

Sources

• [GitHub: Copilot code review effort levels are generally available](https://github.blog/changelog/2026-08-07-copilot-code-review-effort-levels-are-generally-available/)