All work
AI / LLM Engineering

AI-Powered Pull Request Review Pipeline

An AI-powered code review pipeline that automatically reviews every pull request across an entire DevOps organization — current and future repositories alike — from a single shared pipeline definition.

Org-wide — all repos, incl. future ones

Repo coverage

Zero

Per-repo setup required

1 shared definition

Pipeline definitions maintained

The Problem

Manual code review is valuable but inconsistent, and rolling out an automated reviewer repo-by-repo doesn't scale as new repositories get created over time. The goal was "every PR, every repo, forever" with zero ongoing per-repo maintenance.

My Approach

  • Registered a single shared pipeline once, then attached it as an organization-wide, "all repositories" build-validation branch policy — so any new repository created in the future is automatically covered without anyone remembering to configure it.
  • Diagnosed and worked around a real platform gotcha: the CI platform's YAML pull-request triggers are silently ignored when builds are driven by branch policy instead — a subtlety that would otherwise look like "the pipeline just doesn't run."
  • Designed the pipeline so its own checked-out repository is only the review tooling, not the code under review: at runtime it looks up the pull request by ID via the platform's REST API, clones the actual target repository using a narrowly scoped access token, runs the AI review against that clone, and posts threaded comments back to the original PR.
  • Scoped the automation's permissions to exactly what it needs on the repositories it reviews (read/clone + comment on pull requests) — least privilege by design, not an afterthought.
  • Used a generative AI model as the actual review engine, triggered only on pull-request builds (not every commit), keeping cost and noise proportional to actual review events.

Stack

CI/CD

Azure DevOps PipelinesOrganization-wide branch policies

Automation

Node.jsAzure DevOps REST API

AI

Gemini API for automated code review generation

Security

Scoped access tokensLeast-privilege permission design

Practices

Platform-wide automation designSecure token-scoped cross-repo automation

Skills Demonstrated

  • ▸CI/CD platform engineering at an organizational scale, not just a single repo
  • ▸Deep, accurate understanding of a CI platform's actual trigger semantics (not just its documented behavior)
  • ▸Secure automation design: scoped tokens, least privilege, tooling repo separated from target repos
  • ▸Practical, production application of generative AI inside a developer workflow