Dependabot PR Review Runbook

This guide covers the process for reviewing and testing grouped Dependabot PRs before merging them into the master branch.

Overview
Prerequisites
Weekly Review Schedule
Review Process
Handling Multiple Dependabot PRs
Rollback Procedure
Common Issues and Solutions
Best Practices
Useful Commands Cheat Sheet

Overview

Dependabot is configured to create grouped dependency update PRs weekly (Mondays at 09:00). You should expect to see up to 3 PRs:

Python Dependencies (python-dependencies group) - Major version updates for pip packages
NPM Dependencies (npm-dependencies group) - Major version updates for npm packages
GitHub Actions (github-dependencies group) - All updates to GitHub Actions

For Python (pip) and NPM dependencies, minor and patch updates are ignored per the configuration in .github/dependabot.yml. GitHub Actions dependencies receive all update types, including minor and patch.

Prerequisites

Ensure you have the following installed and configured:

Git
Access to the repository with write permissions
All development dependencies installed via make install

See the Development Guide for specific tool version requirements.

Quick Setup:

# Clone the repository (if not already done)
git clone <repository-url>
cd validated-relationships-service-api

# Install all dependencies
make install

Weekly Review Schedule

Recommended Review Time: Monday afternoons or Tuesday mornings (after Dependabot creates PRs at 09:00 Monday)

Estimated Time Required:

Simple review (no breaking changes): 15-30 minutes
Complex review (with breaking changes): 1-2 hours

Review Process

1. Identify Open Dependabot PRs

First, check what Dependabot PRs are currently open:

Using GitHub Web UI:

Navigate to the repository on GitHub
Click on "Pull requests" tab
Filter by author: author:app/dependabot[bot]
Or use the direct URL: https://github.com/OWNER/REPO/pulls?q=is:pr+is:open+author:app/dependabot

Using Git Commands:

# List remote branches containing "dependabot"
git fetch origin
git branch -r | grep dependabot

You should see PRs with titles like:

"Bump the python-dependencies group with X updates" (for grouped Python updates)
"Bump aiohttp from X.X.X to Y.Y.Y" (for individual Python package updates)
"Bump cryptography from X.X.X to Y.Y.Y" (for individual Python package updates)
"Bump the npm-dependencies group with X updates" (for grouped npm updates)
"Bump basic-ftp from X.X.X to Y.Y.Y" (for individual npm package updates)
"Bump the github-dependencies group with X updates" (for grouped GitHub Actions updates)

Note: You may see both grouped PRs (multiple packages) and individual PRs (single packages) depending on:

Whether updates are available for multiple packages simultaneously
Security updates (which may bypass grouping)
Whether grouped updates failed and individual PRs were created instead

2. Review PR Changes

For each Dependabot PR, review the changes:

Using GitHub Web UI:

Click on the PR title to view details
Click "Files changed" tab to see the diff
Click "Checks" tab to see CI/CD status

Using Git Commands:

# Fetch the PR branch
git fetch origin pull/<PR-NUMBER>/head:pr-<PR-NUMBER>

# View the diff
git diff main...pr-<PR-NUMBER>

# View changed files
git diff --name-only main...pr-<PR-NUMBER>

What to look for:

✅ Check if CI/CD checks are passing
✅ Review the changelog/release notes for each updated dependency
✅ Look for breaking changes in major version updates
⚠️ Pay special attention to dependencies that affect:
- API schema validation (openapi-schema-validator, etc.)
- Testing frameworks (pytest, newman, etc.)
- Build tools (webpack, poetry, etc.)

Check Release Notes:

# For Python packages, check PyPI
# Example: https://pypi.org/project/package-name/#history

# For npm packages, check npm or GitHub
# Example: https://www.npmjs.com/package/package-name?activeTab=versions

3. Create Local Test Branch

Create a test branch to verify the changes locally:

Using Git Commands:

# Ensure you're on the latest master
git checkout master
git pull origin master

# Fetch all branches
git fetch origin

# Find the dependabot branch name
git branch -r | grep dependabot

# Checkout the dependabot branch directly
git checkout dependabot/npm_and_yarn/npm-dependencies-abc123

# Or fetch PR by number and checkout
git fetch origin pull/<PR-NUMBER>/head:dependabot-pr-<PR-NUMBER>
git checkout dependabot-pr-<PR-NUMBER>

Note: Dependabot branches may be behind master if other PRs have been merged since the Dependabot PR was created. If you encounter issues or want to test against the latest master, you can ask Dependabot to rebase:

On GitHub Web UI:

Navigate to the Dependabot PR on GitHub

Scroll to the bottom (comment section)

Add a comment: @dependabot rebase

Click "Comment"

Dependabot will automatically rebase the PR branch against the current master branch.

Other useful Dependabot commands:

@dependabot recreate - Recreate the PR from scratch

@dependabot merge - Merge the PR (if checks pass)

@dependabot close - Close the PR

Alternative: Create a named test branch

# Create a test branch from the dependabot branch
git checkout master
git pull origin master
git checkout -b test/dependabot-<ecosystem>-$(date +%Y%m%d)
git merge origin/dependabot/<ecosystem>/<group-name>

4. Run Automated Tests

Run the full test suite to verify nothing is broken:

Install Dependencies

# Install Node dependencies
make install-node

# Install Python dependencies (if updated)
make install-python

# Or install everything
make install

Run Linting

# Run Python linting
make lint

# Run Python formatting check
make format

# If formatting fails, you may need to apply formatting
make format-apply

Run Schema Validation

# Validate all schema examples
make schema-all

# Or validate specific schemas:
make schema-get-consent
make schema-post-consent
make schema-patch-consent
make schema-related-person
make schema-questionnaire
make schema-errors

Run Unit/Integration Tests

# Run Python tests (requires APIGEE_ACCESS_TOKEN)
make test

# Run smoke tests
make smoketest

Test OpenAPI Specification

# Publish the spec to check for errors
make publish

# Check the build output
ls -la build/

# Import the spec into Postman to review
# 1. Open Postman
# 2. Click "Import" button (top left)
# 3. Select "Files" tab
# 4. Browse to and select: build/validated-relationships-service-api.json
# 5. Click "Import" to create a collection from the OpenAPI spec
# 6. Review the imported endpoints and schemas in Postman

Test Postman Collection (if npm updated)

# Generate Postman collection
make generate-postman-collection

# Test against sandbox
make test-postman-collection SANDBOX_BASE_URL=https://sandbox.api.service.nhs.uk/validated-relationships/FHIR/R4

5. Manual Testing (if needed)

For major version updates or if automated tests reveal issues:

Test Sandbox

# Start the sandbox locally (if sandbox dependencies were updated)
cd sandbox
poetry run python -m flask run --port 5000

# Test endpoints manually
curl http://localhost:5000/_ping
curl http://localhost:5000/_status

Review Breaking Changes

If tests fail or you notice breaking changes:

Document the issue:

Using GitHub Web UI:
- Navigate to the PR on GitHub
- Scroll to the bottom and add a comment in the comment box
- Include test output and description of the issue
- Include relevant test output
Check for migration guides:
- Look for UPGRADE.md, MIGRATION.md, or CHANGELOG.md in the dependency's repository
- Check if there are code changes required
Fix issues if possible:
```
# Make necessary code changes
git add .
git commit -m "fix: update code for dependency upgrade"

# Push to a new branch (don't push to dependabot's branch)
git push origin HEAD:fix/dependabot-<PR-NUMBER>
```
Create a new PR with GitHub Web UI:
- Navigate to the repository on GitHub
- Click "Pull requests" → "New pull request"
- Select your branch fix/dependabot-<PR-NUMBER>
- Add title and description referencing the original Dependabot PR
- Click "Create pull request"

6. Approve and Merge

Once all tests pass:

Using GitHub Web UI:

Navigate to the PR on GitHub
Click "Review changes" → "Approve" → "Submit review"
Scroll down and click "Squash and merge" (or "Rebase and merge")
Confirm the merge
Check "Delete branch" option

Clean up local branches:

git checkout master
git pull origin master
git branch -d <dependabot-branch-name>

Handling Multiple Dependabot PRs

If you have multiple ecosystem PRs open simultaneously, you can:

Option A: Test Each Separately (Recommended)

Test and merge each PR individually following the process above. This makes it easier to identify which dependency causes issues if tests fail.

Using Git Commands:

# Test Python dependencies first
git fetch origin
git checkout dependabot/pip/python-dependencies-...
make install-python
make lint format
make schema-all
make test

# If tests pass, merge it via GitHub Web UI

# Then test npm dependencies
git checkout master
git pull origin master
git checkout dependabot/npm_and_yarn/npm-dependencies-...
make install-node
make publish
make generate-postman-collection

# If tests pass, merge it via GitHub Web UI

Option B: Combine and Test Together

If you want to test all updates together before merging:

⚠️ Warning: Merging multiple Dependabot branches together may result in merge conflicts, especially if:

Multiple PRs update the same dependency to different versions

Both poetry.lock and package-lock.json are modified by different PRs

PRs modify overlapping sections of configuration files

If you encounter conflicts and want to undo the merge:
# Abort the merge and return to pre-merge state
git merge --abort

# Your branch will be back to the state before the merge command
# You can then try a different approach or merge PRs individually
If you want to resolve conflicts:

Resolve them manually by choosing the appropriate version

Regenerate lock files (poetry lock, npm install)

Test thoroughly after resolving conflicts

Complete the merge with git commit

Using Git Commands:

# Create a combined test branch
git checkout master
git pull origin master
git checkout -b test/all-dependabot-$(date +%Y%m%d)

# List and merge all dependabot branches
git fetch origin
git branch -r | grep dependabot | while read branch; do
  echo "Merging $branch"
  git merge --no-ff "$branch" -m "Merge $branch for testing"
done

# Run full test suite
make install
make lint format
make schema-all
make test
make publish

# If all tests pass, merge the original PRs individually via GitHub Web UI

# Clean up test branch
git checkout master
git branch -D test/all-dependabot-$(date +%Y%m%d)

Rollback Procedure

If a merged Dependabot PR causes issues in production:

Immediate Rollback

# Find the merge commit
git log --oneline --grep="dependabot"

# Revert the merge commit
git revert -m 1 <merge-commit-sha>

# Push the revert
git push origin master

# Create a PR for the revert
gh pr create --title "Revert: Dependabot updates" \
  --body "Reverting #<PR-NUMBER> due to [issue description]"

Fix Forward (Preferred)

# Create a fix branch
git checkout -b fix/dependency-issue

# Make necessary fixes
# ... edit files ...

git add .
git commit -m "fix: resolve issue from dependency update"

git push origin fix/dependency-issue

Create PR with GitHub Web UI:

Navigate to repository on GitHub
Click "Pull requests" → "New pull request"
Select your branch fix/dependency-issue
Add title and description
Click "Create pull request"

Common Issues and Solutions

Issue: Poetry lock file out of sync

Symptom: poetry install fails with lock file errors

Solution:

# Update the lock file
poetry lock --no-update

# Or regenerate it
poetry lock

# Commit the updated lock file
git add poetry.lock
git commit -m "chore: update poetry lock file"

Issue: npm peer dependency conflicts

Symptom: npm install shows peer dependency warnings

Solution:

# Install with legacy peer deps (already in Makefile)
npm install --legacy-peer-deps

# If issues persist, check package.json for conflicting versions

Issue: Schema validation fails after OpenAPI validator update

Symptom: make schema-all fails after updating openapi-schema-validator

Solution:

# Check if the schema needs updates
poetry run python scripts/validate_schema.py operationoutcome <failing-file>

# Common fix: Update schema to match new validator requirements
# Check the validator's changelog for breaking changes

Issue: Flake8 or Black failures after dependency update

Symptom: make lint or make format fails

Solution:

# Apply black formatting
make format-apply

# Check flake8 config if needed
cat .flake8

# May need to update .flake8 config for new rules

Issue: GitHub Actions fail after actions update

Symptom: CI/CD pipeline fails after updating github-dependencies

Solution:

# Check the action's changelog for breaking changes
# Common issues:
# - Changed input/output names
# - Deprecated features removed
# - Node version requirements changed

# Update workflow files in .github/workflows/
vim .github/workflows/<failing-workflow>.yml

Issue: Tests pass locally but fail in CI

Symptom: Local tests pass but GitHub Actions or Azure DevOps fails

Solution:

# Check environment differences
# - Python version
# - Node version
# - Environment variables

# Try running tests in the same way CI does
poetry run pytest -v --color=yes

# Check CI logs for specific error messages
gh run list --workflow=<workflow-name>
gh run view <run-id>

Best Practices

Review Timing: Review Dependabot PRs within 1-2 days of creation
Batch Review: Try to review all ecosystem PRs together for consistency
Document Issues: Always comment on PRs when you find issues
Test Thoroughly: Don't skip tests even for "simple" updates
Monitor After Merge: Check CI/CD pipelines after merging
Keep Notes: Document any issues encountered for future reference
Stay Informed: Subscribe to notifications for critical dependencies
Security First: Prioritize security updates even if they require more work
Communicate: Let the team know if you're blocking or delaying a merge
Learn from Failures: Document breaking changes for future reference

Useful Commands Cheat Sheet

Git Commands

# List remote dependabot branches
git fetch origin
git branch -r | grep dependabot

# Checkout a PR branch (find branch name first)
git checkout dependabot/npm_and_yarn/npm-dependencies-abc123

# Or fetch PR by number
git fetch origin pull/<PR-NUMBER>/head:pr-<PR-NUMBER>
git checkout pr-<PR-NUMBER>

# View diff from master
git diff master...HEAD

# List changed files
git diff --name-only master...HEAD

# View commit messages
git log master..HEAD

# Merge PR locally (then push to master)
git checkout master
git merge --squash pr-<PR-NUMBER>
git commit -m "Merge dependabot PR #<PR-NUMBER>"
git push origin master

Testing Commands

# Run full test suite
make install && make lint && make format && make schema-all && make test

# Install dependencies
make install-node      # Install npm packages
make install-python    # Install Python packages
make install           # Install everything

# Run linting and formatting
make lint              # Check Python code style
make format            # Check Python formatting
make format-apply      # Apply Python formatting

# Run schema validation
make schema-all        # Validate all schemas
make schema-get-consent
make schema-post-consent
make schema-patch-consent

# Run tests
make test              # Run all tests
make smoketest         # Run smoke tests only

# Build and publish
make publish           # Generate OpenAPI spec
# Import build/validated-relationships-service-api.json into Postman to review the spec

Additional Resources

Last Updated: March 2026

FilesExpand file tree

DEPENDABOT_REVIEW_RUNBOOK.md

Latest commit

History

DEPENDABOT_REVIEW_RUNBOOK.md

File metadata and controls

Dependabot PR Review Runbook

Table of Contents

Overview

Prerequisites

Weekly Review Schedule

Review Process

1. Identify Open Dependabot PRs

2. Review PR Changes

3. Create Local Test Branch

4. Run Automated Tests

Install Dependencies

Run Linting

Run Schema Validation

Run Unit/Integration Tests

Test OpenAPI Specification

Test Postman Collection (if npm updated)

5. Manual Testing (if needed)

Test Sandbox

Review Breaking Changes

6. Approve and Merge

Handling Multiple Dependabot PRs

Option A: Test Each Separately (Recommended)

Option B: Combine and Test Together

Rollback Procedure

Immediate Rollback

Fix Forward (Preferred)

Common Issues and Solutions

Issue: Poetry lock file out of sync

Issue: npm peer dependency conflicts

Issue: Schema validation fails after OpenAPI validator update

Issue: Flake8 or Black failures after dependency update

Issue: GitHub Actions fail after actions update

Issue: Tests pass locally but fail in CI

Best Practices

Useful Commands Cheat Sheet

Git Commands

Testing Commands

Additional Resources