Dependabot alerts across 49 repos in one night: inventory, batch fixes, verification

Table of Contents

1. What did the inventory show?
2. What gets fixed first?
3. How is each repo fixed?
4. What if npm audit fix can't fix it?
5. How do you verify nothing broke?
6. What do you check after merging?
7. Why did the total barely move?

We inventoried the Dependabot alerts across 49 repos under one account in a single night. The process: pull every repo's open alerts with gh api; a 403 means scanning is off, so turn it on; group by severity and package, and fix only critical and high this round; work in a clean clone on a branch and record a build and test baseline before changing anything; run npm audit fix without --force, and use overrides where npm wants a downgrade; after the fix, compare tests one by one, actually load any native package that changed, and check in a browser that the page renders; after merging, wait 5 minutes and pull the alerts again.

The result first: after merging, the 5 public repos showed zero critical and zero high on GitHub. For the other 4 repos we worked on, local npm audit showed 3 at zero that night and 1 with 6 highs left, which a separate round cleared the next day. The account-wide total, though, only went from 1,230 to 1,205. The last section explains why.

#What did the inventory show?

On the evening of November 2 we went through 49 non-fork repos under one GitHub account: 1,230 open alerts, 29 critical and 534 high.

The first surprise was 403s. On 15 private repos, Dependabot alerts were simply turned off, and the API answered 403. One PUT turned each of them on; a 204 means it worked. Archived repos are not scanned at all, so we skipped them.

The second surprise: a freshly enabled repo showing 0 cannot be trusted. One repo with 1,552 packages in its dependency graph (SBOM) still showed 0 about two hours after we turned scanning on. Three others finished scanning later that night and added 178 alerts between them: one went from 15 highs to 3 critical and 63 high, another from 0 to 1 critical and 27 high. A tool returning 0 does not mean clean. It is the same trap as the one in our post on tools that silently return nothing.

The third surprise was branches. Dependabot only scans the default branch. In one of our repos the main development line is not the default branch, and the default branch stopped in June, so the 170 critical and high alerts GitHub showed for it described June's code. For a repo like that, run npm audit --json yourself on the branch you actually deploy.

The inventory script (Git Bash, with gh logged in):

OWNER=your-account
mkdir -p raw
gh repo list "$OWNER" --source --no-archived --limit 200 --json name --jq '.[].name' > repos.txt
while read -r r; do
  if ! gh api "repos/$OWNER/$r/dependabot/alerts?state=open&per_page=100" \
       --paginate --slurp > "raw/$r.json" 2> "raw/$r.err"; then
    echo "$r: $(head -c 100 "raw/$r.err")"      # a 403 usually means scanning is off
    rm -f "raw/$r.json"
  fi
done < repos.txt

# turn scanning on for one repo (204 means it is on)
# gh api -X PUT "repos/$OWNER/<repo>/vulnerability-alerts"

Always pair --paginate with --slurp. Without it, multiple pages of arrays are glued together as ][ and the JSON will not parse. --slurp cannot be combined with --jq, so the summary is a separate Node script (Object.groupBy needs Node 21 or later):

// summarize.mjs: count alerts per repo per severity, then list critical/high
import fs from 'node:fs'
const order = ['critical', 'high', 'medium', 'low']
const rows = []
for (const f of fs.readdirSync('raw').filter(f => f.endsWith('.json'))) {
  const pages = JSON.parse(fs.readFileSync(`raw/${f}`, 'utf8') || '[]')
  for (const a of pages.flat()) rows.push({
    repo: f.slice(0, -5),
    sev: a.security_vulnerability.severity,
    pkg: a.dependency.package.name,
    path: a.dependency.manifest_path,
    dep: `${a.dependency.relationship}/${a.dependency.scope}`,
    fix: a.security_vulnerability.first_patched_version?.identifier ?? '-',
  })
}
for (const [repo, rs] of Object.entries(Object.groupBy(rows, r => r.repo))) {
  console.log(repo, order.map(s => `${s}=${rs.filter(r => r.sev === s).length}`).join(' '))
}
for (const r of rows.filter(r => r.sev === 'critical' || r.sev === 'high')) {
  console.log([r.repo, r.sev, r.pkg, r.path, r.dep, `>=${r.fix}`].join('\t'))
}

The two most useful columns are relationship (direct or transitive) and scope (runtime or development). Direct runtime vulnerabilities go first; anything that only lives in the dev toolchain goes later.

#What gets fixed first?

This round was critical and high only. Most medium and low alerts are stuck behind major upgrades (whole chains like firebase-admin or eslint). That is not a one-night job, and forcing it would mean cutting corners on verification.

Once sorted, a lot of repos turn out to be stuck on the same handful of packages: undici and path-to-regexp (pulled in by a deployment tool's package), @grpc/grpc-js, protobufjs, tar, websocket-driver. The fix for a given package is nearly identical from repo to repo, so whatever worked and was verified in the first repo gets reused.

That night we worked on 9 repos: 3 core repos, 5 small public tools, and 1 product repo that deploys from a non-default branch. The other repos with critical or high alerts stayed on the list, in the queue. We did not pretend they were done.

#How is each repo fixed?

Every repo goes through the same script. The key part is the baseline before the fix:

#!/usr/bin/env bash
# fixrepo.sh <repo>: clean clone, baseline, audit fix, run again
r=$1; OWNER=your-account
W=/c/work/wt-dep-$r; L=/c/work/dep-logs; mkdir -p "$L"
rm -rf "$W" && gh repo clone "$OWNER/$r" "$W" -- -q || exit 1
cd "$W" && git checkout -q -b "fix/deps-$(date +%F)"

npm ci --no-audit --no-fund > "$L/$r-ci.log" 2>&1
npm run build --if-present > "$L/$r-build-before.log" 2>&1; echo "build before rc=$?"
npm test --if-present      > "$L/$r-test-before.log"  2>&1; echo "test before rc=$?"
git checkout -q -- .        # the build may touch tracked files; restore before fixing

npm audit fix --no-fund > "$L/$r-fix.log" 2>&1
npm run build --if-present > "$L/$r-build-after.log" 2>&1; echo "build after rc=$?"
npm test --if-present      > "$L/$r-test-after.log"  2>&1; echo "test after rc=$?"
npm audit --json > "$L/$r-audit.json"
git status --short

Why insist on a baseline? Because red tests after the fix do not mean you broke something. It happened three times that night: one repo's main branch already had 1 failing test before we touched it; one repo had the same 5 eslint errors before and after; and the product repo had 14 test files failing before the change, most of them needing an emulator. With before-and-after logs, all of these are ruled out at a glance. No false alarms, and no wandering off to "fix" unrelated things.

#What if npm audit fix can't fix it?

The short answer: no --force. Some npm audit suggestions are downgrades. That night it suggested rolling a deployment tool's package back to an older major version, and downgrading a PWA plugin, just to swap one nested package.

Use overrides instead, pointing only the vulnerable nested package at its patched version:

"overrides": {
  "@vercel/node": {
    "undici": "^6.29.1",
    "path-to-regexp": "6.3.0"
  },
  "@grpc/grpc-js": "^1.14.7"
}

Before changing overrides, delete the old lock entries for those packages, run npm install, then npm ls and look for invalid.

Where a major upgrade is truly needed, check the ranges your dependencies accept first. For example, to move an MCP server from zod 3 to 4, we first confirmed the MCP SDK accepts ^3.25 || ^4 and that the source code does not import zod directly. We also hit a bug in npm itself: npm 10.9.4 crashes installing vitest 4 with Cannot read properties of null (reading 'edgesOut'). npx -y npm@11 install works.

Some things we deliberately left alone. One repo had 6 high alerts left, all on the puppeteer 24 chain, a dev dependency. Clearing them meant puppeteer 25, a major version that needs Node 22.12 or later and touches 20-odd screenshot and render scripts. None of it ships in the production bundle, so it waited. The next day it got its own round: upgrade, confirm puppeteer launches and renders a page, and only then were the 6 highs gone.

#How do you verify nothing broke?

A passing build is not enough. We use four layers:

1. Compare tests one by one. Not just the totals: each test's result before and after. For the product repo's 60 test files, the results were identical.
2. Actually load any native package that changed. When sharp went from 0.34 to 0.35, we used it for a real PNG to JPEG conversion, a crop and an upscale, then rendered a share image with it. It counts only when you can see the output.
3. Check in a browser that the page renders. HTTP 200 does not mean the app mounted. The most common way a frontend dependency bump breaks things is a blank page.
4. In CI, read conclusion, not status. status=completed only means it finished. conclusion=success means it passed.

A minimal check for layer 3:

// mount-check.mjs: HTTP 200 doesn't count; #root has to contain something
import puppeteer from 'puppeteer'
const url = process.argv[2]
const browser = await puppeteer.launch()
const page = await browser.newPage()
const errors = []
page.on('pageerror', e => errors.push(e.message))
await page.goto(url, { waitUntil: 'networkidle0', timeout: 60000 })
const kids = await page.$eval('#root', el => el.children.length).catch(() => 0)
await browser.close()
console.log(kids > 0 ? `MOUNTED (${kids} children)` : 'EMPTY', errors.length ? `pageerror x${errors.length}` : '')
process.exit(kids > 0 ? 0 : 1)

An MCP server has no page to look at, so we run initialize and tools/list over stdio and check that every tool is still listed.

#What do you check after merging?

First, whether the merge deploys. One of the 5 public repos had Vercel's git integration turned on, so pushing to the default branch is a production deploy. Before merging, we confirmed the fix branch's preview built successfully. After merging, we confirmed the production URL pointed at the new deployment and the page rendered.

Then wait. We pulled the alerts at 4 minutes 43 seconds and at 5 minutes 15 seconds after the merge and got the same answer (we never tested earlier, so 5 minutes is our conservative wait): the 5 public repos went from 205 alerts to 2, one low and one medium, both in dev dependencies and waiting on upstream releases. After the merge, Dependabot opened a PR of its own to bump the remaining low.

Also, local npm audit and GitHub will disagree. For the same repo, local reported 12 moderate and GitHub counted 1. They match against different databases in different ways. We treat GitHub as the number of record and local as a reference.

#Why did the total barely move?

Two things canceled each other out that night. The repos we fixed closed more than 200 alerts, but the three repos that had just started scanning added 178. And three core repos had their fixes on branches or not yet pushed that night; they only show up after merging into the default branch GitHub scans and a rescan.

So the final report was one line per repo: before, after, what is left and why. The account-wide total is for trends only, never for deciding whether the job is done.

The batch fixes in this round were also run by several agents at once. How many you can run before the machine freezes is in another post.

---

Patching dependencies covers the supply chain side. The other side is whether your AI app's own system prompt holds up against injection. UltraProbe is our open source scanner: paste a prompt and see which defenses are missing.

FAQ

The Dependabot alerts API returns 403. Is that a permissions problem?

Usually not. Dependabot alerts can be switched off on private repos, and then GET /repos/{owner}/{repo}/dependabot/alerts returns 403. Turn it on with gh api -X PUT repos/{owner}/{repo}/vulnerability-alerts; a 204 means it is on. A freshly enabled repo can show 0 alerts for hours. That means the scan has not finished, not that the repo is clean.

npm audit fix suggests a downgrade or wants --force. Should I do it?

No. --force applies its suggestion as a major version change, and sometimes that change is a downgrade, such as rolling a framework back a major version just to swap one nested package. The safer route is an overrides entry in package.json that points only the vulnerable nested package at its patched version. Then run npm ls to check for invalid entries, and run the build and tests.

I merged the fix. Why is the alert on GitHub still open?

Three common reasons. Dependabot only scans the default branch, so a fix on another branch does not count. After merging, it has to rescan; we checked twice, about 5 minutes after the merge, and got the same answer both times. We never tested earlier, so we treat 5 minutes as a conservative wait. And if you are looking at an account-wide total, other repos that just had scanning turned on may be adding new alerts at the same time. Check repo by repo.