When the model deprecation notice lands: how we split the gemini-2.5-flash migration into four steps

Table of Contents

1. Step 1: Inventory, and sort every call into three kinds
2. Step 2: Test a parameter matrix
3. Step 3: Move to config and switch one feature at a time
4. Step 4: Compare quality on real inputs
5. Found along the way: broken links in the fallback chain
6. Checklist

On November 4 we got Google's notice: gemini-2.5-flash shuts down on 2026-12-01, and requests after that will be redirected to gemini-3.5-flash-lite. The notice also said plain text requests would carry over, while calls using advanced parameters (thinking budget, cached content) might break.

Our first thought was that the redirect meant there was nothing to do. One request made us worry: calling 3.5-flash-lite directly with thinkingBudget, even thinkingBudget: 0, returns a 400. The notice doesn't say whether the redirect will fix parameters for us. And almost every 2.5 call in our code sends thinkingBudget (usually 0, to turn thinking off and save tokens).

We split the migration into four steps: inventory, test the parameters, switch per feature through config, and compare quality on real inputs. The code and the comparison for all four were done the same day; merging and switching models happen in stages. Each step below includes what we actually used, so you can follow along.

#Step 1: Inventory, and sort every call into three kinds

First find every place that names a model. We use git grep, so it works without any extra search tool installed:

git grep -nE "gemini-2\.5|thinkingBudget|thinkingConfig|cachedContent|responseSchema|GEMINI_MODEL"

Record five columns per hit: file and line, feature, where the model name comes from, which advanced parameters it sends, and last month's call count. "Where the model name comes from" has three kinds:

• Hard-coded: the string is in the code. Changing it means a code change and a deploy.
• env: read from an environment variable. Change the setting and redeploy.
• config: read from a database setting. Takes effect within a minute, no deploy.

Call volume sets the order. In MindThread, the patrol comment judgment made 83,333 calls in October, while the content safety classifier made zero (it is switched off for now). But volume is not the only risk. The inventory showed that MindThread's main paths go through the fallback chain we shipped at the end of September, so they will not stop on December 1; at worst each call hits one 400 before moving on. What actually breaks are two paths that are hard-coded with no fallback: the content safety classifier, off today and broken the moment it is turned back on, and a local content QA job that runs at 3:30 a.m. with thinkingBudget: 1024, which would fail all night after December 1.

There is also a batch of retired backend scripts, all hard-coded to 2.5. We checked Task Scheduler and nothing runs them, so we left them alone and wrote one line in the report: anyone who revives them will get a 400 after December 1.

#Step 2: Test a parameter matrix

Don't guess which model accepts which parameter. Send the same prompt to every combination. Our results on November 4:

Model	thinkingBudget: 0	thinkingLevel: low	thinkingLevel: minimal	none
gemini-2.5-flash	200	400	400	200
gemini-2.5-flash-lite	200	400	400	200
gemini-3.5-flash	200	200 (61 thinking tokens)	200	200
gemini-3.5-flash-lite	400	200	200	200
gemini-3.7-flash	200	200	400	200

The table broke two assumptions. First, you cannot switch every call to thinkingLevel, because 2.5 rejects it. Second, 3.x is not one rule set: 3.5-flash-lite rejects the budget, 3.7-flash rejects minimal, and 3.5-flash with low still thinks for about 60 tokens, which costs money.

The probe is short. Swap in the models you care about and run it (the key comes from an environment variable):

// probe.mjs: node probe.mjs
const key = process.env.GEMINI_API_KEY
const models = (process.env.MODELS || 'gemini-2.5-flash,gemini-3.5-flash-lite,gemini-3.7-flash').split(',')
const variants = {
  none: undefined,
  budget0: { thinkingBudget: 0 },
  levelLow: { thinkingLevel: 'low' },
  levelMinimal: { thinkingLevel: 'minimal' },
}
for (const m of models) for (const [name, tc] of Object.entries(variants)) {
  const generationConfig = { temperature: 0, maxOutputTokens: 64, ...(tc && { thinkingConfig: tc }) }
  const r = await fetch(`https://generativelanguage.googleapis.com/v1beta/models/${m}:generateContent`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json', 'x-goog-api-key': key },
    body: JSON.stringify({ contents: [{ parts: [{ text: 'Reply with one word: hello' }] }], generationConfig }),
  })
  const j = await r.json()
  console.log(m, name, r.status, 'thoughts=' + (j.usageMetadata?.thoughtsTokenCount ?? 0), r.ok ? '' : j.error?.message)
}

Once the matrix is done, turn "thinking off" into a function of the model instead of scattering it across calls:

// "Thinking off" written the way each model accepts it; spread into generationConfig
export function thinkingOff(model) {
  if (/^gemini-2\./.test(model)) return { thinkingConfig: { thinkingBudget: 0 } }
  if (/^gemini-3\.[0-9.]*-flash-lite/.test(model)) return { thinkingConfig: { thinkingLevel: 'low' } }
  if (/^gemini-3\./.test(model)) return { thinkingConfig: { thinkingBudget: 0 } }
  return {} // aliases and untested models get nothing; a wrong field is a 400
}

One more detail: on 3.x, thinking tokens count against the output limit, so when MindThread switches to 3.x it adds 2,048 tokens of output room to avoid truncated answers.

#Step 3: Move to config and switch one feature at a time

The obvious move after the inventory is to replace every model name at once. We didn't, because features have different quality needs and should switch, and roll back, separately.

MindThread gained one database setting, config/geminiModels:

{
  "features": { "haixun-reply": "gemini-3.5-flash-lite" }
}

This switches a single feature. The other two fields: primary replaces every chain whose primary is 2.5 in one go, so we only set it after every feature has been checked; disabled is for the night before shutdown.

Every call already carries a feature label (for example cloud-patrol/judge). The resolution rules:

1. features matches the full label first, then shorter prefixes (cloud-patrol/judge, then cloud-patrol). It wins over everything else.
2. primary only replaces chains whose primary was 2.5; chains already running a canary test stay as they are.
3. disabled is applied last and removes the listed models. If that empties the chain, fall back to the original; one extra hop beats having no model.
4. Only pinned names already on the chain are accepted. Aliases and untested names are ignored.

A simplified version (reading the setting, caching and the name list details are left out):

export function resolveChain(cfg, label, defaultChain, chainWithPrimary) {
  const pinned = (m) => typeof m === 'string' && defaultChain.includes(m)
  let override = null
  const parts = String(label || '').split('/')
  for (let n = parts.length; n > 0 && !override; n--) {
    const m = cfg?.features?.[parts.slice(0, n).join('/')]
    if (pinned(m)) override = m
  }
  let chain = override ? chainWithPrimary(override)
    : pinned(cfg?.primary) && defaultChain[0] === 'gemini-2.5-flash' ? chainWithPrimary(cfg.primary)
    : [...defaultChain]
  const off = Array.isArray(cfg?.disabled) ? cfg.disabled : []
  const kept = chain.filter((m) => !off.includes(m))
  return kept.length ? kept : chain
}

The setting is cached for 60 seconds, and if it can't be read the code defaults apply. When the document doesn't exist, behavior is exactly what it was before, so the compatible code can ship first and models can switch one by one later. We added 9 tests that run before every build, including a static check that the two files which used to hard-code 2.5 never contain "2.5 plus thinkingBudget" again.

Our smaller codebases (Ultra Advisor, the ultralab.tw site, internal scripts) have no database setting, so they read an environment variable instead, still defaulting to 2.5. Merging changes nothing; switching means setting one variable and redeploying.

#Step 4: Compare quality on real inputs

The rule is simple: take October's real inputs, call the same production function (same prompt, same post-processing, same quality gate), and run each model once. The comparison harness turned every database write into a no-op, 151 of them across two batches. It ran on our own internal quota, never on a member's. The bar for switching early was 95% agreement.

Feature	Samples	Agreement	What changed
Extension reply suggestions	30	100%	Median 30 vs 26 characters, no systematic drop
Patrol comment judgment	30	90%	All 3 disagreements were 3.5 being stricter; "worth commenting" went from 18 to 15
Auto-replies	30	90%	All 3 disagreements were 3.5 usable and 2.5 blocked by the quality gate; but one 3.5 reply kept an extra pair of outer quotes
Post generation	14	100%	Posts got clearly shorter, median 274 to 177 characters

Post generation has 14 samples because that is how many accounts had both auto-generation and a persona in October. We didn't pad the set.

The outcome: extension reply suggestions met the bar and have already been switched. The other three go to human review. The part worth remembering is that agreement is not the only metric. Post generation agreed 100% but got about a third shorter, which may fall short for personas that require 150+ characters. Patrol judgment got stricter, which we estimate cuts volume by about a fifth, so members might not reach the numbers they set. Agreement rate shows neither.

The last step is scheduled for the evening of November 30: add disabled: ["gemini-2.5-flash"] so that calls after December 1 stop hitting a 400 first.

#Found along the way: broken links in the fallback chain

While building the matrix, we also called every model in the site's fallback chain one by one. The 4th, gemini-flash-lite-latest, now points at 3.5-flash-lite, and we were calling it with thinkingBudget: 0, a 400 every time. The 5th, gemini-2.0-flash, was shut down on June 1. So whenever the primary was busy, only gemini-flash-latest was left, and on that day it returned 503. The fix was to put pinned 3.5-flash-lite first and drop the retired 2.0; it ships together with the compatible code.

The other finding was MindThread's nightly content refill. The setting said the primary was 2.5, but since October 28 more than 90% of the batches each night had already fallen through to 3.5-flash-lite (86 to 6 on November 4), because the 2.5 quota ran out early.

Both point to the same lesson. We wrote before about whether a fallback silently upgrades you to the most expensive model; that was the cost side. This is the other side: a fallback chain hides its broken links. Everything looks fine, so nobody looks.

#Checklist

1. git grep every model name and advanced parameter; tag each as hard-coded, env or config, with last month's call count.
2. Find the paths that are hard-coded with no fallback first. They are the only ones that fully break on shutdown day.
3. Run the probe across old and new models and every parameter variant; make the matrix.
4. Write "thinking off" as a function of the model and delete the scattered hard-coded parameters.
5. Read model names from config, keep the old model as default, ship, and confirm nothing changed.
6. For each feature, compare old and new on real inputs; check agreement, length and volume; switch the ones that pass.
7. Test every link of the fallback chain on its own, and log which model actually answered each call.
8. The night before shutdown, remove the old model from the chain.

FAQ

The notice says requests will be redirected automatically. Why change anything?

Because the notice doesn't say whether the redirect will rewrite your request. Plain text requests carry over, but nearly all of our calls send a thinkingBudget, and calling gemini-3.5-flash-lite directly with any thinkingBudget, even 0, returns a 400. If the redirect leaves parameters alone, paths with a fallback chain hit one 400 and move to the next model, and hard-coded paths with no fallback simply break. We didn't want to bet on it, so we switch both the model name and the parameters ourselves before the shutdown date.

How much agreement between the old and new model is enough to switch early?

We set the bar at 95%. Take last month's real inputs, call the same production function, and run each model once. Reply suggestions in our browser extension agreed on 30 of 30, so we switched them. Patrol judgment and auto-replies were both at 90%, and post generation agreed on all 14 samples but came out about a third shorter, so those three go to a human review first. Look at length and volume too; agreement rate does not show them.

Wouldn't an alias like gemini-flash-latest mean we never have to migrate?

An alias can jump to a new generation without telling you, and the parameter rules jump with it. This time we found gemini-flash-lite-latest already pointing at 3.5-flash-lite, and our fallback chain was calling it with thinkingBudget, which had been a 400 every time. Our primary model setting now accepts only pinned names, so we decide when to change generations and we test first; the tail of the fallback chain still keeps aliases as a last resort, but calls to an alias carry no thinking parameter at all.