The AI Model Migration Playbook: Switch Models Without Breaking Production

Q: What is a canary cutover for a model migration?

Routing a small, growing percentage of traffic (1%, then 10%, then 50%) to the replacement model while watching task success rate, complaint rate, latency, and cost per request — keeping the old model as an instant rollback target until the new one has survived at least one full business cycle at 100%.

Last updated: October 13, 2026 · 10 min read

By the Ingenious AI team

The deprecation notice just landed: the model your product runs on retires in 90 days. Panic is optional; a playbook is not. Migrations fail when teams treat them as a one-line config change — swap the model ID, ship it, hope. They succeed when teams treat them as what they are: a behavior change to your product's brain, deployed under a hard external deadline. Here is the sequence that works.

One framing note before the steps. The deadline is external and fixed, but your buffer is a choice. Start the week the notice lands, not the month it expires. Everything below assumes a roughly 90-day window; compress it if yours is shorter, but don't skip steps to save days — skipped steps are how migrations become incidents.

Step 1: Audit your blast radius (days 1–3)

Before choosing a replacement, find everything the old model touches. Search your codebase for the model ID and its common aliases, check environment variables and config files, and — critically — check the places code search misses: cron jobs, one-off scripts, notebooks, serverless function configs, automation panels (n8n, Make, Zapier), dashboard-stored prompts, and third-party integrations that pin the model on your behalf. Then there's the vector store built on its embeddings, which is a migration inside your migration.

For each hit, record which user-facing feature it powers and how bad a behavior change would be. A support-bot greeting and a fraud-classification call do not deserve the same caution. The output of this step is a ranked list: what breaks loudly, what degrades quietly, and what nobody will notice. You will allocate your eval effort in that order. (The full technique lives in our 20-minute codebase audit guide.)

Step 2: Shortlist replacements (days 3–7)

Start with the provider's recommended successor, then add one or two alternatives — including at least one from a different provider. Single-provider dependency is how you end up running this playbook under duress next time.

Compare on four axes, in this order:

Capability on your actual tasks — not public benchmarks. A model that tops a leaderboard can still be worse at your weird, domain-specific extraction format.
Behavioral fit — verbosity, refusal style, structured-output reliability, how it handles your edge cases. This is where most "clean" migrations quietly fail.
Price at your real token volumes — per-token price times your actual prompt and completion sizes, not the headline rate. Reasoning-style models bill tokens you never see in the response.
Operational maturity — rate limits, uptime history, and, tellingly, how the provider handled past deprecations. A provider that gave 30 days last time will give 30 days again.

Write the shortlist down with the reasoning. Six months from now, when someone asks why you picked this model, the note is the answer.

Step 3: Evaluate with your real prompts (days 7–14)

This is the step teams skip and regret. Benchmarks tell you how a model does on someone else's tasks; your prompts tell you how it does on yours.

Collect a representative set — the top 20 to 50 prompts by production volume, including the weird edge cases, the prompts with the longest context, and the ones with strict output-format requirements. Run them against each candidate. Then diff the outputs, and do it deterministically where you can: exact-match rates for structured outputs, keyword retention for the facts that must survive, schema-validity checks for JSON. Where the output is free text, have a human or an LLM judge score the pairs blind — but anchor the judgment to specific criteria, not vibes.

What you are looking for is not "the best model." It is "the model that changes my outputs the least in the ways that matter." That framing changes the decision more often than people expect: the highest-scoring candidate on a benchmark is frequently not the least-disruptive one on your prompts.

Save everything. The eval set you build here becomes your regression suite for the next migration — and the reason the second one takes half the time.

Step 4: Fix what the eval found (days 14–21)

The eval will surface surprises. The new model is chattier, so your JSON extraction breaks. It refuses a category the old model handled. It answers in a different register — friendlier, or more formal — and your product's voice shifts with it. Its tool-call format differs just enough to break your parser.

Fix these with prompt adjustments and output parsing, not wishful thinking. Add the explicit instructions the new model needs: "respond with JSON only," "limit to two sentences," "use the exact labels from this list." Re-run the eval after each fix until the diff is clean on everything you classified as must-not-change in step 1.

One judgment call lives here: how much prompt surgery is too much? If the replacement needs three paragraphs of compensating instructions to behave like the old model, that is a smell. A candidate that needs heavy prompt scaffolding is a candidate that will surprise you again later. Prefer the model that is boringly close to your current behavior.

Step 5: Canary the cutover (days 21–45)

Never flip 100% of traffic at once. Route a small percentage — 1%, then 10%, then 50% — and watch the metrics that matter for each feature: task success rate, user complaint rate, latency, cost per request.

Set your rollback triggers before the canary starts, not during. Something like: complaint rate up more than X% at any stage, task success down more than Y points, cost per request up more than Z% — halt and roll back. Writing the thresholds down in advance is what keeps a 2 a.m. decision honest.

Keep the old model as the instant rollback target until the new model has survived at least one full business cycle at 100%. A weekend of clean traffic is not the same as a month-end close, a product launch, or whatever your peak looks like. The deprecation deadline is real, but "migrated" means "migrated and stable," not "migrated and on fire."

Step 6: Handle the special cases

Two migrations-within-the-migration deserve their own plans:

Embeddings. If the old model produced your vectors, you are re-embedding the corpus. Budget the compute, run old and new indexes in parallel during the transition, and validate retrieval quality on your own queries — not just cosine-similarity averages. The retrieval test that matters is "do my users' real questions still return the right documents," and only your query logs can answer it.
Fine-tunes. A fine-tune dies with its base model — OpenAI's rule is that inference on fine-tunes lasts only until the base is deprecated. Decide early whether to retrain on the new base or replace the fine-tune with a well-engineered prompt on a stronger model. The latter is often cheaper and more portable next time, which is a point worth weighing honestly rather than defending the sunk cost of the training run.

Step 7: Validate the economics (before the deadline)

Run the new model's real per-request cost against your actual volume for a week of canary traffic, then project it. Token prices, context-window usage, and output verbosity all move the number — and a chattier replacement can quietly double your bill while passing every quality gate.

Do the arithmetic with your own numbers, but here is the shape of the check, with example figures to show the method: suppose the old model cost you $0.004 per request at 50,000 requests a day — $200 a day. The replacement passes evals beautifully but averages 40% more output tokens and a higher per-token price, landing at $0.007 per request. That is $350 a day, $4,500 a month of new burn you did not budget. Maybe it is worth it. Maybe a cheaper candidate that scored slightly lower is the right business call. The point is to make that decision with a spreadsheet, not discover it in next month's invoice.

Step 8: Document and decommission

When the cutover is stable: remove the old model ID from every config, archive the eval results (they are your evidence if anyone asks why outputs changed), update your model inventory, and set a calendar reminder to re-verify the replacement's own deprecation status quarterly.

Then — and this is the part that compounds — write down what surprised you. Which eval case caught the real problem. Which candidate looked good on paper and failed on your prompts. How long each step actually took versus the plan. The next deprecation notice is already coming; your future self wants the notes. Teams that run this playbook once find the second migration takes half the time, because the inventory, the eval harness, and the notes already exist.

FAQ

How long does an AI model migration take?

For a typical production feature, budget four to six weeks: a few days to audit the blast radius, a week to shortlist and evaluate replacements on your real prompts, two to three weeks for a canary cutover with a full business cycle at 100%, plus time to fix whatever the eval surfaces. Teams with a standing eval harness and a model inventory routinely do it in half that.

Should I just swap the model ID and ship?

Almost never. The replacement model has different instruction-following quirks, verbosity, refusal boundaries, and structured-output reliability. Without evaluating it on your real prompts first, you are deploying a behavior change to your product's brain and hoping for the best.

What is a canary cutover for a model migration?

Routing a small, growing percentage of traffic — 1%, then 10%, then 50% — to the replacement model while watching task success rate, complaint rate, latency, and cost per request, keeping the old model as an instant rollback target until the new one has survived at least one full business cycle at 100%.

The takeaway

Model migration is a discipline, not a scramble: audit, shortlist, evaluate on real prompts, fix, canary, validate costs, document. The deadline is external; the readiness is yours to build.

ModelMinder's one-click migration evals automate step 3 — see how →