Guides · ← All guides

Build a Model Dependency Inventory Before Your Next Deprecation

Last updated: October 13, 2026 · 9 min read

By the Ingenious AI team

Ask a typical engineering team which AI models their product depends on, and you will get a confident answer — followed by a grep that finds three more. Model IDs hide in environment variables, feature flags, cron jobs, Jupyter notebooks, vendor SDK defaults, and the integration a contractor set up two years ago. A model dependency inventory is the fix: a living map of every model your product touches, what it powers, and how painful its retirement would be. Here is how to build one.

Why an inventory beats a search

When a deprecation notice lands, the team without an inventory starts with archaeology: searching repos, pinging ex-employees in Slack, reading Terraform nobody has touched in a year. The team with an inventory starts with a checklist. The difference is measured in days — days you may not have if the sunset window is short.

An inventory also catches the dependencies a code search structurally misses: models pinned by third-party tools on your behalf, embeddings baked into a vector store months ago, fine-tunes whose base model is the thing actually being retired. A grep finds strings. An inventory finds dependencies. Those are different jobs.

What a finished inventory looks like

Before the field list, it helps to see the target. A single entry in a healthy inventory reads something like this — entirely hypothetical, but exactly the shape you want:

Eight fields. Two minutes to fill in per entry once you know the answers. The answers are the work — and the work is what saves you later.

What to record for each model

• Model ID and provider: the exact string your code sends, plus the provider. Aliases and snapshots count as separate entries, because they carry different failure modes (pinned vs alias).
• Where it is referenced: repo, file, config key, or integration name. Link it, don't describe it. "The support bot config" is a description; support-bot/app/config.py:42 is a location.
• Features it powers: which user-facing capabilities break or degrade if this model misbehaves. This is the field that turns a model list into a triage tool.
• Criticality: a simple tier — revenue-critical, user-visible, or internal-only. This is what lets you triage when two deprecations land in the same month. Be honest here; everything cannot be tier one.
• Embeddings flag: whether the model's outputs are persisted — vectors, caches, exported datasets. Persisted outputs turn a migration into a data project, and that changes the timeline by an order of magnitude.
• Fine-tune lineage: if it is a fine-tune, which base model it descends from. The base is what gets retired; the fine-tune is collateral. OpenAI's fine-tuning rule makes this explicit: inference on fine-tunes lasts only until the base model is deprecated.
• Owner: a human who can answer "what does this do?" at 2 a.m. Unowned entries are the ones that surprise you. If nobody can own it, that is itself a finding.
• Last verified: the date someone confirmed this entry is still accurate. Inventories rot; the date is the antidote.

How to discover the hidden ones

Start with the obvious — grep for known model ID patterns and provider SDK imports — then hunt the hiding spots:

• Config and secrets stores: model IDs often live next to API keys, in the same vault entries nobody audits. Check .env files, CI secrets, and your secrets manager, not just the repo.
• Infrastructure as code: Terraform, CloudFormation, Dockerfiles, and deploy scripts pin versions that application code never mentions. The model setting in the deploy config is just as real as the one in the source.
• Vendor defaults: check what model your third-party tools use when you never specified one. LangChain and LlamaIndex default to gpt-3.5-turbo; a "managed" integration choosing a default is still your dependency. If you didn't choose it, you probably aren't watching it — which is exactly why it belongs in the inventory.
• Data artifacts: vector databases, embedding caches, and exported datasets carry the fingerprint of the model that made them. Record the generating model alongside the data, because the data outlives the model and the next migration has to reckon with it.
• Automation panels: n8n, Make, Zapier nodes, and serverless function environment variables each hold their own model setting. Grep can't reach them; your account dashboards can.
• Dashboard-stored prompts: if your code references prompt IDs or names instead of text, the actual model choice may live in the provider's dashboard. (Heads-up: OpenAI's v1/prompts API shuts down November 30, 2026 — copy that content into your code.)
• Shadow AI: ask teams what they actually use, not what is officially approved. The marketing team's automation, the support team's side project, the founder's weekend experiment that somehow serves production traffic — all of it counts. Shadow AI is where the retired model ID hides that nobody on the engineering team has ever seen.

Cross-check against reality

The repo tells you what is written down. The usage dashboards tell you what actually runs. OpenAI's usage page can group by model over the last 30 days; Anthropic's Console has a usage CSV export broken down by API key and model. Anything in the usage logs that wasn't in your repo grep is a shadow integration: note it, find its owner, add it to the inventory.

This step takes ten minutes and routinely finds the entry that would have hurt most. Do it first if you are short on time — it is the highest-yield ten minutes in the whole exercise.

Keep it alive with a cadence

An inventory is a process, not a document. Three habits keep it honest:

• Quarterly verification: every entry's "last verified" date gets refreshed — or the entry gets investigated. Tie it to an existing ritual, like a quarterly planning week, so it survives reorgs and roadmap churn.
• Change-triggered updates: any PR that adds or changes a model reference updates the inventory in the same PR. Make it a review checklist item: "model reference changed — inventory updated?" One line in the PR template, enforced by reviewers, beats any amount of good intentions.
• Deprecation-feed watch: assign someone — or a tool — to watch provider changelogs and announcement feeds, and cross-reference every announcement against the inventory the same day. An announcement you never see is the same as no announcement. The email goes to whoever created the API key; make sure that person, or their successor, is on a list that gets read.

The payoff: deprecation becomes a checklist

With a current inventory, a deprecation notice triggers a mechanical response: look up the model, read its criticality and blast radius, pull the eval harness, start the migration playbook. No archaeology, no guessing, no discovering the fourth usage on shutdown day.

The inventory is also the artifact auditors and acquirers ask for when they want to know your AI supply-chain risk. "Which models do you depend on, and what happens when each one retires?" is a due-diligence question now. Having the answer ready — with owners and verified dates — is a business asset, not just an engineering nicety.

FAQ

What is a model dependency inventory?

A living map of every AI model your product touches: the exact model ID and provider, where it is referenced, which features it powers, its criticality, whether its outputs are persisted, its fine-tune lineage, an owner, and a last-verified date. When a deprecation notice lands, the inventory turns archaeology into a checklist.

What should each inventory entry record?

The exact model ID and provider, where it is referenced (repo, file, config key, or integration), the features it powers, a criticality tier (revenue-critical, user-visible, or internal-only), whether its outputs are persisted (embeddings flag), its fine-tune lineage, a human owner, and the date the entry was last verified.

How do I find model dependencies that aren't in my code?

Check config and secrets stores, infrastructure-as-code files, framework defaults, third-party tools that pick a model on your behalf, automation panels like n8n, Make, and Zapier, dashboard-stored prompts, and your provider usage dashboards — the usage logs show what actually runs, which is often more than the repo shows.

The takeaway

You cannot manage what you cannot see, and model dependencies are nearly invisible by default. Spend a day building the inventory — exact IDs, owners, criticality, persisted-output flags — and put a quarterly heartbeat on it. The next deprecation notice will still be unwelcome, but it will arrive as a to-do list instead of a mystery.

ModelMinder keeps this inventory for you automatically — see how →