Don't marry a model: why your AI setup should survive a swap

The question I get most often about AI systems is "which model do you use?" It's a reasonable question with an unsatisfying answer: it barely matters. The question that matters is how cheaply you can change the answer.

The ground moves every few months

The model market has a rhythm now. Every few months a new release leapfrogs the field - better at some class of task, or dramatically cheaper, or both. Prices for a given level of capability keep falling. Providers deprecate old models on their own schedule, whether you were done with them or not.

Any specific model choice made today is a snapshot of a moving market. Six months from now there will be a better or cheaper option for at least part of what your system does. That's not a risk to hedge against - it's a benefit to position for. A business running AI at any real volume should expect its cost per task to fall and its quality to rise over time, without a rebuild. That only happens if the system is built to let it.

What being married looks like

The wrong version is easy to build, which is why it's common: the provider's SDK called directly from forty places, prompts tuned to one model's quirks scattered through the codebase, output parsing that assumes one model's habits. It works fine - until the provider raises prices, deprecates the version you're on, or a competitor ships something twice as good for a third of the cost. Then every one of those forty places is a migration, the swap becomes a project, the project never gets scheduled, and the system quietly falls behind the market while paying last year's prices.

Committing your architecture to one model isn't backing a winner. It's opting out of every future improvement that doesn't come from that one vendor.

What an open setup looks like

The right version isn't complicated:

• One interface in front of every model call. The rest of the system asks for work; which model answers is configuration, changed in one place - per task, not globally.
• Prompts and evals owned per task, so any candidate model can be dropped in and scored on your real cases before it touches production.
• The eval suite as the swap's safety net. This is the piece that makes model changes boring instead of frightening: run the new model against the golden dataset, read the report, decide on evidence. Without evals, every swap is a leap of faith, so nobody ever swaps.

With this in place, adopting a newly released model is an afternoon of testing and a config change - not a quarter of migration work.

Different steps, different models

The other thing an open setup unlocks: you stop using one model for everything, because nothing about your workflow says you should.

A typical automation has a few model-touched steps, and they're not equal. Pulling fields out of a semi-structured document at high volume is easy work - a small, fast, cheap model does it indistinguishably from a frontier one, at a fraction of the cost. Judging an ambiguous edge case or drafting customer-facing text is hard work - that's where the frontier model earns its price. Matching each step to the cheapest model that passes its eval usually beats one-premium-model-everywhere on cost and on latency, with no quality loss you can measure.

There's a whole discipline in doing this systematically - driving the running cost of an AI system down over time while holding quality constant. That deserves its own article, and I'll write it. The multi-model setup is the prerequisite; without it there's nothing to optimize.

The question to ask your builder

Whoever builds your system - me or anyone else - ask them: what does it cost us to change models?

The answer you want is "an eval run and a config change." If the answer involves the word "project," you're being wired into somebody's ecosystem, and you'll pay for that either in switching costs later or in missed improvements forever. If you're not sure which kind of system you have - or you know, and it's the wrong kind - I can help with both the answer and the fix.