Putting It Together

Four days, a lot of models, a lot of version numbers. How do you choose, and what do you build when you combine them?

My answer is the same one I give for every agent system I build: don't pick a model, hire a team. One model per role, each chosen for the job it does best.

A design: the voice agent

Let me show what that looks like with a design that uses almost everything from this week. Imagine a multilingual voice assistant for a community event, the kind of help desk I'd love at an AgentCon: an attendee walks up, asks a question in whatever language they speak, and gets a spoken answer.

Here's the team:

The ears: MAI-Transcribe-2-Streaming. It transcribes while the person is still talking and detects the language on its own. Every millisecond saved here is time the agent can use to think.

The brain: MAI-Thinking-1. It takes the transcribed question, works out what's being asked, calls tools for the schedule or the venue map, and decides on an answer. This is where reasoning earns its cost.

The voice: MAI-Voice-2.1-Flash. It speaks the answer quickly, in the attendee's language, with the same voice no matter which language that is.

The eyes, when needed: MAI-Image-2.6. "Where is room B?" is easier to answer with a quick generated floor sketch on a screen than with a paragraph.

The safety net: a local model. If the connection fails, as conference Wi-Fi tends to do, a model running on the device itself can still answer the simple, common questions: Wi-Fi password, lunch time, where the toilets are. Not brilliant, but always there. That's Day 4's hybrid idea doing its job.

Notice the shape. The fast, cheap specialists sit at the edges where speed matters. The expensive thinker sits in the middle where judgment matters. And the local model sits underneath as the thing that keeps working when the cloud is out of reach. That last role is the lesson from my open weights post, built into the design: never depend on something you can't leave, or in this case, something that can leave you.

How to choose: four questions

When I decide which model gets which role, I ask four questions.

Does this job need reasoning, or just speed? Most tasks in an app are simple: classify, extract, transcribe, format. Give those to fast, specialized models. Save the reasoning model for the steps where a wrong answer is expensive.

Where does the data have to stay? If it can't leave a device or a country, that decides for you: a model on the device, or a cloud region that meets your requirements. Settle this before you compare benchmarks.

What does it cost at your volume? A model that's slightly better but costs four times as much is the wrong choice for a task you run a million times. Look at Day 3's prices per hour of audio and per image; at scale, those are the numbers that decide.

Can you leave? Every MAI model I covered this week either launched or got a new version in the last four months, and one was deprecated after three. That's progress, and it's also a warning. Build so you can swap any model in your team without rebuilding the app. A gateway helps here: when every model sits behind one API, trying a new one means changing a name, not rewriting code.

And the non-Microsoft models?

Microsoft Foundry also offers models from OpenAI, Anthropic, Meta, Mistral and many others. In my own projects I pick whichever model is best for each role, whoever made it.

What this week showed me is that Microsoft's own models have become real candidates for many of those roles, especially speech, images and fast coding, where they compete strongly on quality per euro. That's a different situation from a year ago, when the answer for most roles was simply "someone else's model."

What I take from this week

Microsoft now has answers to two different questions. MAI says: we can build competitive models ourselves, quickly, and specialized for real jobs. Windows says: and they can run where you are.

The smartest systems will use both. A cloud team for the hard and heavy work, local models for privacy, speed and reach, and an architecture that lets you change your mind next quarter, because next quarter there will be new versions again.

Thanks for following along this week. If you build something with these models, I'd love to see it. Show me the demo, and especially show me the part that broke.