Budgeting and the Cost of Building AI

What actually makes an AI model powerful?

Two factors largely determine how capable an AI model is: the size and quality of its dataset, and the size of the model itself.

A larger, higher-quality dataset gives a model more to learn from. A larger model — one with more parameters — has more capacity to absorb what's in that dataset and capture intricate patterns.

Here's a useful way to picture it: think of the dataset as a library full of books, and the model as a student learning from that library. The more high-quality books the library holds, the more there is to learn from. And the more a student can remember and internalize, the more of that library they can actually put to use. A bigger, more capable model is like a student with a larger capacity to absorb and apply what the library contains.

All else being equal, more (and better) data plus a larger model tends to mean stronger performance.

But "biggest" isn't the same as "best"

Here's the catch: building the largest possible model isn't automatically the right strategy, because there's a sharp trade-off between performance and cost.

High-quality, large-scale datasets are expensive to acquire. The computing infrastructure needed to train a very large model is even more expensive to build and run. And bigger models don't just cost more to train — they cost more to operate afterward, too.

This isn't a small-stakes trade-off. Sam Altman has revealed in an interview that training GPT-4 cost OpenAI over $100 million. Numbers like that change how organizations have to think.

Diagram — What Drives Capability vs. Cost

You have to budget before you build — not during

Here's the part that catches people off guard: you can't start building an AI model and figure out the size and cost as you go. Most organizations simply cannot afford multiple attempts at training a large model — the cost of a single failed run can be enormous, and they often have to get it right on the first try.

That means budgeting has to happen up front, as a single decision made before any training begins: set a total spend, then split it deliberately between acquiring data and buying computing power. Treating that split as something to "figure out once we see how it goes" isn't viable at this scale — it's a foundational skill that AI developers and the organizations they work for need to get right from day one.

When smaller is the smarter choice

Given all this, building the biggest model possible should never be the default goal. It only makes sense when a significant boost in capability clearly aligns with what the organization actually needs and can afford.

In many cases, the better strategy is to build smaller, specialized models — ones that can be retrained and fine-tuned frequently as needs change. Training a smaller model focused on a specific application is typically faster and more cost-effective than pouring vast resources into a large, general-purpose model that may be overkill for the task at hand.

In other words: match the size of the model to the size of the problem you're actually trying to solve — not to the size of the headlines.

Key takeaway

An AI model's power comes from the combination of dataset size/quality and model size — but bigger always means more expensive, both to train and to run. Because most organizations can't afford to experiment their way to the right size (training GPT-4 reportedly cost OpenAI over $100 million), budgeting has to happen before training begins, with clear decisions about how much goes toward data versus compute. Often, a smaller, specialized model that can be fine-tuned over time is the more practical — and more cost-effective — choice.

What's next?

Cost is one constraint on AI systems. Speed is another — and it's one that users feel directly, every time they wait for a response. The next lesson looks at latency: why generating text takes time, and why that matters so much for products people actually use.