Where AI Models Actually Run: Compute and the Cloud

A question worth pausing on

When you type a question into a chatbot and a thoughtful answer appears a second later, where did that answer actually get produced? Not on your phone, and not on your laptop — those devices are just the window you're looking through. The real work happens somewhere else entirely: on powerful, specialized computers, often thousands of miles away, that you'll never see or touch.

Understanding where AI runs — and why it has to run there — fills in a piece of the picture that the rest of this course leans on constantly: every API call in Working with APIs, every "expensive to train" comment in The Importance of Open Source, and every "free GPU" mentioned back in Python — The Language of AI all point back to this same place.

Why AI needs unusual hardware

Ordinary computer chips — CPUs (Central Processing Units) — are excellent all-around problem solvers. Think of a CPU as one highly-skilled chef: capable of handling almost any recipe, one careful step at a time, in order.

AI doesn't need a chef. It needs a stadium-sized kitchen. Running a model means performing millions of small, repetitive math operations — mostly multiplying and adding numbers in large grids — and doing as many of them as possible at the same time. That's exactly what a GPU (Graphics Processing Unit) is built for. GPUs were originally designed to draw thousands of pixels on a screen at once for video games; it turned out that the same "do enormous numbers of simple calculations in parallel" design is precisely what AI models need to train and run. NVIDIA, originally a graphics-card company, has become one of the most valuable companies in the world largely because its GPUs power the AI boom.

	A CPU	A GPU
Best at	Doing one complex task very well, in sequence	Doing thousands of simple tasks at the same time
Analogy	One expert chef cooking an elaborate dish	A thousand cooks each chopping one vegetable, simultaneously
Role in AI	Manages the overall program	Does the heavy mathematical lifting models depend on

More chips than anyone could own

A single GPU helps — but training or running a frontier model takes thousands of them, wired together into one coordinated system, drawing enormous amounts of electricity and requiring specialized cooling. Almost no individual, and very few companies, could justify buying, housing, and maintaining hardware like that themselves.

This is the problem cloud computing solves. Companies like Amazon (AWS), Google (Google Cloud), and Microsoft (Azure) have already built giant data centers full of this hardware — and instead of keeping it to themselves, they rent out access to it, often by the second. Your code sends a request over the internet; their hardware does the work; the result comes back. You never see the machine, and you never have to buy one.

You've already brushed up against this

Remember Google Colab from the Python lesson — "Jupyter notebooks in the browser, with free access to GPUs"? That's cloud computing, scaled down and handed to learners for free. When you run a notebook in Colab, your code isn't executing on your laptop at all — it's running on a Google-owned machine somewhere else, and the results are streamed back to your browser. It's the exact same pattern as a billion-dollar AI company calling an API, just at a much smaller, friendlier scale.

The other half of "expensive"

Back in the open source lesson, you saw that very few organizations can afford to train a frontier model from scratch. Compute is a major reason why. Building and running an AI product costs money in two main places: acquiring good data, and paying for the computing power to train on it and serve it afterward. Bigger models need more of both — more data to learn from, and more GPU time to learn it and to respond to users afterward.

That's also why the open source community's push to shrink hardware requirements (mentioned in that same lesson) was such a big deal: every bit of computing power a model doesn't need is a cost — and a barrier to entry — that disappears. It's the difference between needing a data center and being able to run a capable model on a laptop, or even a phone.

Key takeaway

AI models don't run on your device — they run on enormous, specialized hardware (GPUs, built for doing huge numbers of simple calculations in parallel) housed in cloud data centers owned by companies like Amazon, Google, and Microsoft. Cloud computing is what makes that hardware rentable, turning a problem that used to require owning a supercomputer into something any developer can tap into with an API call and a billing account. Compute, alongside data, is one of the two forces that make AI expensive — and one of the two forces that open source has been working hardest to shrink.

What's next?

Renting compute gets you the hardware. The next question is: where do you get the models to run on it — especially if you don't want to train one from scratch? That's where Hugging Face comes in.