How Computers Actually Read Data

The translation problem

A computer doesn't "see" a photo of a cat. It doesn't "hear" a voice recording. It only processes numbers — and specifically, numbers in binary form (0s and 1s). Everything a machine learning model ever learns from has to pass through this bottleneck first.

This lesson explains how that translation works, and why understanding it gives you a clearer picture of what AI can and can't do.

MNIST: the "Hello World" of machine learning

When programmers learn a new language, they almost always start with a "Hello World" program — the simplest possible thing that shows the language works. In machine learning, the equivalent is the MNIST dataset.

MNIST is a collection of 70,000 handwritten digits — the numbers 0 through 9 — written by thousands of different people. Each image is 28 pixels wide by 28 pixels tall, in grayscale. The goal: train a model to look at an image and correctly identify which digit it shows.

It sounds simple, but it's surprisingly hard. Everyone's handwriting is different. A "7" scrawled by one person looks almost nothing like a "7" written by someone else. A computer seeing these images for the first time has no concept of what a "7" is supposed to look like.

The MNIST database: a grid of handwritten digits 0–9 showing natural variation across writers, with a zoomed-in 28×28 pixel example of a handwritten 3

How images become numbers

Each pixel in an MNIST image has a single value from 0 to 255:

0 = pure white (the paper)
255 = pure black (the ink)
Values in between = shades of gray

A 28×28 image contains exactly 784 pixels. So a single handwritten digit becomes a list of 784 numbers, each between 0 and 255 — something like:

[0, 0, 0, 12, 87, 204, 255, 230, 180, 45, 0, 0, ...]

That list is what the machine learning model actually receives as input. Not an image — a sequence of numbers.

Everything is binary underneath

Those pixel values (0–255) are themselves stored in binary. The number 255 in binary is 11111111 — eight 1s. The number 87 in binary is 01010111. Every piece of digital information — every pixel, every audio sample, every character of text — is ultimately stored as a string of 0s and 1s.

This is not unique to images. The same principle applies to everything:

Text: each character maps to a number (the letter "A" is 65 in ASCII), stored in binary
Audio: a sound wave is sampled thousands of times per second, each sample recorded as a number
Video: a sequence of image frames, each frame a grid of pixel values, played at 24–60 frames per second

Whatever the original format, the computer reduces it to numbers, which it stores in binary. Machine learning models work on those numbers.

How a model learns from numbers

Once data is in numerical form, training can begin. Using MNIST as the example:

The model receives the 784 pixel values for one image
It makes a prediction: "I think this is a 3"
It checks the correct answer: it was actually a 7
It adjusts its internal settings slightly to do better next time
Repeat — for all 70,000 images, many times over

After enough repetitions, the model learns which combinations of pixel values correspond to each digit. It has never been told "a 7 has a horizontal stroke at the top and a diagonal line going down" — it figured that out entirely from patterns in the numbers.

From images to everything else

The MNIST example generalizes directly. AI researchers apply the same logic to every kind of data:

Medical scans: an MRI becomes a 3D grid of pixel-like values; a model learns which patterns indicate a tumor
Voice assistants: audio is converted to a numerical waveform; a model learns which patterns correspond to words
Sentiment analysis: text is converted to numbers (word embeddings); a model learns which numerical patterns correspond to positive or negative tone
Self-driving cars: cameras produce pixel arrays; radar and lidar produce distance measurements; a model learns which combinations mean "pedestrian crossing"

The translation layer — reality → numbers → model — is always the same. The model never touches raw reality. It only ever sees numbers.

Where the data comes from

For this numerical pipeline to work, you first have to collect data. Common methods include:

Web scraping: automatically downloading text, images, and other content from websites
APIs: pulling structured data feeds from services (weather data, financial prices, social media)
Sensors and IoT devices: cameras, microphones, temperature sensors, GPS units producing continuous streams
User activity: clicks, purchases, search queries, app usage — collected at scale by large platforms

And once you have the data, quality matters enormously. The old engineering adage applies directly: "Garbage in, garbage out." A model trained on blurry images won't recognize sharp ones reliably. A model trained on biased text will reproduce that bias. The quality of a model's output is capped by the quality of its training data.

Key takeaway

Computers only process numbers. To learn from images, audio, video, or text, AI systems must first translate that information into numerical form. In images, that means pixel values (0–255 per pixel). In audio, it means sampled waveform values. The model trains by repeatedly seeing these numbers, making predictions, checking the answers, and adjusting until it gets good. The MNIST handwriting dataset is the simplest working example of this whole pipeline.

What's next?

Numbers alone aren't enough. The model also needs to know what each example means — is this a 7 or a 9? That's the role of labels, and the choice of whether to use them or not is one of the most important decisions in machine learning.