AI in Action: Key Branches and Applications
Computer Vision: Teaching Machines to See
6 min read
The problem of seeing
Human vision looks effortless. You glance at a street corner and instantly know: there's a parked car, a cyclist, a stop sign, a child on the sidewalk. You don't think about it. You just know.
Machines have no such luxury. To a computer, an image is a grid of numbers — pixel brightness values, stacked across three color channels (red, green, blue). There is no inherent "car" in those numbers. There is no "child." There is only a pattern of values that a well-trained model has learned to associate with those concepts.
Computer vision is the field devoted to that learning process: giving machines the ability to interpret and understand visual information from the world.
Images vs. video: a difference in complexity
Computer vision handles both still images and video, but video introduces a layer of complexity that still images don't have.
A still image is a single grid of pixels. The model looks at it once, makes a prediction, and moves on.
Video is a continuous sequence of image frames — typically 24 to 60 frames per second. To understand video, a model must not only interpret each frame but also track what's changing between frames. Where did that object come from? Where is it going? How fast is it moving? Is the object on the left the same person who was on the right three seconds ago?
This temporal dimension — understanding motion and continuity over time — makes video analysis significantly harder than image analysis.
The model families powering computer vision
Several distinct neural network architectures have emerged for visual tasks, each with its own strengths.
Convolutional Neural Networks (CNNs)
CNNs were the architecture that made modern computer vision practical. The key idea is spatial hierarchy. Rather than treating a 28×28 pixel image as 784 independent values, a CNN processes the image in local patches — small grids of pixels — and learns which local patterns matter.
Early layers in a CNN detect low-level features: horizontal edges, vertical edges, diagonals, corners. Middle layers combine those edges into shapes — curves, circles, rectangles. Later layers combine shapes into objects — wheels, faces, leaves.
This hierarchical feature learning mirrors how the human visual cortex works, and it makes CNNs extraordinarily efficient at tasks where spatial structure matters — which is nearly every image task.
Transformers in computer vision
Originally developed for language tasks, Transformers have increasingly been adapted for visual work. A Vision Transformer (ViT) breaks an image into a grid of patches — treating each patch the way a language model treats a word — and uses attention mechanisms to learn relationships between patches across the whole image.
Transformers have become particularly important in the generative AI wave. Systems that generate images from text descriptions rely heavily on Transformer-based architectures.
Generative Adversarial Networks (GANs)
GANs don't classify images — they create them. Two networks compete: a generator that produces synthetic images, and a discriminator that tries to tell real images from fakes. Each one improves by competing with the other. The result is a generator that can produce photorealistic images indistinguishable from real photographs.
GANs originated in computer vision research and remain important for image synthesis tasks. We'll see them again in the generative AI lesson.
Specialized architectures for specific visual tasks
Some domains demand architectures engineered for their particular constraints:
U-Net was developed for medical image segmentation — outlining the precise boundary of a tumor, a lesion, or an organ within an MRI or CT scan. Unlike classification (which assigns a single label to a whole image), segmentation assigns a label to every individual pixel. U-Net's distinctive "encoder-decoder" shape allows it to capture both the broad context of the whole image and the fine detail needed to trace precise boundaries.
EfficientNet was designed for environments where resources are constrained — smartphones, drones, edge computing devices. EfficientNet achieves high accuracy on a fraction of the computational budget of earlier architectures by systematically scaling the network's depth, width, and resolution together rather than expanding one dimension at a time.
Where computer vision is deployed today
Self-driving vehicles
Autonomous vehicles must simultaneously detect lane markings, read traffic signs, track pedestrians and cyclists, and identify the intentions of other drivers — reliably, at highway speeds. The stakes make accuracy requirements extreme. This is why self-driving remains a genuinely hard unsolved problem despite enormous investment.
Medical imaging
Computer vision systems trained on millions of labeled scans can now match or exceed human radiologists on specific detection tasks. Google's DeepMind demonstrated that its system could detect over 50 types of eye disease from retinal scans with accuracy comparable to world-leading specialists. Similar systems flag lung nodules in CT scans, detect diabetic retinopathy, and screen mammograms for early signs of breast cancer.
Face recognition
Your phone's Face ID, airport e-passport gates, and large-scale surveillance systems all run versions of the same underlying technology: a model that embeds a face into a numerical representation and compares it against a database. Accuracy disparities across demographic groups — a consequence of unrepresentative training data — and surveillance applications have made this one of the more contested areas of AI deployment.
Augmented and virtual reality
AR applications like Snapchat filters and Apple's spatial computing interface require computer vision to map the physical environment and register virtual objects onto it correctly. Your phone needs to understand the geometry of your face to put a filter in the right place. An AR headset needs to understand the walls of a room to anchor a virtual object to them.
Security and surveillance
Airports, stadiums, and large retail chains use computer vision for crowd monitoring, anomaly detection, and object identification — from detecting unattended luggage to identifying fire or crowd-crush conditions.
The trade-off: accuracy vs. interpretability
Computer vision models — especially deep CNNs — are famously opaque. When a model says "I'm 97% confident this mammogram contains a malignant mass," a radiologist cannot easily ask why. Techniques like Grad-CAM produce visual heatmaps showing which pixels most influenced the prediction, but these explanations are approximate.
In regulated industries (healthcare, finance, transportation), this interpretability gap is a real deployment barrier — and an active area of research.
Key takeaway
Computer vision gives machines the ability to interpret visual information by learning statistical patterns across millions of labeled images. CNNs introduced spatial hierarchy and made image classification practical. Transformers extended that capability to larger-scale and generative tasks. Specialized architectures like U-Net and EfficientNet handle domain-specific constraints. Today, computer vision powers everything from hospital diagnostics to autonomous vehicles — often exceeding human accuracy on narrow tasks, while still grappling with robustness, fairness, and interpretability challenges.
What's next?
Not all AI makes headlines with self-driving cars or robotic surgery. A huge portion of AI's economic value comes from much quieter applications in traditional business operations — predicting which customers will leave, forecasting demand, detecting fraud. That's the subject of the next lesson.