Similarity and Distance Metrics

"Closest" needs a precise definition

Every lesson so far has talked about vectors being "close" or "similar" without pinning down what that actually means mathematically. It turns out there isn't one single definition — there are several, and which one a system uses can meaningfully change its search results.

• Cosine similarity — the cosine of the angle between two vectors, ranging from −1 (opposite) to 1 (identical). It only cares about direction, not magnitude, which makes it a natural fit for text: two chunks can be similar in meaning regardless of how long they are. It's the default choice for most normalized embedding spaces.
• Squared Euclidean (L2) distance — the straight-line distance between two vectors, ranging from 0 (identical) upward with no ceiling. Intuitive and geometric, but sensitive to vector magnitude and, in very high-dimensional spaces, prone to the "curse of dimensionality" — distances start to look artificially similar to each other.
• Dot product — the product of two vectors' magnitudes and the cosine of the angle between them, ranging from −∞ to ∞. For normalized embeddings (unit length), the dot product and cosine similarity produce the same ranking — but the dot product is computationally cheaper, which matters at scale.
• Manhattan (L1) distance — the sum of the absolute differences between each corresponding coordinate, as if you could only move along grid lines rather than diagonally. More sensitive to differences in individual dimensions than L2, and more robust to outliers.
• Hamming distance — counts how many dimensions differ between two vectors outright. Useful for binary or categorical data, but it requires vectors of equal length and isn't a natural fit for dense text embeddings.

A worked example

Say a user asks "Tell me something about diseases." That query gets embedded into vector A. A candidate passage — say, one about the Black Death — gets embedded into vector B. Cosine similarity between them is computed as:

cosine_similarity(A, B) = (A · B) / (‖A‖ ‖B‖)

— the dot product of the two vectors, divided by the product of their magnitudes. Every candidate passage gets scored this way against the query, and the highest-scoring passages are returned as the search results.

Key takeaway

"Similarity" isn't one fixed formula — cosine similarity, Euclidean distance, dot product, Manhattan distance, and Hamming distance each define "close" differently, and the choice affects what a search system considers a good match. Cosine similarity and the dot product are the two most common choices for text embeddings.

What's next?

Picking a distance metric is only half of "how search actually runs at scale" — the other half is picking the vector database that stores and indexes your embeddings in the first place.