Loading, Splitting, and Storing Documents

Documents don't start out as clean text

A PDF isn't a string — it's a mix of headings, paragraphs, tables, bullet lists, page breaks, and formatting noise. The first real step in a RAG pipeline is parsing: opening the raw file and breaking it into a structured list of elements, each tagged with useful metadata like its element type (heading vs. paragraph vs. list item), the file path it came from, and which page or section it appeared on.

That metadata matters more than it might seem. It's what lets a finished RAG system tell a user not just what the answer is, but where it came from — "see page 12 of the Refund Policy" is a far more trustworthy answer than an unsourced one.

From elements to a vector store

Once a document has been parsed and split into chunks (covered in depth over the next few lessons), the same embedding model does two jobs:

• At indexing time, it embeds every chunk and writes the resulting vectors — along with their metadata — into a vector store.
• At query time, it embeds the incoming question the same way, and that query vector is compared against everything already stored.

Because both documents and queries pass through the same embedding model, their vectors land in the same space and can be compared directly. The vector store's whole job is to make that comparison fast, even across millions of stored chunks — a challenge covered in more depth in Choosing a Vector Database.

Why the model can never change mid-pipeline

Each embedding model builds its own private map of meaning — the exact coordinates it assigns to a sentence depend entirely on how that specific model was trained. Swap in a different model for just one half of the pipeline, and you're no longer comparing points on the same map.

At best, this fails loudly: many embedding models output vectors of different lengths (say, 384 numbers versus 1536), and most vector stores require every vector to be the same size, so mismatched models can throw an error before a search ever runs. At worst, it fails silently — two models can coincidentally produce same-sized vectors, so the search runs without any error at all, but the results come back essentially random, because "closeness" only means something within a single model's own geometry.

This is also why upgrading to a better embedding model isn't as simple as switching which model handles new queries. Every chunk already sitting in the vector store was placed there using the old model's map, so the whole document set has to be re-embedded and re-indexed with the new model before search will work correctly again.

A word on approximate search

Exhaustively comparing a query vector against every single stored vector works fine for a few thousand chunks, but it doesn't scale to millions. Production vector stores instead use approximate nearest neighbor (ANN) indexes — algorithms like HNSW (Hierarchical Navigable Small World graphs) that organize vectors so a search only has to check a small, well-chosen fraction of them, trading a tiny amount of accuracy for a massive speedup. You don't need to implement one of these yourself — virtually every vector database ships with ANN indexing built in — but it's worth knowing the name, since it comes up constantly in vector database documentation.

Key takeaway

Loading a document means parsing it into structured elements with metadata, not just extracting raw text. Both documents and queries get embedded by the same model into the same vector store, which uses approximate nearest-neighbor indexing to keep search fast even at huge scale.

What's next?

We skipped over one crucial detail: how a document gets split into chunks in the first place. That decision has a bigger effect on search quality than almost anything else in the pipeline — the next few lessons dig into it.