Fixed-Size, Token, and Overlapping Chunks

Splitting by count

The most widely used chunking strategy is also the simplest: pick a fixed size and cut the text every time you hit it. Two common units:

• Character splitting — cut every N characters. Fast and predictable, but it doesn't know where words begin or end, so it can (and regularly does) slice a word in half.
• Token splitting — cut every N tokens instead. Tokens roughly correspond to word pieces, so this respects word boundaries better than raw character counting — but it can still cut a sentence off mid-thought at the boundary.

The boundary problem

Whichever unit you split by, a hard cutoff has a real cost: whatever meaning depended on words before the cutoff connecting to words after it gets severed. A pronoun on one side of the boundary might lose the noun it refers to on the other side. A sentence explaining a term might get separated from the term itself.

The fix: overlap

The common fix is overlap — instead of starting the next chunk exactly where the last one ended, start it slightly earlier, so a small amount of text (a sentence, a handful of tokens) appears in both chunks. Any meaning that would otherwise be stranded right at the boundary now has a chance to appear fully within at least one chunk.

Overlap isn't free — it means storing (and later retrieving) somewhat more text overall, and duplicate content across chunks. In practice, teams typically overlap somewhere between 10% and 20% of a chunk's length, tuning up or down based on how densely connected their source text tends to be.

How big should a chunk be?

There's no universal right answer — it depends on what you're indexing and what model will eventually read the retrieved chunks. A short chunk (a sentence or two) is very precise but may lack surrounding context; a long chunk carries more context but risks the same "blurry average" problem from the previous lesson, just at a smaller scale. It's also worth remembering that the language model generating your final answer has a maximum context window — retrieving five oversized chunks can quietly blow past that limit before generation even starts.

Key takeaway

Fixed-size and token-based splitting are simple and predictable, but a hard cutoff can sever meaning right at the chunk boundary. Overlapping a small amount of text between adjacent chunks is the standard fix, at the cost of some duplicated content.

What's next?

Fixed-size splitting doesn't know or care what the text is about. The next lesson covers a chunking method that does — using the embeddings themselves to find where a document's topic actually shifts.