← Course Home

Chunking Strategies

Agentic Chunking

4 min read

Letting an LLM decide what belongs together

Fixed-size chunking splits by count; semantic chunking splits by embedding similarity. Agentic chunking takes the idea a step further and asks a large language model to read the text and decide, directly, what constitutes one coherent, self-contained idea.

Diagram — How Agentic Chunking Uses an LLM to Decide Chunk Boundaries

Documentsplit intoparagraphsLLMextractpropositionslist_of_propositions"Statement 1...""Statement 2...""Statement 3..."LLMgroup +de-duplicateRefined propositionsEmbedding modelone vector per propositionVectordatabaseInstead of cutting text by length or sentence boundary, an LLM reads the meaning and decides whatcounts as one self-contained fact — at the cost of an extra model call per document.

The typical flow:

  1. Split into paragraphs — a coarse, structural first pass.
  2. Generate propositions — for each paragraph, an LLM extracts a list of propositions: small, standalone factual statements that make sense entirely on their own, without needing the surrounding paragraph for context. A sentence like "It was released the same year" gets rewritten as something like "The film was released in 2014" — resolving pronouns and implicit references so the statement stands alone.
  3. Collect and refine — all the propositions from every paragraph are gathered into one list. A second LLM pass reviews that full list to group closely related propositions together and drop ones that are redundant.
  4. Embed and store — the refined propositions become the final chunks, embedded and written to the vector database exactly like any other chunk.

Watching it happen on a real paragraph

Take this paragraph from an article about the film Interstellar: "Interstellar is a science-fiction film directed by Christopher Nolan. It was released the same year as the World Cup was held in Brazil. The film starred Matthew McConaughey and Anne Hathaway, and it went on to gross over $700 million worldwide."

After splitting into paragraphs (step 1), that whole passage becomes one unit heading into the next step. Generating propositions (step 2) turns it into a list of standalone facts, with every pronoun and vague reference resolved so each one reads clearly on its own:

  • "Interstellar is a science-fiction film."
  • "Interstellar was directed by Christopher Nolan."
  • "Interstellar was released in 2014."
  • "Interstellar starred Matthew McConaughey."
  • "Interstellar starred Anne Hathaway."
  • "Interstellar grossed over $700 million worldwide."

Notice "the same year as the World Cup was held in Brazil" got resolved into an actual year — a proposition can't lean on outside trivia to be understood.

At collect and refine (step 3), these six propositions join every other proposition extracted from the rest of the document. If a later section of the same document repeats "Interstellar grossed over $700 million worldwide" — say, in a dedicated box-office section — that duplicate gets dropped, and the two casting facts might be merged into "Interstellar starred Matthew McConaughey and Anne Hathaway." Finally, at embed and store (step 4), each surviving proposition is embedded and written to the vector store as its own independent chunk.

The payoff shows up at query time: someone asking "What year did Interstellar come out?" gets back the chunk "Interstellar was released in 2014" directly — precise and immediately readable, instead of a paragraph fragment still leaning on a stripped-out "it."

Why go to this much trouble

A proposition-based chunk is about as self-contained as a chunk can get — each one is a complete, standalone statement of fact, deliberately rewritten so it doesn't depend on anything outside itself for meaning. That tends to make retrieval more precise: when a chunk is retrieved on its own, out of its original context, it still reads clearly.

The tradeoff

This precision comes at a real cost. Agentic chunking calls an LLM at least twice per document — once to extract propositions, once to refine them — which is far more expensive and slower than counting characters or even embedding sentences. It's typically reserved for high-value document sets where retrieval precision matters enough to justify the extra compute, not applied indiscriminately across everything you index.

Key takeaway

Agentic chunking uses an LLM to read a document and rewrite it into standalone, self-contained factual statements, then refines and deduplicates that list before embedding it — trading significant extra cost for chunks that are unusually precise and context-independent.

What's next?

Chunking decides what gets stored. The next section turns to what happens at query time: how a stored chunk's vector actually gets compared against a query vector to decide "how similar" the two really are.