← Back to Technical Section

Chunking Strategies

Text Splitting Techniques for RAG and Embedding Systems

🔧 Try the RAG Visualizer

Visualize chunking, embeddings, and retrieval interactively with your own documents.

Open RAG Visualizer →

📋 14 Advanced Chunking Strategies

Explore detailed strategies like Semantic Chunking, Parent-Child, Metadata-First, and more.

View 14 Strategies →

What is Chunking?

Chunking is the process of splitting documents into smaller, semantically meaningful segments for embedding and retrieval. It's a critical step in RAG pipelines that directly impacts retrieval quality, LLM context efficiency, and overall system performance.

Poor chunking leads to poor retrieval: chunks that are too large dilute relevance, while chunks that are too small lose context. Finding the right balance is both an art and a science.

Why Chunking Matters

Retrieval Precision

Smaller, focused chunks match queries more precisely. A chunk about "Python decorators" will rank higher for that query than a whole chapter.

Token Efficiency

Smaller chunks use fewer tokens in the LLM context, reducing cost and leaving room for more relevant information.

Context Preservation

Chunks must be large enough to preserve meaning. "It increased by 50%" is useless without knowing what "it" refers to.

Embedding Quality

Embedding models have context limits (512–8192 tokens). Chunks exceeding this get truncated, losing information.

Chunking Strategies

1. Fixed-Size Chunking

Split text into chunks of fixed character or token count. Simple but often breaks mid-sentence or mid-paragraph.

from langchain.text_splitter import CharacterTextSplitter

splitter = CharacterTextSplitter(
    chunk_size=1000,        # Characters per chunk
    chunk_overlap=200,      # Overlap between chunks
    separator="\n"          # Try to split on newlines
)

chunks = splitter.split_text(document)

✓ Simple to implement
✓ Predictable chunk sizes
✓ Works for any text

✗ Ignores semantic boundaries
✗ May split mid-sentence
✗ Poor for structured docs

2. Recursive Character Splitting

Tries to split on natural boundaries (paragraphs, then sentences, then words) while respecting chunk size limits. The most commonly used strategy.

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200,
    separators=["\n\n", "\n", ". ", " ", ""]  # Priority order
)

chunks = splitter.split_text(document)

💡 Recommended Default: RecursiveCharacterTextSplitter with chunk_size=500-1000 and overlap=50-200 is a solid starting point for most use cases.

3. Semantic Chunking

Uses embeddings to find natural semantic breakpoints. Splits where the meaning changes significantly, creating more coherent chunks.

from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings()
splitter = SemanticChunker(
    embeddings,
    breakpoint_threshold_type="percentile",
    breakpoint_threshold_amount=95
)

chunks = splitter.split_text(document)

✓ Preserves semantic coherence
✓ Variable but meaningful sizes
✓ Better retrieval quality

✗ Requires embedding API calls
✗ Higher latency and cost
✗ Unpredictable chunk sizes

4. Document Structure-Based

Respects document structure: headers, sections, paragraphs. Best for well-formatted documents like Markdown, HTML, or PDFs with clear headings.

from langchain.text_splitter import MarkdownHeaderTextSplitter

headers_to_split_on = [
    ("#", "Header 1"),
    ("##", "Header 2"),
    ("###", "Header 3"),
]

splitter = MarkdownHeaderTextSplitter(headers_to_split_on)
chunks = splitter.split_text(markdown_document)
# Each chunk includes header metadata for context

5. Sentence-Based Chunking

Groups complete sentences up to a token limit. Uses NLP sentence detection (spaCy, NLTK) for accurate boundaries.

import spacy

nlp = spacy.load("en_core_web_sm")

def sentence_chunk(text, max_sentences=5):
    doc = nlp(text)
    sentences = list(doc.sents)
    chunks = []
    
    for i in range(0, len(sentences), max_sentences):
        chunk = " ".join(str(s) for s in sentences[i:i+max_sentences])
        chunks.append(chunk)
    
    return chunks

6. Late Chunking (Contextual)

Embed the full document first, then chunk the embeddings while preserving context from the whole document. Emerging technique for better coherence.

# Conceptual approach (implementations vary)
# 1. Process full document through embedding model
# 2. Get token-level or span-level embeddings
# 3. Pool embeddings into chunks AFTER contextual encoding
# 4. Each chunk embedding "knows" about the whole document

# Libraries: Jina AI late-chunking, custom implementations

Chunk Size Guidelines

Use Case	Chunk Size	Overlap	Rationale
Q&A / Factoid retrieval	200–500 tokens	50–100	Precise matching, focused answers
Summarization	500–1000 tokens	100–200	Enough context per chunk
Code documentation	Function/class level	0 (structure-based)	Respect code boundaries
Legal / Contracts	Paragraph/clause	0–50	Preserve legal meaning
Chat / Conversational	Message or turn	Previous turn	Maintain dialogue context

⚠️ Match Your Embedding Model: If your embedding model has a 512-token limit, chunks exceeding this will be truncated. Always check model.max_seq_length.

Understanding Chunk Overlap

Overlap creates redundancy between consecutive chunks, ensuring that information at chunk boundaries isn't lost during retrieval.

Too little overlap (0–10%): Risk losing context at boundaries
Moderate overlap (10–20%): Good balance for most use cases
High overlap (20–50%): Increases storage/cost, but may improve recall

Enriching Chunks with Metadata

Attach metadata to chunks for filtering, citation, and context. This enables hybrid search (semantic + filters) and better answer attribution.

# Chunk with metadata
chunk = {
    "id": "doc123_chunk_5",
    "content": "The revenue increased by 15% in Q3 2024...",
    "metadata": {
        "source": "annual_report_2024.pdf",
        "page": 42,
        "section": "Financial Results",
        "date": "2024-10-15",
        "chunk_index": 5,
        "total_chunks": 128,
        "parent_id": "doc123",      # For parent-child retrieval
        "prev_chunk": "doc123_chunk_4",
        "next_chunk": "doc123_chunk_6"
    },
    "embedding": [0.023, -0.045, ...]
}

Useful Metadata Fields

source: Original document name/path for citations
page/section: Location for navigation
date: For temporal filtering (show only recent docs)
author/owner: For access control and attribution
chunk_index: For ordering and context expansion
parent_id: Link to parent document for hierarchical retrieval

Advanced Techniques

Parent-Child Chunking

Index small chunks for precise retrieval, but retrieve the parent (larger) chunk for context. Best of both worlds: precision + context.

Hypothetical Document Embeddings (HyDE)

Generate a hypothetical answer to the query, embed that, and use it to find similar real chunks. Can improve retrieval for complex queries.

Contextual Retrieval (Anthropic)

Prepend each chunk with LLM-generated context explaining what the chunk is about. Improves embedding quality for ambiguous content.

Multi-Vector Retrieval

Generate multiple embeddings per chunk (e.g., from different perspectives or with generated questions). Increases recall for diverse queries.

Evaluating Chunking Quality

Retrieval Recall@k: What % of relevant chunks are retrieved in top k?
Chunk Coherence: Do chunks make sense in isolation? (human evaluation)
Answer Quality: End-to-end RAG evaluation—does better chunking = better answers?
Token Efficiency: How many tokens needed per successful retrieval?

💡 Pro Tip: Build a test set of questions with known answer locations. Compare chunking strategies by measuring retrieval success rate.

Test Your Knowledge

Score 8/10 or higher to pass

Chunking Strategies

🔧 Try the RAG Visualizer

📋 14 Advanced Chunking Strategies

What is Chunking?

Why Chunking Matters

Retrieval Precision

Token Efficiency

Context Preservation

Embedding Quality

Chunking Strategies

1. Fixed-Size Chunking

2. Recursive Character Splitting

3. Semantic Chunking

4. Document Structure-Based

5. Sentence-Based Chunking

6. Late Chunking (Contextual)

Chunk Size Guidelines

Understanding Chunk Overlap

Enriching Chunks with Metadata

Useful Metadata Fields

Advanced Techniques

Parent-Child Chunking

Hypothetical Document Embeddings (HyDE)

Contextual Retrieval (Anthropic)

Multi-Vector Retrieval

Evaluating Chunking Quality

Related Topics

Test Your Knowledge