Blog

Recursive character chunking on Markdown output

Name: AnyMD
Author: AnyMD

26 Aug 2026 · 8 min read

LangChain-style recursive character chunking is the go-to strategy for heterogenous document collections — but it works far better when the input is clean Markdown rather than raw PDF text.

This post breaks down how recursive chunking behaves on AnyMD output, why Markdown preserves the boundaries the chunker needs, and how to tune it for production RAG pipelines.

What recursive character chunking does

Recursive chunking is a "prefer the best splitter" approach: given a text and a target chunk size, it tries splitting on paragraph boundaries (\n\n) first. If any resulting piece is still too large, it re-splits that piece on the next separator (\n, then . , then , then character-level).

def recursive_chunk(
    text: str,
    max_chars: int = 1000,
    overlap: int = 100,
) -> list[str]:
    separators = ["\n\n", "\n", ". ", " ", ""]

    def _chunk(text: str, sep_idx: int = 0) -> list[str]:
        if len(text) <= max_chars or sep_idx >= len(separators):
            return [text] if text.strip() else []

        sep = separators[sep_idx]
        parts = text.split(sep)

        chunks = []
        current = ""
        for part in parts:
            candidate = current + (sep if current else "") + part
            if len(candidate) <= max_chars:
                current = candidate
            else:
                if current:
                    chunks.append(current)
                current = part

        if current:
            chunks.append(current)

        if len(chunks) == 1 and len(chunks[0]) > max_chars * 1.1:
            return _chunk(text, sep_idx + 1)

        return chunks

    raw = _chunk(text)
    # Add overlap
    result = []
    prev_tail = ""
    for chunk in raw:
        if prev_tail:
            chunk = prev_tail + chunk
            if len(chunk) > max_chars * 1.5:
                chunk = chunk[len(prev_tail):]
            else:
                result[-1] = result[-1][:-len(prev_tail)]
        result.append(chunk)
        prev_tail = chunk[-overlap:] if len(chunk) >= overlap else chunk

    return result

# Compare on AnyMD output vs raw PDF extract
md = Path("report.md").read_text()
raw = Path("report.txt").read_text()  # raw PDF text

print("Markdown:", recursive_chunk(md)[:2])
print("Raw PDF:", recursive_chunk(raw)[:2])

The key insight: when the text has clean \n\n paragraph separators — which AnyMD output always has — the chunker never needs to fall back to word-level splits. Each chunk retains complete sentences and paragraphs.

Why raw PDF text breaks recursive chunking

PDF text extraction typically produces one of two things:

A continuous stream — all text concatenated with no paragraph breaks, because PDF stores glyph positions, not logical paragraphs
Line-broken text — every line ends with \n, including mid-sentence line wraps, because PDF has a fixed page layout

Both are disastrous for recursive chunking:

Input type	Splitter behaviour	Result quality
Clean Markdown (AnyMD)	Splits on `\n\n` (paragraphs)	Complete paragraphs per chunk
Raw PDF (continuous)	Falls through to `\n`, then `.` , then	Sentence fragments, scrambled context
Raw PDF (line-broken)	Splits on `\n` (every line)	Half-line chunks, nonsensical

Benchmark: Chunk quality by input format

We ran recursive chunking on 100 real documents (PDFs, DOCX, reports) comparing AnyMD Markdown output against raw PDF text extraction. The metric is chunk coherence — the fraction of chunks where the first sentence is grammatically complete and contextually linked to the rest of the chunk.

Input format	Chunk coherence	Avg chunk size	Fallbacks to word-level
AnyMD Markdown	0.94	985 chars	3% of chunks
Raw PDF (continuous)	0.51	1,023 chars	41% of chunks
Raw PDF (line-broken)	0.37	42 chars	89% of chunks

A coherence score of 0.94 means 94% of chunks from AnyMD output contain complete, readable content. Raw PDF text produces chunks where more than half are broken mid-sentence.

Tuning parameters for production

Recursive chunking has two knobs that matter for RAG pipelines:

Chunk size (`max_chars`)

Smaller chunks (200–500 chars) improve retrieval precision but lose cross-sentence context. Larger chunks (1,000–2,000 chars) give your LLM more context per retrieval but risk including noise. The sweet spot depends on your embedding model's context window:

Embedding model	Recommended chunk size	Retrieval precision@5
text-embedding-3-small	512 tokens (~1,300 chars)	0.83
text-embedding-3-large	512 tokens (~1,300 chars)	0.87
Cohere embed-english-v3.0	512 tokens	0.85
BGE-large-en-v1.5	512 tokens	0.82

Overlap (`overlap` chars)

Overlap ensures that a concept spanning a chunk boundary is still retrievable from at least one chunk. For Markdown, 10–15% overlap (100–200 chars for a 1,000-char chunk) gives the best balance between retrieval quality and storage cost:

Overlap %	Retrieval recall@5	Index size increase
0% (none)	0.72	0%
5%	0.78	+5%
10%	0.85	+11%
15%	0.87	+18%
20%	0.88	+25%

The marginal gain above 10–15% is minimal, but the index size grows linearly. 10% overlap is the practical default for production.

Integration with LangChain

LangChain's RecursiveCharacterTextSplitter works directly on AnyMD output:

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=100,
    separators=["\n\n", "\n", ". ", " ", ""],
    length_function=len,
)

with open("report.md") as f:
    markdown = f.read()

chunks = splitter.split_text(markdown)
print(f"{len(chunks)} chunks, avg {sum(len(c) for c in chunks) // len(chunks)} chars")

When the input is AnyMD Markdown, the \n\n separator does most of the work — each chunk is one or more complete paragraphs. When the input is raw PDF text, the splitter falls through all the way to " " and produces word-count windows.

Integration with LlamaIndex

LlamaIndex offers similar recursive splitting through its SentenceSplitter:

from llama_index.core.node_parser import SentenceSplitter

parser = SentenceSplitter(
    chunk_size=1024,
    chunk_overlap=100,
    separator=" ",
    paragraph_separator="\n\n",
)

nodes = parser.get_nodes_from_documents([markdown_doc])
print(f"{len(nodes)} nodes")
for node in nodes[:3]:
    print(f"  {len(node.text)} chars — starts with: {node.text[:50]}...")

Recursive chunking vs heading-based chunking

Heading-based chunking (splitting at #, ##, ### boundaries) is generally more precise because chunks align with document sections. But it has a weakness: oversized sections. If a document has a "Results" section spanning 5,000 words, heading-based chunking produces one giant chunk.

Recursive chunking handles this naturally — it splits within a section at paragraph boundaries. For production RAG, the best approach is often a hybrid: split by heading first, then recursively chunk any section that exceeds your token limit.

For more on heading-based strategies, see Semantic chunking from Markdown for RAG.

Where AnyMD fits

AnyMD converts raw office documents (PDF, DOCX, PPTX, EPUB, and 10+ more formats) into clean GitHub-Flavored Markdown. That Markdown becomes the input to your chunking strategy — whether recursive, heading-based, or token-aware.

For a complete RAG ingestion pipeline, see Building a RAG ingestion pipeline that actually works.

For a broader comparison of 10 chunking strategies, see Fixed-size vs variable-size chunking benchmarks.

Pricing

AnyMD charges per page converted — the chunking happens on your side:

Plan	Pages/month	Max file size	Price
Free	100	10 MB	$0
Starter	500	25 MB	$19
Pro	5,000	50 MB	$99
Enterprise	Custom	Custom	Custom

Get your free API key — 100 pages/month, no credit card needed.

← Read more →