Not all chunking strategies are equal. Pick the wrong one and your RAG pipeline silently returns garbage — embeddings that don't match, context windows that overflow, chunks that split mid-table. The choice between fixed-size and variable-size chunking is the single biggest lever you have for retrieval quality.
We benchmarked 10 chunking strategies across 500 real-world documents (PDFs, DOCX files, PPTX slide decks, EPUB books) to answer one question: which strategy maximizes retrieval precision for RAG?
This post presents the results, with production-ready code in Python and Node.js.
What we tested
Every document was first converted to GitHub-Flavored Markdown via the AnyMD API. Then we applied 10 chunking strategies:
| # | Strategy | Type | Chunk size | Overlap |
|---|---|---|---|---|
| 1 | Fixed 256 chars | Fixed | 256 chars | 0 |
| 2 | Fixed 512 chars | Fixed | 512 chars | 0 |
| 3 | Fixed 256 tokens | Fixed | 256 tokens | 0 |
| 4 | Fixed 512 tokens | Fixed | 512 tokens | 0 |
| 5 | Fixed 1,024 tokens | Fixed | 1,024 tokens | 0 |
| 6 | Heading-based (variable) | Variable | Per-section | 0 |
| 7 | Recursive character (1,000) | Variable | ~1,000 chars | 200 chars |
| 8 | Recursive character (2,000) | Variable | ~2,000 chars | 200 chars |
| 9 | Semantic (heading + paragraph) | Variable | Per-section | Heading context |
| 10 | Hybrid (heading + token limit) | Variable | Max 512 tok | Heading context |
Overall results
| Rank | Strategy | Precision@5 | Recall@5 | F1 | Avg chunks/doc |
|---|---|---|---|---|---|
| 🥇 | Hybrid (heading + token limit) | 0.912 | 0.894 | 0.903 | 14 |
| 🥈 | Heading-based (variable) | 0.887 | 0.862 | 0.874 | 11 |
| 🥉 | Semantic (heading + paragraph) | 0.875 | 0.858 | 0.866 | 13 |
| 4 | Recursive character 2,000 | 0.841 | 0.823 | 0.832 | 9 |
| 5 | Recursive character 1,000 | 0.819 | 0.795 | 0.807 | 17 |
| 6 | Fixed 1,024 tokens | 0.782 | 0.761 | 0.771 | 16 |
| 7 | Fixed 512 tokens | 0.735 | 0.708 | 0.721 | 31 |
| 8 | Fixed 256 tokens | 0.664 | 0.632 | 0.648 | 61 |
| 9 | Fixed 512 chars | 0.618 | 0.589 | 0.603 | 35 |
| 10 | Fixed 256 chars | 0.541 | 0.507 | 0.523 | 71 |
Hybrid chunking — using heading boundaries as semantic breakpoints capped by a token limit — outperforms every fixed-size strategy by at least 13% F1. The worst fixed strategy (256 chars) scores 0.523 F1, meaning nearly half of retrievals are useless for downstream generation.
Why variable-size chunking wins
Fixed-size chunking has a fundamental problem: it ignores document structure. A chunk boundary that lands in the middle of a paragraph, table, or list produces an embedding with no coherent semantic center.
Here's a 512-token fixed chunk of a real document converted via AnyMD:
consumer. Under these conditions, the system must
handle at least 10,000 requests per second with a P99
latency under 200ms."
### 3.2 Security Requirements
All API endpoints must be authenticated via OAuth 2.0.
TLS 1.3 is mandatory in transit; AES-256-GCM at rest.
### 3.3 Storage Requirements
The database must support horizontal sharding across at
least 3 availability zones. Backup frequency: hourly
incremental, daily full. Recovery time objective (RTO):
15 minutes. Recovery point objective (RPO): 1 minute.
### 4. Architecture Overview
Figure 1 shows the high-level system architecture.
[Diagram: load balancer → API gateway → service mesh →
microservices → database cluster]
### 4.1 Frontend Layer
The frontend is built with React 19 and communicates
with the API gateway via WebSocket for real-time
updates and REST for CRUD operations. Static assets
are served through a CDN with a 90% cache hit target.
### 4.2 API Gateway Layer
The gateway is implemented as a Rust service using
Axum. It handles rate limiting (1,000 req/min per
client), authentication (JWT validation), and request
routing to the appropriate microservice.
| Service | Language | Replicas | Resource Limit |
|-------------|----------|----------|----------------------|
| Auth | Rust | 3 | 2 vCPU, 4 GB RAM |That's four ### subsections and the start of a table in a single chunk. An embedding model trying to represent that will produce a blur — part security, part storage, part frontend, part architecture. A query about "OAuth 2.0 authentication" competes with "horizontal sharding" and "React 19".
Variable-size chunking avoids this by respecting document structure. Each heading becomes a natural semantic boundary:
Chunk 1: ### 3.1 Performance Requirements → 298 tokens
Chunk 2: ### 3.2 Security Requirements → 187 tokens
Chunk 3: ### 3.3 Storage Requirements → 256 tokens
Chunk 4: ### 4. Architecture Overview → 124 tokens
Chunk 5: ### 4.1 Frontend Layer → 312 tokens
Chunk 6: ### 4.2 API Gateway Layer → 408 tokens (includes table, kept whole)Each chunk has a single semantic focus. The embedding for "Security Requirements" cleanly represents authentication, TLS, and encryption — and will match a query about "OAuth 2.0" with high precision.
When fixed-size chunking still makes sense
Fixed-size isn't useless. Three scenarios where it's acceptable:
- Uniform content — technical manuals or log files with no headings, where all text is roughly the same density. Fixed 512 tokens works fine.
- Very short documents — a single-paragraph memo or a one-slide deck. Just embed the whole thing.
- Real-time streaming — when you need chunk boundaries on a clock (e.g., 5-second audio transcription segments), fixed windows are simpler to implement.
For everything else — reports, whitepapers, invoices, contracts, academic papers, slide decks, spreadsheets — variable-size chunking on clean Markdown is strictly superior.
Python: hybrid chunker (heading + token limit)
import re
import tiktoken
from pathlib import Path
def hybrid_chunk(
markdown: str,
model: str = "text-embedding-3-small",
max_tokens: int = 512,
min_tokens: int = 50,
) -> list[dict]:
"""
Split Markdown into chunks at heading boundaries, capped by a
hard token limit. Sections smaller than min_tokens are merged
into the previous chunk.
"""
enc = tiktoken.get_encoding(tiktoken.encoding_name_for_model(model))
lines = markdown.split("\n")
chunks = []
current = {"heading": "", "content": "", "tokens": 0}
i = 0
while i < len(lines):
line = lines[i]
heading_match = re.match(r"^(#{1,6})\s+(.+)$", line)
if heading_match:
# Start a new section
if current["content"].strip():
if current["tokens"] >= min_tokens or not heading_match:
chunks.append(current)
else:
# Merge small section into previous
if chunks:
chunks[-1]["content"] += "\n" + current["content"]
chunks[-1]["tokens"] = len(enc.encode(chunks[-1]["content"]))
current = {
"heading": line.strip("# ").strip(),
"content": line + "\n",
"tokens": len(enc.encode(line + "\n")),
}
i += 1
continue
# Check token limit
line_tokens = len(enc.encode(line + "\n"))
if current["tokens"] + line_tokens > max_tokens and current["content"].strip():
# Flush current chunk, keep heading context
chunks.append(current)
current = {
"heading": current["heading"],
"content": line + "\n",
"tokens": line_tokens,
}
i += 1
continue
current["content"] += line + "\n"
current["tokens"] += line_tokens
i += 1
if current["content"].strip():
chunks.append(current)
# Add metadata
for idx, c in enumerate(chunks):
c["chunk_id"] = idx
c["source"] = "heading"
return chunks
# Usage with AnyMD output
md = Path("converted_report.md").read_text()
chunks = hybrid_chunk(md)
print(f"Created {len(chunks)} chunks, "
f"avg {sum(c['tokens'] for c in chunks) // len(chunks)} tokens")
for c in chunks[:3]:
print(f" [{c['heading']}] {c['tokens']} tok")
Node.js: fixed-size vs variable-size comparison
import { getEncoding } from "js-tiktoken";
interface Chunk {
heading: string;
content: string;
tokens: number;
}
/** Fixed-size chunking: every chunk is exactly `maxTokens`. */
function fixedChunk(text: string, maxTokens: number = 512): string[] {
const enc = getEncoding("cl100k_base");
const tokens = enc.encode(text);
const chunks: string[] = [];
for (let i = 0; i < tokens.length; i += maxTokens) {
const slice = tokens.slice(i, i + maxTokens);
chunks.push(new TextDecoder().decode(enc.decode(slice)));
}
return chunks;
}
/** Variable-size (heading-aware) chunking. */
function variableChunk(markdown: string, maxTokens: number = 512): Chunk[] {
const enc = getEncoding("cl100k_base");
const chunks: Chunk[] = [];
let current: Chunk = { heading: "", content: "", tokens: 0 };
for (const line of markdown.split("\n")) {
const headingMatch = line.match(/^(#{1,6})\s+(.+)$/);
const lineTokens = enc.encode(line + "\n").length;
if (headingMatch) {
if (current.content.trim()) chunks.push(current);
current = {
heading: line.replace(/^#+\s*/, ""),
content: line + "\n",
tokens: lineTokens,
};
continue;
}
if (current.tokens + lineTokens > maxTokens && current.content.trim()) {
chunks.push(current);
current = { heading: current.heading, content: "", tokens: 0 };
}
current.content += line + "\n";
current.tokens += lineTokens;
}
if (current.content.trim()) chunks.push(current);
return chunks;
}
// Benchmark on a real document
const md = await Bun.file("report.md").text();
const fixed = fixedChunk(md, 512);
const variable = variableChunk(md, 512);
console.log(`Fixed-size: ${fixed.length} chunks`);
console.log(`Variable-size: ${variable.length} chunks`);
Benchmark methodology
We tested on a corpus of 500 documents:
- 100 PDFs — academic papers, financial reports, legal contracts
- 100 DOCX files — Word documents, business proposals, manuals
- 100 PPTX files — slide decks with mixed text/table/bullet content
- 100 EPUB files — books and long-form publications
- 100 mixed — CSVs, RTFs, ODTs, XLSX exports
Each document was converted using the AnyMD API. Then each chunking strategy was evaluated on:
- Precision@5 — of the top-5 retrieved chunks, what fraction are relevant to the query?
- Recall@5 — of all relevant chunks, what fraction appear in the top-5?
- Embedding quality — cosine similarity between chunk embeddings and query embeddings for 200 hand-curated query-document pairs.
Embeddings were generated with text-embedding-3-small (OpenAI), and retrieval was done via cosine similarity. The same queries and relevance judgments were used across all strategies.
The AnyMD advantage
All of these strategies depend on one critical input: clean Markdown. When you chunk raw PDF text (extracted by PyMuPDF or pdfminer), you get:
- Broken tables — columns merged, headers missing
- Missing list hierarchy — indentation lost, nesting flat
- Orphaned headings — no visual distinction between
#and### - Garbled code blocks — indentation mangled, syntax lost
AnyMD converts 15+ document formats to GitHub-Flavored Markdown with full structural fidelity: headings, tables, lists, code blocks, and inline formatting are all preserved. The chunking strategies above work because the input is clean.
Recommendations by use case
| Use case | Recommended strategy | Why |
|---|---|---|
| Document Q&A (reports, papers) | Hybrid heading + token | Best F1; sections stay coherent |
| Chatbot with structured data | Heading-based variable | Preserves section context |
| Code documentation RAG | Hybrid + code block preservation | Code blocks must stay whole |
| Semantic search (any doc) | Recursive character (2,000) | Good balance of size and precision |
| Real-time / streaming | Fixed 512 tokens | Predictable chunk boundaries |
| High-throughput batch | Heading-based variable | Fewer chunks = fewer embeddings |
Get started
The best chunking strategy starts with the right conversion. Convert your documents to clean Markdown with AnyMD, then apply the hybrid chunker above.
Sign up for a free API key — 100 pages per month, no credit card required — and benchmark our output against your current pipeline: anymd.net/pricing →