Tables and lists contain dense relational data that naive chunking destroys. Split a table across two chunks and you lose the column headers in one half, the row context in the other. The embedding of the orphaned half is meaningless — and your RAG pipeline silently returns garbage.
Markdown preserves table and list structure faithfully. After converting with AnyMD, every column header, every list level, every table row is intact — which means you can chunk intelligently around these structural elements instead of through them.
This post shows how to chunk Markdown tables and lists for structured retrieval, with code examples in Python and JavaScript.
Why structured data needs different chunking
Most chunking algorithms treat documents as prose: paragraphs, sentences, headings. Tables and lists break that model because they contain relational data — rows that share column semantics, list items that share a parent context.
A typical table in AnyMD output looks like this:
| Protocol | Latency (ms) | Throughput (req/s) | Max Payload |
|----------|-------------|--------------------|-------------|
| HTTP/1.1 | 12.4 | 1,450 | 10 MB |
| HTTP/2 | 8.2 | 3,200 | 16 MB |
| HTTP/3 | 5.1 | 4,800 | 64 MB |A naive 512-token chunk that splits between HTTP/2 and HTTP/3 produces two chunks:
- Chunk A: "| Protocol | Latency (ms) | Throughput (req/s) | Max Payload | |----------|-------------|--------------------|-------------| | HTTP/1.1 | 12.4 | 1,450 | 10 MB | | HTTP/2 | 8.2 | 3,200 | 16 MB |"
- Chunk B: "| HTTP/3 | 5.1 | 4,800 | 64 MB |"
Chunk B has no column headers. An embedding model sees "HTTP/3", "5.1", "4,800", "64 MB" with zero semantic context. A query about "Which protocol has the lowest latency?" retrieves both chunks — but Chunk B alone is useless for answering.
The rule: never split a table
Tables should always be kept whole. A complete table of 10 rows and 4 columns is worth more than 10 partial tables of 1 row each. The overhead of storing one larger chunk (~3 KB for most tables) is negligible compared to the retrieval precision gain.
Python: table-preserving chunker
import re
from pathlib import Path
def chunk_markdown_keep_tables(markdown: str, max_chars: int = 2000) -> list[dict]:
"""
Split Markdown into chunks while keeping every GFM table intact.
Lists are kept whole when they fit, otherwise split at top-level items.
"""
chunks = []
current = {"source": "heading", "heading": "", "content": ""}
# Tokenize the document preserving table blocks
# GFM tables are lines starting with |
lines = markdown.split("\n")
i = 0
while i < len(lines):
line = lines[i]
# Detect table block
if line.lstrip().startswith("|") and "|" in line:
table_lines = []
while i < len(lines) and lines[i].lstrip().startswith("|"):
table_lines.append(lines[i])
i += 1
table_text = "\n".join(table_lines) + "\n"
# Flush current chunk if adding table would overflow
if current["content"] and len(current["content"]) + len(table_text) > max_chars:
chunks.append(current)
current = {"source": "table", "heading": current["heading"], "content": ""}
current["content"] += table_text
continue
# Detect heading
heading_match = re.match(r'^(#{1,6})\s+(.+)$', line)
if heading_match:
if current["content"].strip():
chunks.append(current)
current = {
"source": "heading",
"heading": line.strip("# ").strip(),
"content": line + "\n",
}
i += 1
continue
current["content"] += line + "\n"
i += 1
if current["content"].strip():
chunks.append(current)
return chunks
# Use with AnyMD output
md = Path("report.md").read_text()
chunks = chunk_markdown_keep_tables(md)
for c in chunks:
print(f"[{c['source']}] {c['heading'] or '—'} — {len(c['content'])} chars")
if '|' in c['content']:
print(f" (contains table)")
JavaScript: tokenizer-aware table chunker
/**
* Split markdown into chunks that preserve table and list boundaries.
* Uses tiktoken for token-aware sizing.
*/
import { getEncoding } from "js-tiktoken";
interface Chunk {
heading: string;
content: string;
tokens: number;
}
function chunkMarkdown(
markdown: string,
maxTokens: number = 512,
model: string = "text-embedding-3-small"
): Chunk[] {
const enc = getEncoding("cl100k_base");
const chunks: Chunk[] = [];
let current: Chunk = { heading: "", content: "", tokens: 0 };
let inTable = false;
for (const line of markdown.split("\n")) {
const trimmed = line.trim();
// Table start/continuation
if (trimmed.startsWith("|") && trimmed.endsWith("|")) {
if (!inTable) {
// Flush prose chunk before table
if (current.content.trim()) {
chunks.push(current);
}
current = { heading: current.heading, content: "", tokens: 0 };
inTable = true;
}
current.content += line + "\n";
current.tokens = enc.encode(current.content).length;
continue;
} else {
inTable = false;
}
// Heading boundary
if (/^#{1,6}\s/.test(trimmed)) {
if (current.content.trim()) {
chunks.push(current);
}
current = {
heading: trimmed.replace(/^#+\s*/, ""),
content: line + "\n",
tokens: enc.encode(line + "\n").length,
};
continue;
}
// Check token limit
const lineTokens = enc.encode(line + "\n").length;
if (current.tokens + lineTokens > maxTokens && current.content.trim()) {
chunks.push(current);
current = { heading: current.heading, content: "", tokens: 0 };
}
current.content += line + "\n";
current.tokens += lineTokens;
}
if (current.content.trim()) {
chunks.push(current);
}
return chunks;
}
const md = await Bun.file("report.md").text();
const chunks = chunkMarkdown(md);
console.log(`Created ${chunks.length} chunks`);
Chunking lists: ordered, unordered, nested
Lists present a different problem than tables. A deeply nested list with 50 items doesn't need to be kept whole — but the nesting level and parent context must survive the split.
This list from a converted DOCX:
1. **Server Setup**
- Choose a provider (AWS, GCP, or bare metal)
- Provision at least 4 vCPUs and 16 GB RAM
- Configure firewall rules:
- Ingress: ports 80, 443, 22
- Egress: all (default)
2. **Docker Installation**
- Install Docker CE 24+
- Enable containerd
- Add your user to the docker group
3. **Application Deployment**
- Clone the repository
- `docker compose up -d`
- Verify with `curl http://localhost:8080/health`Should never be split in the middle of an item's sub-items. The boundary rules are:
| Rule | Description | Priority |
|---|---|---|
| Keep item + sub-items whole | Never split between "2. Docker Installation" and its sub-points | Highest |
| Split at top-level items | If a list exceeds the chunk limit, split between top-level numbered/bullet items | High |
| Maintain list indent context | If splitting mid-list, include the parent heading in metadata | Medium |
| Never split inline code | Code spans and fenced blocks stay whole | Highest |
Recursive strategy: tables first, then lists, then prose
The most robust chunking pipeline for structured documents uses a priority-based recursive approach:
- Extract table blocks — each table becomes a chunk. Tables are never subdivided.
- Extract full fenced code blocks — each code block becomes a chunk for the same reason.
- Split remaining content by headings — each section is a candidate chunk.
- Within each section, try to keep lists whole — if a list exceeds the token limit, split at top-level items.
- Prose falls back to recursive character splitting — only after all structural boundaries have been exhausted.
Benchmark: structure-aware vs naive chunking
We benchmarked 500 documents (PDF, DOCX, PPTX, XLSX) converted to Markdown via AnyMD, comparing structure-aware chunking (tables kept whole, lists split at items) against naive token-based chunking at 512 tokens.
| Metric | Naive token chunking | Structure-aware chunking | Improvement |
|---|---|---|---|
| Precision@5 (table queries) | 0.52 | 0.94 | +81% |
| Precision@5 (list queries) | 0.68 | 0.91 | +34% |
| Precision@5 (all queries) | 0.76 | 0.89 | +17% |
| Avg chunks per document | 18 | 15 | -17% (fewer, better chunks) |
| Avg chunk size | 1,890 chars | 2,140 chars | +13% (denser chunks) |
The largest improvement is on table queries — +81% precision@5 — confirming that splitting tables is the single biggest mistake a chunking strategy can make.
Production pipeline with AnyMD
Here's a complete ingestion pipeline that converts office documents with AnyMD and applies structure-aware chunking:
import requests
import os
from pathlib import Path
API_KEY = os.environ["ANYMD_API_KEY"]
def convert_and_chunk(file_path: str) -> list[dict]:
"""Convert a document with AnyMD, then apply structure-aware chunking."""
# Step 1: Convert to Markdown
with open(file_path, "rb") as f:
resp = requests.post(
"https://api.anymd.net/api/convert",
headers={"Authorization": f"Bearer {API_KEY}"},
files={"file": f},
)
resp.raise_for_status()
markdown = resp.json()["markdown"]
# Step 2: Structure-aware chunking
chunks = chunk_markdown_keep_tables(markdown)
# Step 3: Add document metadata to each chunk
for c in chunks:
c["source_file"] = file_path
c["source_format"] = Path(file_path).suffix
return chunks
# Convert a whole directory
base = Path("invoices/")
for f in sorted(base.glob("*.*")):
if f.suffix.lower() in {".docx", ".pdf", ".xlsx"}:
chunks = convert_and_chunk(str(f))
print(f"{f.name}: {len(chunks)} chunks")
Integration with LangChain
LangChain's MarkdownHeaderTextSplitter preserves headings but doesn't know about tables. For production, write a custom splitter that extends the Markdown splitter with table-awareness:
from langchain_text_splitters import MarkdownHeaderTextSplitter
class TableAwareMarkdownSplitter(MarkdownHeaderTextSplitter):
"""Like MarkdownHeaderTextSplitter but keeps tables whole."""
def _split_text(self, text: str):
# Phase 1: extract table blocks wholesale
import re
table_pattern = re.compile(
r'(?m)^\|.+\|\s*\n\|[-:| ]+\|\s*\n(?:\|.+\|\s*\n)*'
)
tables = []
rest = text
while (m := table_pattern.search(rest)):
start, end = m.start(), m.end()
if rest[:start].strip():
# Split non-table content with parent class
for chunk in super()._split_text(rest[:start]):
yield chunk
yield {"content": m.group(), "metadata": {"type": "table"}}
rest = rest[end:]
if rest.strip():
for chunk in super()._split_text(rest):
yield chunkWhich AnyMD formats benefit most
Not all formats produce equally table-heavy Markdown. Based on our conversion data:
| Input format | Tables per doc (avg) | Chunking strategy |
|---|---|---|
| DOCX | 3.2 | Table-preserving |
| PDF (text-based) | 0.8 | Standard heading-based |
| XLSX (spreadsheets) | 12.4 | Must use table-preserving |
| PPTX (slides) | 1.1 | Standard (small tables per slide) |
| EPUB | 0.3 | Standard heading-based |
| ODT | 2.8 | Table-preserving |
If your pipeline processes XLSX or DOCX files, structure-aware chunking is non-negotiable. For PDF-heavy pipelines, standard heading-based chunking (covered in our semantic chunking guide) is usually sufficient.
Pricing
AnyMD converts documents to clean Markdown under standard page-based pricing. The chunking happens on your side — you only pay for the conversion:
| Plan | Pages/month | Max file size | Price |
|---|---|---|---|
| Free | 100 | 10 MB | $0 |
| Starter | 500 | 25 MB | $19 |
| Pro | 5,000 | 50 MB | $99 |
| Enterprise | Custom | Custom | Custom |
Get your free API key — 100 pages/month, no credit card required.