Building a RAG ingestion pipeline that actually works

RAG (Retrieval-Augmented Generation) pipelines are only as good as the data they retrieve. And the data they retrieve is only as good as the document parsing step that feeds the chunker and embedder.

The most common mistake we see: feeding raw PDF text into a vector database and wondering why retrieval quality is poor. PDFs don't contain structured text — they contain positioned glyphs. A "paragraph" in a PDF is a loose collection of text blocks with no semantic grouping. Tables aren't tables; they're coordinates. Headings aren't headings; they're bold text at a larger font size.

The pipeline

1. Document → Markdown — Convert PDF, DOCX, PPTX, or any other format to clean, structured Markdown using AnyMD's API.
2. Markdown → Chunks — Split by headings and paragraphs (semantic chunking, not fixed-length token splitting).
3. Chunks → Embeddings — Pass each chunk through your embedding model.
4. Embeddings → Vector store — Index for similarity search.

Step 1 is where most RAG pipelines fail silently. Garbage in, garbage out.

Why Markdown is the best intermediate format

Markdown preserves document structure — headings, lists, tables, code blocks — in a lightweight format that chunkers and embedding models understand natively. Compare these two representations of the same data:

Raw PDF text:

Revenue Q1 Q2 Q3
EU €1.2M €1.4M €1.1M
US €0.9M €1.1M €1.3M

After AnyMD conversion:

| Region | Q1    | Q2    | Q3    |
| :----- | :---- | :---- | :---- |
| EU     | €1.2M | €1.4M | €1.1M |
| US     | €0.9M | €1.1M | €1.3M |

The Markdown version is semantically meaningful. A semantic chunker can split at heading boundaries. An embedding model produces better vectors for a table than for a blob of whitespace-padded text. And when a user asks "what was EU revenue in Q2?", the retrieved chunk is the actual table with column headers, not orphaned numbers.

Practical example

# Convert a quarterly report to Markdown
curl -X POST https://anymd.net/api/convert \
  -H "Authorization: Bearer your-key" \
  -F "file=@quarterly-report.pdf" \
  -o report.md

# Now feed that Markdown into your chunker/embedder pipeline

The Markdown output preserves every heading, table, list, and paragraph — ready for ingestion into any RAG framework (LangChain, LlamaIndex, or your own).

Performance matters at scale

AnyMD converts a 48-page report in under a second. If your pipeline processes 10,000 documents per day, that's the difference between 2.2 hours of conversion time (AnyMD) vs 16–39 hours (Python-based alternatives). At RAG pipeline scale, conversion latency isn't a minor detail — it's a throughput bottleneck.