RAG, Document Parsing & Web Data

Docling

docling-project/docling

Trending

IBM-born toolkit that converts PDFs, DOCX, PPTX and images into structured Markdown/JSON with layout and tables.

MarkItDown

microsoft/markitdown

Trending

Microsoft utility that converts Office files, PDFs, HTML and more into Markdown for LLM input.

Unstructured

Unstructured-IO/unstructured

Popular

Open-source ETL library for partitioning and chunking dozens of document types for RAG pipelines.

MinerU

opendatalab/MinerU

Rising

High-accuracy PDF-to-Markdown/JSON extractor from OpenDataLab, strong on formulas, tables and Chinese docs.

LlamaParse

llamaindex.ai

LlamaIndex's hosted, LLM-powered document parser for complex PDFs, tables and charts.

Firecrawl

firecrawl/firecrawl

Trending

API and open-source crawler that turns websites into clean Markdown or structured data for LLMs.

Crawl4AI

unclecode/crawl4ai

Trending

Open-source async web crawler and scraper producing LLM-friendly Markdown, built on Playwright.

RAGFlow

infiniflow/ragflow

Trending

Open-source RAG engine with deep document understanding, chunk visualization and agent workflows.

GraphRAG

microsoft/graphrag

Microsoft's pipeline that builds a knowledge graph from documents to answer global, multi-hop questions.

LightRAG

HKUDS/LightRAG

Rising

Simple, fast graph-based RAG combining knowledge graphs with vector retrieval at lower cost than GraphRAG.

Tavily

tavily.com

Search API built for agents and RAG that returns clean, relevant web results in one call.

Exa

exa.ai

Neural web search API with embeddings-based retrieval and content extraction for AI apps.