HKUDS/RAG-Anything
AI EngineeringA multimodal RAG framework that parses documents, indexes text, images, tables and equations, and answers queries over them.
Project Overview
RAG-Anything is a Python framework for building retrieval-augmented generation over documents that mix text with images, tables, equations, and other media. It sits on top of LightRAG and combines several stages in one package: document parsing through MinerU, Docling, or PaddleOCR; separate processors for visual, tabular, and mathematical content; multimodal entity and relation extraction into a knowledge graph; and hybrid retrieval that fuses vector similarity with graph traversal README.
The pipeline runs concurrent text and multimodal processing while preserving the original document hierarchy. Parsed elements are turned into structured knowledge-graph entities, and a modality-aware ranking step combines them during retrieval README. This is aimed at people building RAG applications for research papers, technical documentation, financial reports, or enterprise documents, where a text-only retriever would lose the figures and tables.
RAG-Anything also supports a query path that sends retrieved images to a vision model, so questions about charts and figures can be answered from the images themselves rather than from captions alone README. The technical report is available on arXiv.
Quick Start
RAG-Anything supports Python 3.10 and later. The default MinerU parser, however, only supports Python 3.10 through 3.13; on Python 3.14, install without MinerU and use the Docling parser with RAGAnythingConfig(parser="docling") instead README.
- Install the package from PyPI:
pip install raganything
pip install 'raganything[all]' # optional format support; Docling is not included
-
For Office documents (.doc, .docx, .ppt, .pptx, .xls, .xlsx), install LibreOffice, for example
brew install --cask libreofficeon macOS orsudo apt-get install libreofficeon Debian/Ubuntu README. -
Create a
.envfile with your model credentials:
OPENAI_API_KEY=your_openai_api_key
OPENAI_BASE_URL=your_base_url # optional
OUTPUT_DIR=./output
PARSER=mineru
PARSE_METHOD=auto
- Process a document and query it. The example below uses the end-to-end API with an OpenAI-compatible model; replace the API key and paths with your own values:
import asyncio
from functools import partial
from raganything import RAGAnything, RAGAnythingConfig
from lightrag.llm.openai import openai_complete_if_cache, openai_embed
from lightrag.utils import EmbeddingFunc
async def main():
config = RAGAnythingConfig(
working_dir="./rag_storage",
parser="mineru",
parse_method="auto",
enable_image_processing=True,
enable_table_processing=True,
enable_equation_processing=True,
)
rag = RAGAnything(
config=config,
llm_model_func=lambda prompt, **kwargs: openai_complete_if_cache(
"gpt-4o-mini", prompt, api_key="your-api-key", **kwargs
),
embedding_func=EmbeddingFunc(
embedding_dim=3072,
max_token_size=8192,
func=partial(openai_embed.func, model="text-embedding-3-large", api_key="your-api-key"),
),
)
await rag.process_document_complete(file_path="document.pdf", output_dir="./output/")
print(await rag.aquery("Summarize the figures and tables", mode="hybrid"))
asyncio.run(main())
A single-file CLI example is also available: python examples/raganything_example.py path/to/document.pdf --api-key YOUR_API_KEY --parser mineru README.
Constraints and Configuration
- Parser choice is fixed when the
RAGAnythingobject is created, viaRAGAnythingConfig(parser=...)or thePARSERenvironment variable; it cannot be switched perprocess_document_complete()call README. - MinerU 4.x is not supported. The framework drives MinerU 3.x; if
mineru --versionreports 4.x, install a 3.x release in the same environment README. On minimal Linux or Docker images, MinerU also needs OpenCV system libraries such aslibxcb.so.1andlibGL.so.1README. - Optional formats require their own dependencies: extended image formats need
raganything[image], text files needraganything[text], audio needsraganything[audio]andenable_audio_processing=True, and video needsraganything[video],ffmpegon PATH, andenable_video_processing=TrueREADME. - HTML files are accepted only by the Docling parser; MinerU and PaddleOCR reject them README.
- Re-indexing a document after switching graph or vector storage backends requires passing
force_multimodal_reprocess=True; otherwise multimodal entities and relations are not written to the new backend README. - Markdown image references to local files are sent to the configured vision model, so only ingest Markdown whose image references you trust README.
- RAG-Anything is MIT licensed.
HotRepo's Take
On the Python daily list for 2026-11-17, RAG-Anything sat at rank 12 with 9 new stars for the day, against a total of about 24.6k stars and 2.9k forks snapshot. The single recorded snapshot does not support a trend claim, but it does show the repository still receiving daily attention long after its June 2025 creation. For readers evaluating it, the differentiating factor is the breadth of the pipeline: one package covers parsing, multimodal graph indexing, and vision-enhanced querying, which is more than most text-oriented RAG libraries offer. That breadth also means the setup surface is wide, with parser versions, system libraries, and optional extras that need to be matched to the formats you actually ingest.
Written by AI from the project's README, updated . The project's own docs are the source of truth.
You might also like
- JackWang666666/my-ragA dual-backend RAG knowledge-base system letting teams compare Python/LangChain and Java/Spring AI over one shared data set.
- VectifyAI/PageIndexReasoning-based RAG for long professional PDFs, replacing vector similarity search with an LLM navigating a document tree.
- Marker-Inc-Korea/AutoRAGA local agent that indexes your files, mail and chats and answers questions with cited sources.
- infiniflow/ragflowA self-hostable RAG and document-understanding engine that turns messy PDFs and office files into grounded LLM context.
- neuml/txtaiAn all-in-one Python framework for semantic search, RAG pipelines, LLM workflows and agents built on an embeddings database.