StarTrail-org/LEANN

git clone https://github.com/yichuan-w/LEANN.git leann
cd leann
git submodule update --init --recursive

macOS:

Note: DiskANN requires MacOS 13.3 or later.

brew install libomp boost protobuf zeromq pkgconf
uv sync --extra diskann

Linux (Ubuntu/Debian):

Note: On Ubuntu 20.04, you may need to build a newer Abseil and pin Protobuf (e.g., v3.20.x) for building DiskANN. See Issue #30 for a step-by-step note.

You can manually install Intel oneAPI MKL instead of libmkl-full-dev for DiskANN. You can also use libopenblas-dev for building HNSW only, by removing --extra diskann in the command below.

sudo apt-get update && sudo apt-get install -y \
  libomp-dev libboost-all-dev protobuf-compiler libzmq3-dev \
  pkg-config libabsl-dev libaio-dev libprotobuf-dev \
  libmkl-full-dev
uv sync --extra diskann

Linux (Arch Linux):

sudo pacman -Syu && sudo pacman -S --needed base-devel cmake pkgconf git gcc \
  boost boost-libs protobuf abseil-cpp libaio zeromq
For MKL in DiskANN
sudo pacman -S --needed base-devel git
git clone https://aur.archlinux.org/paru-bin.git
cd paru-bin && makepkg -si
paru -S intel-oneapi-mkl intel-oneapi-compiler
source /opt/intel/oneapi/setvars.sh
uv sync --extra diskann

Linux (RHEL / CentOS Stream / Oracle / Rocky / AlmaLinux):

See Issue #50 for more details.

sudo dnf groupinstall -y "Development Tools"
sudo dnf install -y libomp-devel boost-devel protobuf-compiler protobuf-devel \
  abseil-cpp-devel libaio-devel zeromq-devel pkgconf-pkg-config
For MKL in DiskANN
sudo dnf install -y intel-oneapi-mkl intel-oneapi-mkl-devel \
  intel-oneapi-openmp || sudo dnf install -y intel-oneapi-compiler
source /opt/intel/oneapi/setvars.sh
uv sync --extra diskann

Windows:

Requires Visual Studio 2022 Build Tools with the C++ desktop development workload, and vcpkg.

# Install toolchain (if not already present)
choco install cmake swig pkgconfiglite nuget.commandline -y
Install C++ dependencies via vcpkg
vcpkg install zeromq:x64-windows openblas:x64-windows lapack:x64-windows `
  boost-program-options:x64-windows protobuf:x64-windows
Set environment variables (adjust VCPKG_ROOT to your vcpkg path)
$env:CMAKE_PREFIX_PATH = "$env:VCPKG_ROOT\installed\x64-windows"
$env:PKG_CONFIG_PATH = "$env:VCPKG_ROOT\installed\x64-windows\lib\pkgconfig"
$env:PKG_CONFIG_EXECUTABLE = "C:\ProgramData\chocolatey\bin\pkg-config.exe"
$env:OPENBLAS_LIB = "$env:VCPKG_ROOT\installed\x64-windows\lib\openblas.lib"
$env:PATH += ";$env:VCPKG_ROOT\installed\x64-windows\bin"
$env:PATH += ";$env:VCPKG_ROOT\installed\x64-windows\tools\protobuf"
uv sync --extra diskann

Set your OpenAI API key as an environment variable:

export OPENAI_API_KEY="your-api-key-here"

Make sure to use --llm openai flag when using the CLI. You can also specify the model name with --llm-model flag.

Thanks to the widespread adoption of the OpenAI API format, LEANN is compatible out-of-the-box with a vast array of LLM and embedding providers. Simply set the OPENAI_BASE_URL and OPENAI_API_KEY environment variables to connect to your preferred service.

export OPENAI_API_KEY="xxx"
export OPENAI_BASE_URL="http://localhost:1234/v1" # base url of the provider

To use OpenAI compatible endpoint with the CLI interface:

If you are using it for text generation, make sure to use --llm openai flag and specify the model name with --llm-model flag.

If you are using it for embedding, set the --embedding-mode openai flag and specify the model name with --embedding-model .

-----

Below is a list of base URLs for common providers to get you started.

🖥️ Local Inference Engines (Recommended for full privacy)

| Provider | Sample Base URL | | ---------------- | --------------------------- | | Ollama | http://localhost:11434/v1 | | LM Studio | http://localhost:1234/v1 | | vLLM | http://localhost:8000/v1 | | llama.cpp | http://localhost:8080/v1 | | SGLang | http://localhost:30000/v1 | | LiteLLM | http://localhost:4000 |

-----

☁️ Cloud Providers

🚨 A Note on Privacy: Before choosing a cloud provider, carefully review their privacy and data retention policies. Depending on their terms, your data may be used for their own purposes, including but not limited to human reviews and model training, which can lead to serious consequences if not handled properly.

| Provider | Base URL | | ---------------- | ---------------------------------------------------------- | | OpenAI | https://api.openai.com/v1 | | OpenRouter | https://openrouter.ai/api/v1 | | Gemini | https://generativelanguage.googleapis.com/v1beta/openai/ | | x.AI (Grok) | https://api.x.ai/v1 | | Groq AI | https://api.groq.com/openai/v1 | | DeepSeek | https://api.deepseek.com/v1 | | SiliconFlow | https://api.siliconflow.cn/v1 | | Zhipu (BigModel) | https://open.bigmodel.cn/api/paas/v4/ | | Mistral AI | https://api.mistral.ai/v1 | | Anthropic | https://api.anthropic.com/v1 | | Jina AI (Embeddings) | https://api.jina.ai/v1 |

💡 Tip: Separate Embedding Provider

> To use a different provider for embeddings (e.g., Jina AI) while using another for LLM, use --embedding-api-base and --embedding-api-key:

> leann build my-index --docs ./docs \

If your provider isn't on this list, don't worry! Check their documentation for an OpenAI-compatible endpoint—chances are, it's OpenAI Compatible too!

macOS:

First, download Ollama for macOS.

# Pull a lightweight model (recommended for consumer hardware)
ollama pull llama3.2:1b

Linux:

# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh
Start Ollama service manually
ollama serve &
Pull a lightweight model (recommended for consumer hardware)
ollama pull llama3.2:1b

All RAG examples share these common parameters. Interactive mode is available in all examples - simply run without --query to start a continuous Q&A session where you can ask multiple questions. Type 'quit' to exit.

# Environment Variables (GPU Device Selection)
LEANN_EMBEDDING_DEVICE       # GPU for embedding model (e.g., cuda:0, cuda:1, cpu)
LEANN_LLM_DEVICE             # GPU for HFChat LLM (e.g., cuda:1, or "cuda" for multi-GPU auto)
Core Parameters (General preprocessing for all examples)
--index-dir DIR              # Directory to store the index (default: current directory)
--query "YOUR QUESTION"      # Single query mode. Omit for interactive chat (type 'quit' to exit), and now you can play with your index interactively
--max-items N                # Limit data preprocessing (default: -1, process all data)
--force-rebuild              # Force rebuild index even if it exists
Embedding Parameters
--embedding-model MODEL      # e.g., facebook/contriever, text-embedding-3-small, mlx-community/Qwen3-Embedding-0.6B-8bit or nomic-embed-text
--embedding-mode MODE        # sentence-transformers, openai, mlx, or ollama
LLM Parameters (Text generation models)
--llm TYPE                   # LLM backend: openai, ollama, hf, or anthropic (default: openai)
--llm-model MODEL            # Model name (default: gpt-4o) e.g., gpt-4o-mini, llama3.2:1b, Qwen/Qwen2.5-1.5B-Instruct
--thinking-budget LEVEL      # Thinking budget for reasoning models: low/medium/high (supported by o3, o3-mini, GPT-Oss:20b, and other reasoning models)
Search Parameters
--top-k N                    # Number of results to retrieve (default: 20)
--search-complexity N        # Search complexity for graph traversal (default: 32)
Chunking Parameters
--chunk-size N               # Size of text chunks (default varies by source: 256 for most, 192 for WeChat)
--chunk-overlap N            # Overlap between chunks (default varies: 25-128 depending on source)
Index Building Parameters
--backend-name NAME          # Backend to use: hnsw or diskann (default: hnsw)
--graph-degree N             # Graph degree for index construction (default: 32)
--build-complexity N         # Build complexity for index construction (default: 64)
--compact / --no-compact     # Use compact storage (default: true). Must be no-compact for no-recompute build.
--recompute / --no-recompute # Enable/disable embedding recomputation (default: enabled). Should not do a no-recompute search in a recompute build.

Parameters

--data-dir DIR           # Directory containing documents to process (default: data)
--file-types .ext .ext   # Filter by specific file types (optional - all LlamaIndex supported types if omitted)

Example Commands

# Process all documents with larger chunks for academic papers
python -m apps.document_rag --data-dir "~/Documents/Papers" --chunk-size 1024
Filter only markdown and Python files with smaller chunks
python -m apps.document_rag --data-dir "./docs" --chunk-size 256 --file-types .md .py
Enable AST-aware chunking for code files
python -m apps.document_rag --enable-code-chunking --data-dir "./my_project"
Or use the specialized code RAG for better code understanding
python -m apps.code_rag --repo-dir "./my_codebase" --query "How does authentication work?"

Prerequisites

# Install dependencies
uv pip install colpali_engine pdf2image pillow matplotlib qwen_vl_utils einops seaborn
brew install poppler  # macOS only, for PDF processing

Build Index

python -m apps.colqwen_rag build \
  --pdfs ./pdf_directory/ \
  --index my_index \
  --model colqwen2  # or colpali

Search

python -m apps.colqwen_rag search my_index "your question here" --top-k 5

Models

• ColQwen2 (colqwen2): Latest vision-language model with improved performance
• ColPali (colpali): Proven multimodal retriever

Parameters

--mail-path PATH         # Path to specific mail directory (auto-detects if omitted)
--include-html          # Include HTML content in processing (useful for newsletters)

Example Commands

# Search work emails from a specific account
python -m apps.email_rag --mail-path "~/Library/Mail/V10/WORK_ACCOUNT"
Find all receipts and order confirmations (includes HTML)
python -m apps.email_rag --query "receipt order confirmation invoice" --include-html

Once the index is built, you can ask questions like:

• "Find emails from my boss about deadlines"
• "What did John say about the project timeline?"
• "Show me emails about travel expenses"