langroid/langroid

Documentation · Examples Repo · Discord · Contributing

Langroid is an intuitive, lightweight, extensible and principled Python framework to easily build LLM-powered applications, from CMU and UW-Madison researchers. You set up Agents, equip them with optional components (LLM, vector-store and tools/functions), assign them tasks, and have them collaboratively solve a problem by exchanging messages. This Multi-Agent paradigm is inspired by the Actor Framework (but you do not need to know anything about this!).

Langroid is a fresh take on LLM app-development, where considerable thought has gone into simplifying the developer experience; it does not use Langchain, or any other LLM framework, and works with practically any LLM.

🔥 ✨ A Claude Code plugin is available to accelerate Langroid development with built-in patterns and best practices.

🔥 Read the (WIP) overview of the langroid architecture, and a quick tour of Langroid.

🔥 MCP Support: Allow any LLM-Agent to leverage MCP Servers via Langroid's simple MCP tool adapter that converts the server's tools into Langroid's ToolMessage instances.

📢 Companies are using/adapting Langroid in production. Here is a quote:

Nullify uses AI Agents for secure software development.

It finds, prioritizes and fixes vulnerabilities. We have internally adapted Langroid's multi-agent orchestration framework in production, after evaluating CrewAI, Autogen, LangChain, Langflow, etc. We found Langroid to be far superior to those frameworks in terms of ease of setup and flexibility. Langroid's Agent and Task abstractions are intuitive, well thought out, and provide a great developer experience. We wanted the quickest way to get something in production. With other frameworks it would have taken us weeks, but with Langroid we got to good results in minutes. Highly recommended!
-- Jacky Wong, Head of AI at Nullify.

🔥 See this Intro to Langroid blog post from the LanceDB team

🔥 Just published in ML for Healthcare (2024): a Langroid-based Multi-Agent RAG system for pharmacovigilance, see blog post

We welcome contributions: See the contributions document for ideas on what to contribute.

Are you building LLM Applications, or want help with Langroid for your company, or want to prioritize Langroid features for your company use-cases? Prasad Chalasani is available for consulting (advisory/development): pchalasani at gmail dot com.

Sponsorship is also accepted via GitHub Sponsors

Questions, Feedback, Ideas? Join us on Discord!

Quick glimpse of coding with Langroid

This is just a teaser; there's much more, like function-calling/tools, Multi-Agent Collaboration, Structured Information Extraction, DocChatAgent (RAG), SQLChatAgent, non-OpenAI local/remote LLMs, etc. Scroll down or see docs for more. See the Langroid Quick-Start Colab that builds up to a 2-agent information-extraction example using the OpenAI ChatCompletion API.

🔥 just released! Example script showing how you can use Langroid multi-agents and tools to extract structured information from a document using only a local LLM (Mistral-7b-instruct-v0.2).

import langroid as lr
import langroid.language_models as lm
set up LLM
llm_cfg = lm.OpenAIGPTConfig(
  # any model served via an OpenAI-compatible API
  chat_model=lm.OpenAIChatModel.GPT4o, # or, e.g., "ollama/mistral"
)
use LLM directly
mdl = lm.OpenAIGPT(llm_cfg)
response = mdl.chat("What is the capital of Ontario?", max_tokens=10)
use LLM in an Agent
agent_cfg = lr.ChatAgentConfig(llm=llm_cfg)
agent = lr.ChatAgent(agent_cfg)
agent.llm_response("What is the capital of China?") 
response = agent.llm_response("And India?") # maintains conversation state
wrap Agent in a Task to run interactive loop with user (or other agents)
task = lr.Task(agent, name="Bot", system_message="You are a helpful assistant")
task.run("Hello") # kick off with user saying "Hello"
2-Agent chat loop: Teacher Agent asks questions to Student Agent
teacher_agent = lr.ChatAgent(agent_cfg)
teacher_task = lr.Task(
  teacher_agent, name="Teacher",
  system_message="""
    Ask your student concise numbers questions, and give feedback. 
    Start with a question.
    """
)
student_agent = lr.ChatAgent(agent_cfg)
student_task = lr.Task(
  student_agent, name="Student",
  system_message="Concisely answer the teacher's questions.",
  single_round=True,
)
teacher_task.add_sub_task(student_task)
teacher_task.run()

🔥 Updates/Releases

Click to expand

• Sep 2026:
• 0.68.0: Removed OpenAIAssistant.

OpenAI sunset the Assistants API beta on 2026-08-26; every endpoint now returns HTTP 404, so the class could not function. It has been removed along with its tests and examples. OpenAIGPT -- which nearly all Langroid code uses -- is unaffected. See the migration note for equivalents to threads, file_search, and code_interpreter.

• Aug 2026:
• 0.67.0 Security hardening:

per-provider env_prefix for vector-store configs (env-var naming change -- see migration notes), generalized taint propagation across tool re-emission paths, and a one-time warning when FileAttachment payloads inflate context preflight.

• 0.66.0 Big community batch (14 PRs):

Milvus vector store (thanks @zc277584121); Markdown/HTML document parsing (thanks @nuthalapativarun); cooperative max_time task budgets, MCP tool namespacing for multi-server agents, and portable JSON chat-history snapshots (thanks @Whxuan0701); video attachments (thanks @octo-patch); retrieval score thresholds (thanks @Koushik-Salammagari); even context-overflow truncation and several routing/parsing fixes -- full details in the release notes.

• Aug 2025:
• 0.59.0 Complete Pydantic V2 Migration -

5-50x faster validation, modern Python patterns, 100% backward compatible.

• Jul 2025:
• 0.58.0 Crawl4AI integration -

browser-based web crawling with Playwright for JavaScript-heavy sites, no API key required (thank you @abab-dev!).

• 0.57.0 HTML Logger for interactive task visualization -

self-contained HTML logs with collapsible entries, auto-refresh, and persistent UI state.

• Jun 2025:
• 0.56.0 TaskTool for delegating tasks to sub-agents -

enables agents to spawn sub-agents with specific tools and configurations.

• 0.55.0 Event-based task termination with done_sequences -

declarative task completion using event patterns.

• 0.54.0 Portkey AI Gateway support - access 200+ models

across providers through unified API with caching, retries, observability.

• Mar-Apr 2025:
• 0.53.0 MCP Tools Support.
• 0.52.0 Multimodal support, i.e. allow PDF, image

inputs to LLM.

• 0.51.0 LLMPdfParser, generalizing

GeminiPdfParser to parse documents directly with LLM.

• 0.50.0 Structure-aware Markdown chunking with chunks

enriched by section headers.

• 0.49.0 Enable easy switch to LiteLLM Proxy-server
• 0.48.0 Exa Crawler, Markitdown Parser
• 0.47.0 Support Firecrawl URL scraper/crawler -

thanks @abab-dev

• 0.46.0 Support LangDB LLM Gateway - thanks @MrunmayS.
• 0.45.0 Markdown parsing with Marker - thanks @abab-dev
• 0.44.0 Late imports to reduce startup time. Thanks

@abab-dev

• Feb 2025:
• 0.43.0: GeminiPdfParser for parsing PDF using

Gemini LLMs - Thanks @abab-dev.

• 0.42.0: markitdown parser for pptx,xlsx,xls files

Thanks @abab-dev.

• 0.41.0: pinecone vector-db (Thanks @coretado),

Tavily web-search (Thanks @Sozhan308), Exa web-search (Thanks @MuddyHope).

• 0.40.0: pgvector vector-db. Thanks @abab-dev.
• 0.39.0: ChatAgentConfig.handle_llm_no_tool for

handling LLM "forgetting" to use a tool.

• 0.38.0: Gemini embeddings - Thanks @abab-dev)
• 0.37.0: New PDF Parsers: docling, pymupdf4llm
• Jan 2025:
• 0.36.0: Weaviate vector-db support (thanks @abab-dev).
• 0.35.0: Capture/Stream reasoning content from

Reasoning LLMs (e.g. DeepSeek-R1, OpenAI o1) in addition to final answer.

• 0.34.0: DocChatAgent

chunk enrichment to improve retrieval. (collaboration with @dfm88).

• 0.33.0 Move from Poetry to uv! (thanks @abab-dev).
• 0.32.0 DeepSeek v3 support.
• Dec 2024:
• 0.31.0 Azure OpenAI Embeddings
• 0.30.0 Llama-cpp embeddings (thanks @Kwigg).
• 0.29.0 Custom Azure OpenAI Client (thanks

@johannestang).

• 0.28.0 ToolMessage: _handler field to override

default handler method name in request field (thanks @alexagr).

• 0.27.0 OpenRouter Support.
• 0.26.0 Update to latest Chainlit.
• 0.25.0 True Async Methods for agent and

user-response (thanks @alexagr).

• Nov 2024:
• 0.24.0:

Enables support for Agents with strict JSON schema output format on compatible LLMs and strict mode for the OpenAI tools API. (thanks @nilspalumbo).

• 0.23.0:

support for LLMs (e.g. Qwen2.5-Coder-32b-Instruct) hosted on glhf.chat

• 0.22.0:

Optional parameters to truncate large tool results.

• 0.21.0 Direct support for Gemini models via OpenAI client instead of using LiteLLM.
• 0.20.0 Support for

ArangoDB Knowledge Graphs.

• Oct 2024:
• [0.18.0] LLMConfig.async_stream_quiet flag to

turn off LLM output in async + stream mode.

• [0.17.0] XML-based tools, see docs.
• Sep 2024:
• 0.16.0 Support for OpenAI o1-mini and o1-preview models.
• 0.15.0 Cerebras API support -- run llama-3.1 models hosted on Cerebras Cloud (very fast inference).
• 0.14.0 DocChatAgent uses Reciprocal Rank Fusion (RRF) to rank chunks retrieved by different methods.
• 0.12.0 run_batch_task new option -- stop_on_first_result - allows termination of batch as soon as any task returns a result.
• Aug 2024:
• 0.11.0 Polymorphic Task.run(), Task.run_async.
• 0.10.0 Allow tool handlers to return arbitrary result type, including other tools.
• 0.9.0 Orchestration Tools, to signal various task statuses, and to pass messages between agents.
• 0.7.0 OpenAI tools API support, including multi-tools.
• Jul 2024:
• 0.3.0: Added FastEmbed embeddings from Qdrant
• Jun 2024:
• 0.2.0: Improved lineage tracking, granular sub-task configs, and a new tool, RewindTool,

that lets an agent "rewind and redo" a past message (and all dependent messages are cleared out thanks to the lineage tracking). Read notes here.

• May 2024:
• Slimmer langroid: All document-parsers (i.e. pdf, doc, docx) and most

vector-databases (except qdrant) are now optional/extra dependencies, which helps reduce build size, script start-up time, and install time. For convenience various grouping of "extras" are provided, e.g. doc-chat, db (for database-related dependencies). See updated install instructions below and in the docs.

• Few-shot examples for tools: when defining a ToolMessage, previously you were able to include a classmethod named examples,

and a random example from this list would be used to generate a 1-shot example for the LLM. This has been improved so you can now supply a list of examples where each example is either a tool instance, or a tuple of (description, tool instance), where the description is a "thought" that leads the LLM to use the tool (see example in the docs). In some scenarios this can improve LLM tool generation accuracy. Also, now instead of a random example, ALL examples are used to generate few-shot examples.

• Infinite loop detection for task loops of cycle-length <= 10 (configurable

in TaskConfig. Only detects _exact_ loops, rather than _approximate_ loops where the entities are saying essentially similar (but not exactly the same) things repeatedly.

• "@"-addressing: any entity can address any other by name, which can be the name

of an agent's responder ("llm", "user", "agent") or a sub-task name. This is a simpler alternative to the RecipientTool mechanism, with the tradeoff that since it's not a tool, there's no way to enforce/remind the LLM to explicitly specify an addressee (in scenarios where this is important).

• Much-Improved Citation

generation and display when using DocChatAgent.

• gpt-4o is now the default LLM throughout; Update tests and examples to work

with this LLM; use tokenizer corresponding to the LLM.

• gemini 1.5 pro support via litellm
• QdrantDB: update to support learned sparse embeddings.
• Apr 2024:
• 0.1.236: Support for open LLMs hosted on Groq, e.g. specify

chat_model="groq/llama3-8b-8192". See tutorial.

• 0.1.235: Task.run(), Task.run_async(), run_batch_tasks have max_cost

and max_tokens params to exit when tokens or cost exceed a limit. The result ChatDocument.metadata now includes a status field which is a code indicating a task completion reason code. Also task.run() etc can be invoked with an explicit session_id field which is used as a key to look up various settings in Redis cache. Currently only used to look up "kill status" - this allows killing a running task, either by task.kill() or by the classmethod Task.kill_session(session_id). For example usage, see the test_task_kill in tests/main/test_task.py

• Mar 2024:
• 0.1.216: Improvements to allow concurrent runs of DocChatAgent, see the

test_doc_chat_agent.py in particular the test_doc_chat_batch(); New task run utility: run_batch_task_gen where a task generator can be specified, to generate one task per input.

• 0.1.212: ImagePdfParser: support for extracting text from image-based PDFs.

(this means DocChatAgent will now work with image-pdfs).

• 0.1.194 - 0.1.211: Misc fixes, improvements, and features:
• Big enhancement in RAG performance (mainly, recall) due to a fix in Relevance

Extractor

• DocChatAgent context-window fixes
• Anthropic/Claude3 support via Litellm
• URLLoader: detect file time from header when URL doesn't end with a

recognizable suffix like .pdf, .docx, etc.

• Misc lancedb integration fixes
• Auto-select embedding config based on whether sentence_transformer module is available.
• Slim down dependencies, make some heavy ones optional, e.g. unstructured,

haystack, chromadb, mkdocs, huggingface-hub, sentence-transformers.

• Easier top-level imports from import langroid as lr
• Improve JSON detection, esp from weak LLMs
• Feb 2024:
• 0.1.193: Support local LLMs using Ollama's new OpenAI-Compatible server:

simply specify chat_model="ollama/mistral". See release notes.

• 0.1.183: Added Chainlit support via callbacks.

See examples.

• Jan 2024:
• 0.1.175
• Neo4jChatAgent to chat with a neo4j knowledge-graph.

(Thanks to Mohannad!). The agent uses tools to query the Neo4j schema and translate user queries to Cypher queries, and the tool handler executes these queries, returning them to the LLM to compose a natural language response (analogous to how SQLChatAgent works). See example script using this Agent to answer questions about Python pkg dependencies.

• Support for .doc file parsing (in addition to .docx)
• Specify optional formatter param

in OpenAIGPTConfig to ensure accur