1
ollama / ollama
Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
View ollama/ollamaTrending open-source projects · updated daily
Local & On-Device AI Projects
Running AI locally means the model, the weights and the data all stay on hardware you control: no API key, no per-token bill, no prompt leaving the machine. This board collects the projects that make that practical — local model runners and servers, chat interfaces and desktop assistants, mobile and on-device inference, and the quantization and compression tooling that lets a model fit into laptop or phone memory. Entries come from GitHub topic pages for local LLMs, llama.cpp-style inference, on-device AI and Ollama, ranked by stars, so a mature server sits next to a desktop client published last month. Each card lists language, stars and forks and opens a page with description, license, activity dates, README and related local AI projects. The practical question is almost always memory: check the quantization notes on the detail page before assuming a model will fit.
Trending Local & On-Device AI Projects
2
open-webui / open-webui
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
View open-webui/open-webui3
HKUDS / nanobot
Ultra-lightweight, open-source, self-hosted personal AI agent framework in Python with WebUI, tools, memory, MCP, multi-agent workflows, automation, and chat apps
View HKUDS/nanobot4
chatchat-space / Langchain-Chatchat
Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM, Qwen 与 Llama 等语言模型的 RAG 与 Agent 应用 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM
View chatchat-space/Langchain-Chatchat5
Zackriya-Solutions / meetily
Privacy first, AI meeting assistant with 4x faster Parakeet/Whisper live transcription, speaker diarization, and Ollama summarization built on Rust. 100% local processing. no cloud required.
View Zackriya-Solutions/meetily6
→7
dyad-sh / dyad
Local, open-source AI app builder for power users ✨ v0 / Lovable / Replit / Bolt alternative 🌟 Star if you like it!
View dyad-sh/dyad8
lss233 / kirara-ai
🤖 可 DIY 的 多模态 AI 聊天机器人 | 🚀 快速接入 微信、 QQ、Telegram、等聊天平台 | 🦈支持DeepSeek、Grok、Claude、Ollama、Gemini、OpenAI | 工作流系统、网页搜索、AI画图、人设调教、虚拟女仆、语音对话 |
View lss233/kirara-ai9
langbot-app / LangBot
Production-grade platform for building agentic IM bots - 生产级多平台智能机器人开发平台/ Agent、知识库编排、插件系统 / Bots for Discord / Slack / LINE / Telegram / WeChat(企业微信, 企微智能机器人, 公众号) / 飞书 / 钉钉 / QQ / Matrix e.g.
View langbot-app/LangBot10
Open-LLM-VTuber / Open-LLM-VTuber
Talk to any LLM with hands-free voice interaction, voice interruption, and Live2D avatar running locally across platforms
View Open-LLM-VTuber/Open-LLM-VTuber11
altic-dev / FluidVoice
Fastest and only macOS Dictation app with on-device STT and custom trained AI enhancement model. Windows pre-build available! A local Wispr Flow alternative.
View altic-dev/FluidVoice12
getumbrel / llama-gpt
A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
View getumbrel/llama-gpt13
→14
sigoden / aichat
All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents, with access to OpenAI, Claude, Gemini, Ollama, Groq, and more.
View sigoden/aichat15
RunanywhereAI / runanywhere-sdks
Production ready toolkit to run AI locally
View RunanywhereAI/runanywhere-sdks16
LearningCircuit / local-deep-research
~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents.
View LearningCircuit/local-deep-research17
reorproject / reor
Private & local AI personal knowledge management app for high entropy people.
View reorproject/reor18
qualcomm / GenieX
Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code
View qualcomm/GenieX19
n4ze3m / page-assist
Use your locally running AI models to assist you in your web browsing
View n4ze3m/page-assist20
OpenCoworkAI / open-codesign
Open-source Claude Design alternative. One-click import your Claude Code / Codex API key. Prompt → prototype / slides / PDF. Multi-model (Claude, GPT, Gemini, Kimi, GLM, Ollama).
View OpenCoworkAI/open-codesign21
ArvinLovegood / go-stock
🦄🦄🦄AI赋能股票分析：AI加持的股票分析/选股工具。股票行情获取，AI热点资讯分析，AI资金/财务分析，涨跌报警推送。支持A股，港股，美股。支持市场整体/个股情绪分析，AI辅助选股等。数据全部保留在本地。支持DeepSeek，OpenAI， Ollama，LMStudio，AnythingLLM，硅基流动，火山方舟，阿里云百炼等平台或模型。
View ArvinLovegood/go-stock22
JerryZLiu / Dayflow
The automatic work journal/time tracker. Privately turns your screen into a timeline of what you actually accomplished. Open-source and local-first.
View JerryZLiu/Dayflow23
open-multi-agent / open-multi-agent
Self-hosted TypeScript agent runtime with durable approvals and verifiable run records. Own it, approve it, audit it.
View open-multi-agent/open-multi-agent24
vas3k / TaxHacker
Self-hosted AI accounting app. LLM analyzer for receipts, invoices, transactions with custom prompts and categories
View vas3k/TaxHacker25
google-ai-edge / LiteRT-LM
LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.
View google-ai-edge/LiteRT-LM26
cactus-compute / cactus
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.
View cactus-compute/cactus27
gluonfield / enchanted
Enchanted is iOS and macOS app for chatting with private self hosted language models such as Llama2, Mistral or Vicuna using Ollama.
View gluonfield/enchanted28
clusterzx / paperless-ai
An automated document analyzer for Paperless-ngx using OpenAI API, Ollama, Deepseek-r1, Azure and all OpenAI API compatible Services to automatically analyze and tag your documents.
View clusterzx/paperless-ai29
dograh-hq / dograh
Open source voice AI platform. Self-hosted alternative to Vapi and Retell. On Prem, BYOK across Speech to Speech or LLM/STT/TTS, with a visual workflow builder, MCP native and telephony support.
View dograh-hq/dograh30
maziyarpanahi / openmed
Local-first healthcare AI: clinical NER & HIPAA PII de-identification that runs 100% on-device. 2,200+ medical models, 21 languages, Apple MLX + Python, no cloud
View maziyarpanahi/openmed31
buxuku / SmartSub
视频转字幕、字幕翻译、AI 配音与声音克隆、字幕烧录——免费开源的一站式桌面工具。基于 Whisper / FunASR 等本地模型离线语音转文字，批量处理 + 全平台 GPU 加速，跨 Windows / macOS / Linux。Free, open-source desktop app to generate, translate
View buxuku/SmartSub32
vinta / pangu.js
Opinionated paranoid text spacing in JavaScript, with on-device AI semantic judgment
View vinta/pangu.js33
CaviraOSS / LongMemory
Local persistent memory store for LLM applications including claude desktop, github copilot, codex, antigravity, etc.
View CaviraOSS/LongMemory34
→35
yuruotong1 / autoMate
Like Manus, Computer Use Agent(CUA) and Omniparser, we are computer-using agents.AI-driven local automation assistant that uses natural language to make computers work by themselves
View yuruotong1/autoMate36
raullenchai / Rapid-MLX
The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing.
View raullenchai/Rapid-MLX37
SciSharp / LLamaSharp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
View SciSharp/LLamaSharp38
twinnydotdev / twinny
The most no-nonsense, locally or API-hosted AI code completion plugin for Visual Studio Code - like GitHub Copilot but 100% free.
View twinnydotdev/twinny39
→40
QiuYannnn / Local-File-Organizer
An AI-powered file management tool that ensures privacy by organizing local texts, images. Using Llama3.2 3B and Llava v1.6 models with the Nexa SDK, it intuitively scans, restructures
View QiuYannnn/Local-File-Organizer41
nicedreamzapp / claude-code-local
Run Claude Code 100% on-device with local AI on Apple Silicon. MLX-native Anthropic-API server. 6 fighters incl.
View nicedreamzapp/claude-code-local42
off-grid-ai / OGAM
The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers.
View off-grid-ai/OGAM43
SharpAI / DeepCamera
Open-Source AI Camera Skills Platform, AI NVR & CCTV Surveillance. Local VLM video analysis with Qwen, DeepSeek, SmolVLM, LLaVA, YOLO26.
View SharpAI/DeepCamera44
→45
Mininglamp-AI / Mano-P
Mano-P: Open-source GUI-VLA agent for edge devices. #1 on OSWorld (specialized, 58.2%). Runs locally on Apple M4 Mac mini/MacBook
View Mininglamp-AI/Mano-P46
Mobile-Artificial-Intelligence / maid
Maid is a free and open source application for interfacing with llama.cpp models locally, and with Anthropic, DeepSeek, Ollama, Mistral and OpenAI models remotely.
View Mobile-Artificial-Intelligence/maid47
datawhalechina / handy-ollama
动手学Ollama，CPU玩转大模型部署，在线阅读地址：https://datawhalechina.github.io/handy-ollama/
View datawhalechina/handy-ollama48
nikmcfly / MiroFish-Offline
Offline multi-agent simulation & prediction engine. English fork of MiroFish with Neo4j + Ollama local stack.
View nikmcfly/MiroFish-Offline49
AtomicBot-ai / atomic-agent
Atomic Agent is a local-first AI agent. Runs open-weight models on your own machine via llama.cpp.
View AtomicBot-ai/atomic-agent50
darrenburns / elia
A snappy, keyboard-centric terminal user interface for interacting with large language models. Chat with ChatGPT, Claude, Llama 3, Phi 3, Mistral, Gemma and more.
View darrenburns/elia51
youssofal / MTPLX
The fastest way to run Qwen 3.8 Flash Next and Qwen 3.8 27B on a Mac: 125 tok/s in OpenCode on an M5 Max. Native MTP speculative decoding on Apple Silicon, exact at any temperature.
View youssofal/MTPLX52
heshengtao / comfyui_LLM_party
LLM Agent Framework in ComfyUI includes MCP sever, Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llms with similar openai / aisuite
View heshengtao/comfyui_LLM_party53
techjarves / Portable-AI-USB
A 100% offline, fully portable, zero-trace AI (Ollama + Llama 3 + AnythingLLM) that runs natively from a USB drive on Windows and Mac.
View techjarves/Portable-AI-USB54
antoinezambelli / forge
A Python framework for self-hosted LLM tool-calling and multi-step agentic workflows
View antoinezambelli/forge55
withcatai / node-llama-cpp
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level
View withcatai/node-llama-cpp56
cogwheel0 / conduit
Native iOS and Android client for Open WebUI, direct OpenAI-compatible, Ollama, and Hermes agents.
View cogwheel0/conduit57
ozgrozer / ai-renamer
A Node.js CLI that uses Ollama and LM Studio models (Llava, Gemma, Llama etc.) to intelligently rename files by their contents
View ozgrozer/ai-renamer58
travisvn / openai-edge-tts
Free, high-quality text-to-speech API endpoint to replace OpenAI, Azure, or ElevenLabs
View travisvn/openai-edge-tts59
fikrikarim / parlor
On-device, real-time multimodal AI with features similar to GPT-Live
View fikrikarim/parlor60
techjarves / USB-Uncensored-LLM
The ultimate zero-install, portable local AI environment. Run high-quality, uncensored LLMs (Gemma, Qwen, NemoMix) directly from any USB drive or SSD.
View techjarves/USB-Uncensored-LLM61
kubewall / kubewall
kubewall - Kubernetes Dashboard for Multi-Cluster Management Real-Time, Self-Hosted, Single Binary, AI-powered. Manage, monitor & debug K8s clusters in your browser, no cloud, no agents.
View kubewall/kubewall62
szczyglis-dev / py-gpt
Desktop AI Assistant powered by GPT-6, GPT-5, Gemini, Claude, Ollama, Grok, DeepSeek, Perplexity, Bielik, chat, agents, vision, voice, RAG, image and video generation, tools, MCP, plugins
View szczyglis-dev/py-gpt63
→64
skalesapp / skales
Personal AI agent for macOS, Windows, Linux, Android & iOS. Set a goal, it works alone: coding (Skales Code), desktop + browser automation, autonomous scheduled Tasks.
View skalesapp/skales65
john-rocky / CoreML-Models
Core ML model zoo for iOS/macOS — PyTorch models converted to ready-to-use .mlpackage, each with a conversion script and SwiftUI sample app.
View john-rocky/CoreML-Models66
→67
software-mansion / react-native-executorch
High-performance, privacy-first on-device AI inference library for React Native, powered by PyTorch's ExecuTorch runtime
View software-mansion/react-native-executorch68
PurpleDoubleD / locally-uncensored
The all-in-one local AI studio for your desktop: chat, image and video generation and a coding agent in one free, open source app. Windows and Linux. No Docker, no terminal, no cloud required.
View PurpleDoubleD/locally-uncensored69
QwenAudio / Fun-ASR
Fun-ASR speech recognition models, with native Hugging Face Transformers support for Fun-ASR-Nano and separate FunASR, vLLM and llama.cpp deployment paths.
View QwenAudio/Fun-ASR70
AtomicBot-ai / Atomic-Chat
Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V
View AtomicBot-ai/Atomic-Chat71
→72
siddsachar / row-bot
Row-Bot - Personal AI Sovereignty. A local-first AI assistant with integrated tools, a personal knowledge graph, voice, vision, shell, browser automation, scheduled tasks, health tracking
View siddsachar/row-bot73
callstackincubator / ai
On-device LLM execution in React Native with Vercel AI SDK compatibility
View callstackincubator/ai74
→75
jofizcd / Soul-of-Waifu
🌌 Give a soul to your favorite characters. Soul of Waifu is a desktop roleplay & AI companion app featuring Live2D/VRM avatars, voice chat & local LLM.
View jofizcd/Soul-of-Waifu76
Muesli-HQ / muesli
Muesli: agent-native local meeting transcription + dictation for macOS (Granola + WisprFlow alternative)
View Muesli-HQ/muesli77
kellyvv / PhoneClaw
PhoneClaw turns phones into local AI agent runtimes with on-device models, native mobile Skills, LiveLand, and optional Mac Gateway inference.
View kellyvv/PhoneClaw78
axoviq-ai / synthadoc
Synthadoc: An open-source LLM knowledge compilation engine that turns raw documents into structured, local-first wikis.
View axoviq-ai/synthadoc79
cosmo-wander-ai / cosmo-edge
Production-grade C++ edge AI engine for video analytics and on-device VLM across Sophon, Rockchip RKNN, and x86, with visual orchestration, real-time OSD, events, and reproducible benchmarks.
View cosmo-wander-ai/cosmo-edge80
chongzhou96 / EdgeSAM
Official PyTorch implementation of "EdgeSAM: Prompt-In-the-Loop Distillation for On-Device Deployment of SAM"
View chongzhou96/EdgeSAM81
mrkrsl / web-search-mcp
A simple, locally hosted Web Search MCP server for use with Local LLMs
View mrkrsl/web-search-mcp82
NativeMindBrowser / NativeMindExtension
NativeMind: Your fully private, open-source, on-device AI assistant
View NativeMindBrowser/NativeMindExtension83
nobodywho-ooo / nobodywho
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
View nobodywho-ooo/nobodywho84
agents-io / PokeClaw
PokeClaw (PocketClaw) — first on-device AI that controls your Android phone. Gemma 4, no cloud, no API key. Poke is short for Pocket.
View agents-io/PokeClaw85
→86
FuJacob / cotabby
Cotabby is local AI autocomplete for your entire Mac. Open source. On device. Everywhere you type.
View FuJacob/cotabby87
facebookresearch / EdgeTAM
[CVPR 2025] Official PyTorch implementation of "EdgeTAM: On-Device Track Anything Model"
View facebookresearch/EdgeTAM88
mit-han-lab / TinyChatEngine
TinyChatEngine: On-Device LLM Inference Library
View mit-han-lab/TinyChatEngine89
kossakovsky / selfhost-ai
🚀 Self-hosted AI automation platform. Deploy n8n, Ollama, Flowise, RAG, Supabase & 30+ tools with one command. Auto HTTPS. Free Zapier/Make alternative.
View kossakovsky/selfhost-ai90
notatestuser / ds4-control
macOS menubar app for fast local DeepSeek V4.1, with 1M context.
View notatestuser/ds4-control91
TheStageAI / TheWhisper
Optimized Whisper models for streaming and on-device use
View TheStageAI/TheWhisper92
ciddwd / overlay-translator
无需 ROOT 的开源 Android 屏幕实时翻译工具，适合游戏、视觉小说和漫画。支持端侧与云端 OCR、离线 LLM、多种翻译服务和文字朗读（TTS），译文可直接显示在画面上。Open-source no-root Android real-time screen translator for games, visual novels, and manga.
View ciddwd/overlay-translator93
Pangu-Immortal / MagicWX
Android 17 local LLM prototype with Jetpack Compose and ONNX Runtime for offline AI inference experiments.
View Pangu-Immortal/MagicWX94
→95
jonigl / mcp-client-for-ollama
Harness the power of local LLMs with this TUI MCP Client for Ollama. Featuring all core MCP primitives (tools, prompts, resources), agent mode, multi-server, model switching, streaming responses
View jonigl/mcp-client-for-ollama96
gavamedia / deltafin
Run full Kimi K3 on a single device. And an OpenAI-compatible API server for local chat and coding agents.
View gavamedia/deltafin97
travisvn / chatterbox-tts-api
Local, OpenAI-compatible text-to-speech (TTS) API using Chatterbox, enabling users to generate voice cloned speech anywhere the OpenAI API is used (e.g. Open WebUI, AnythingLLM, etc.)
View travisvn/chatterbox-tts-api98
leonickson1 / Swiftlet
Swiftlet is a Swift and Metal runtime that runs large Qwen Mixture-of-Experts models locally on Apple devices by streaming expert weights from storage
View leonickson1/Swiftlet99
tetherto / qvac
Open-source local AI SDK - run AI on-device with no cloud, no API keys. Supports GGUF, RAG, image, music, and video generation, speech-to-text, P2P inference, and more.
View tetherto/qvac100
gokayfem / ComfyUI_VLM_nodes
ComfyUI nodes for vision-language models: Qwen3-VL, Moondream 3, Florence-2, SmolVLM2, InternVL, Gemma 3, MiniCPM-V.
View gokayfem/ComfyUI_VLM_nodesLocal Model Runners & Servers
- ollama/ollama — ★ 181,337
- open-webui/open-webui — ★ 152,659
- Zackriya-Solutions/meetily — ★ 30,978
- lss233/kirara-ai — ★ 19,032
- langbot-app/LangBot — ★ 17,931
- ollama/ollama-python — ★ 10,545
- sigoden/aichat — ★ 10,461
- LearningCircuit/local-deep-research — ★ 9,113
Chat UIs & Desktop Assistants
- HKUDS/nanobot — ★ 48,435
- chatchat-space/Langchain-Chatchat — ★ 38,654
- dyad-sh/dyad — ★ 21,594
- altic-dev/FluidVoice — ★ 11,669
- getumbrel/llama-gpt — ★ 10,937
- gluonfield/enchanted — ★ 6,002
- dograh-hq/dograh — ★ 5,702
- buxuku/SmartSub — ★ 5,277
On-Device & Mobile AI
- reorproject/reor — ★ 8,553
- google-ai-edge/LiteRT-LM — ★ 6,485
- cactus-compute/cactus — ★ 6,042
- maziyarpanahi/openmed — ★ 5,347
- vinta/pangu.js — ★ 4,828
- nicedreamzapp/claude-code-local — ★ 3,323
- Mininglamp-AI/Mano-P — ★ 2,779
- Mobile-Artificial-Intelligence/maid — ★ 2,695
Quantization & Model Compression
- off-grid-ai/OGAM — ★ 3,136
- heshengtao/comfyui_LLM_party — ★ 2,364
- chongzhou96/EdgeSAM — ★ 1,178
- tetherto/qvac — ★ 621
- gokayfem/ComfyUI_VLM_nodes — ★ 588
More AI Trending Projects
1
→2
→3
→4
→5
→6
→Running an LLM locally: what you actually need
A local model needs three things: weights in a format the runtime understands, enough memory to hold them, and a runner to load them. The rule of thumb for four-bit quantization is roughly two thirds of a gigabyte per billion parameters, plus room for the context window — so a 7B model fits comfortably in 8 GB of unified memory, while a 70B model does not fit on a laptop at all without aggressive compression or offloading into system RAM. Server-grade engines change that arithmetic with paging, continuous batching and GPU offload, which is why the same weights can be impractical in one tool and comfortable in another.
Start with a runner rather than a chat client. Once the model answers from the command line, everything above it is interface.
Local chat UIs, desktop assistants and on-device inference
The desktop layer is what most people actually want: a chat window that keeps transcripts on disk, can read local files, and talks to whichever local server is running. Those clients are deliberately model-agnostic — they speak the same HTTP API as the popular runners — so the choice of client and the choice of model are independent decisions. On phones the constraints change completely: memory is small, thermals are real, and the interesting projects are the ones shipping compressed models that run in a few hundred megabytes with acceptable latency, rather than the ones streaming a full-size model over the network.
Quantization: why the same model runs on your laptop
Quantization stores weights at lower precision — four bits instead of sixteen — which cuts memory use roughly fourfold and speeds up inference on hardware with no tensor cores. It is not free: below four bits the quality loss becomes visible on reasoning tasks, and some quantized formats only load in specific runtimes. The practical guidance is boring and effective: prefer four-bit formats for local use, keep a higher-precision copy on a server if you need to compare, and test the quantized build on your own inputs instead of trusting a benchmark table.
Local AI questions
What is a local LLM? A language model whose weights run on your own hardware, so prompts and outputs never leave the machine and there is no per-token cost.
Can a local chatbot work offline? Yes, once the model has been downloaded. Runners and desktop clients on this board keep working with the network unplugged; only features that fetch from the internet stop.
What is on-device AI? Inference that happens on the device that produced the data — phone, laptop or embedded board — rather than in a hosted service. Its binding constraints are memory bandwidth and thermal budget.
How much RAM do I need to run a model locally? Roughly two thirds of a gigabyte per billion parameters at four-bit quantization, plus context. An 8B model is comfortable in 8 GB; anything past 30B wants a workstation or a GPU with real VRAM.
Is a local model as good as a hosted one? For summarising, drafting and light coding a good 8B–14B model is often enough. For hard multi-step reasoning the largest hosted models still lead, and they lead by more the longer the task runs.