Footer

Your go-to destination for discovering trending open-source projects and uncovering the insights that matter

Legal

  • Terms and conditions
  • Privacy policy

More

  • Pricing
  • Advertise
  • onedump
  • gosnakego
  • Contact us

Social

  • X (Twitter)
  • GitHub
Trendshift owl
What they build speaks louder than words
© 2026 Trendshift, created by Julian Li with ❤️
SignalAdvertise
Trendshift
SignalAdvertise
Sign in
Trendshift owl

Discover trends that matter

DailyWeeklyMonthlyYearly
Live mentionsTopics
RepositoriesDevelopers
InsightsStats
Bifrost
Bifrost

unclecode/crawl4ai

Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf

#Web scraping
Visit GitHub
Python
86.6k
9.3k
81 contributors
Apache License 2.0
website
AchievementsView all →
FeaturedAdvertise to 180K+ monthly visitors
Bifrost

Fastest Enterprise AI Gateway for Routing, Governing, and Securing all your AI Traffic

Momentum

How fast unclecode/crawl4ai is gaining stars, and where it lands on Trendshift and GitHub Trending

Activity and trending

Stars, forks and merged pull requests each day or month, lined up with the repository's rank on Trendshift Trending

Sep 1 – Oct 31, 2026

Stars +6,045 in 61 days

0200400600
598

Forks +836 in 61 days

050100
75

Merged PRs 46 in 61 days

051015
14

Trendshift Trendingwhere the repository ranked each day it made the top 25

Top 25 overall

2110

Top 25 in Python

2384
Sep 1Sep 9Sep 17Sep 25Oct 3Oct 11Oct 19Oct 27
The left and right arrow keys step from day to day; Home and End go to the first and the last

Stars, forks, merged pull requests and Trendshift Trending rank, by day
DayStars gainedForks gainedMerged PRsAll languagesPython
Tue, Sep 1, 2026+254+270Not trendingNot trending
Wed, Sep 2, 2026+288+180Not trendingNot trending
Thu, Sep 3, 2026+86+60Not trendingNot trending
Fri, Sep 4, 2026+88+80Not trendingNot trending
Sat, Sep 5, 2026+289+390Not trending#23
Sun, Sep 6, 2026+231+220Not trendingNot trending
Mon, Sep 7, 2026+92+130Not trendingNot trending
Tue, Sep 8, 2026+79+70Not trendingNot trending
Wed, Sep 9, 2026+63+71Not trendingNot trending
Thu, Sep 10, 2026+68+100Not trendingNot trending
Fri, Sep 11, 2026+65+70Not trendingNot trending
Sat, Sep 12, 2026+401+500#21#8
Sun, Sep 13, 2026+598+750#10#4
Mon, Sep 14, 2026+297+388Not trendingNot trending
Tue, Sep 15, 2026+93+110Not trendingNot trending
Wed, Sep 16, 2026+123+140Not trendingNot trending
Thu, Sep 17, 2026+60+120Not trendingNot trending
Fri, Sep 18, 2026+78+90Not trendingNot trending
Sat, Sep 19, 2026+41+60Not trendingNot trending
Sun, Sep 20, 2026+99+170Not trendingNot trending
Mon, Sep 21, 2026+63+50Not trendingNot trending
Tue, Sep 22, 2026+47+45Not trendingNot trending
Wed, Sep 23, 2026+79+112Not trendingNot trending
Thu, Sep 24, 2026+54+90Not trendingNot trending
Fri, Sep 25, 2026+48+150Not trendingNot trending
Sat, Sep 26, 2026+45+30Not trendingNot trending
Sun, Sep 27, 2026+52+80Not trendingNot trending
Mon, Sep 28, 2026+57+50Not trendingNot trending
Tue, Sep 29, 2026+67+140Not trendingNot trending
Wed, Sep 30, 2026+93+100Not trendingNot trending
Thu, Oct 1, 2026+37+80Not trendingNot trending
Fri, Oct 2, 2026+35+110Not trendingNot trending
Sat, Oct 3, 2026+47+90Not trendingNot trending
Sun, Oct 4, 2026+60+140Not trendingNot trending
Mon, Oct 5, 2026+36+92Not trendingNot trending
Tue, Oct 6, 2026+61+100Not trendingNot trending
Wed, Oct 7, 2026+61+140Not trendingNot trending
Thu, Oct 8, 2026+107+210Not trendingNot trending
Fri, Oct 9, 2026+71+80Not trendingNot trending
Sat, Oct 10, 2026+67+120Not trendingNot trending
Sun, Oct 11, 2026+98+130Not trendingNot trending
Mon, Oct 12, 2026+129+2514Not trendingNot trending
Tue, Oct 13, 2026+73+82Not trendingNot trending
Wed, Oct 14, 2026+88+72Not trendingNot trending
Thu, Oct 15, 2026+73+170Not trendingNot trending
Fri, Oct 16, 2026+45+120Not trendingNot trending
Sat, Oct 17, 2026+55+60Not trendingNot trending
Sun, Oct 18, 2026+54+100Not trendingNot trending
Mon, Oct 19, 2026+85+120Not trendingNot trending
Tue, Oct 20, 2026+68+151Not trendingNot trending
Wed, Oct 21, 2026+61+100Not trendingNot trending
Thu, Oct 22, 2026+63+62Not trendingNot trending
Fri, Oct 23, 2026+61+120Not trendingNot trending
Sat, Oct 24, 2026+71+140Not trendingNot trending
Sun, Oct 25, 2026+75+150Not trendingNot trending
Mon, Oct 26, 2026+65+95Not trendingNot trending
Tue, Oct 27, 2026+84+190Not trendingNot trending
Wed, Oct 28, 2026+56+130Not trendingNot trending
Thu, Oct 29, 2026+76+102Not trendingNot trending
Fri, Oct 30, 2026+73+60Not trendingNot trending
Sat, Oct 31, 2026+12+10Not trendingNot trending
Total+6,045+83646
  • 1#1 on the list
  • 7Rank that day
  • Not trending
1

    • GitHub trending history

    Where the repository ranked on GitHub Trending, one dot for each day it was on a list

    Sep 29, 2024 – Sep 1, 2026#1 on 3 days

    #1#5#10#15#20#25
    Top 10
    Oct 2024Jan 2025AprJulOctJan 2026AprJul
    The left and right arrow keys step from day to day; Home and End go to the first and the last

    Mentions and achievements

    Where people are talking about unclecode/crawl4ai, and every badge it has earned
    1. Oct 301 post

      • 别再让 Agent「开一场会忘一场」——这 6 个开源项目，把记忆、爬网、深挖装上： 1 → Mem0（约 6.8 万星） 跨会话记住偏好与决策，少重复交代背景 2 → Graphiti（约 3.2 万星） 产品知识 / 竞品口径沉淀成实时图谱

        Read full on X, posted by @perry_weijie

        @perry_weijie · x.com

    2. 9 days earlier
    3. Oct 211 post

      • 想让 AI 读取网页，正文却混着导航栏？🕸️ Crawl4AI 帮你抓取网页并整理成 Markdown 等内容，适合做检索资料和 AI 数据准备。📄 #WebScraping #浇朋友 #蓝朋友 #互关 #fo

        Read full on X, posted by @T1doo_AI

        @T1doo_AI · x.com

    4. Oct 191 post

      • 朋友靠 Crawl4AI + Remotion，上个月赚了 1.5 万 完整提示词完放评论区 我本来以为，他又在折腾什么复杂的 AI Agent 结果发现，他做的事情很直接：帮电商商家盯竞品、做商品视频，再把整个流程自动化 他先用上了 Crawl4AI，可以帮助采集网页内容。 他让 Codex 把采集、数据存储、历史对比和定时任务串起来，持续监测公开商品页面的价格、促销、库存和上新变化…

        Read full on X, posted by @goan999999

        @goan999999 · x.com

    5. 8 days earlier
    6. Oct 111 post

      • 6 GitHub repos to turn any website into clean, structured data

        Read full on Reddit

        r/BestGitHubRepos

    7. Oct 91 post3 posts

      • 做 AI Agent 最烦网页数据清洗。开源的 Crawl4AI 简直是神器，直接把任意复杂网页转成极简、大模型友好的 Markdown，还自带结构化提取和浏览器并发。独立开发做 RAG 和智能体爬虫能省一大半时间：

        Read full on X, posted by @suoyu007

        @suoyu007 · x.com

      • 想用 Hermes 和 Openclaw 白薅全网数据、又不想被反爬逮住的，收拢这 10 个仓库就齐了！ 1️⃣ Firecrawl：URL 往里一撂，它自个儿就把整站刨穿，吐出来直接是 AI 张口就能吞的干净数据，JS 渲染的页面也照扛，15.2 万★，杀进 GitHub Top 100。 🔗 2️⃣ Crawl4AI：把网站捏成 LLM 扫一眼就能读的文本，API key…

        Read full on X, posted by @0xGky

        @0xGky · x.com

      • Claude 加上一套开源研究工具，已经强得有点离谱了。 7 个仓库，就能把单个模型变成一套完整的自主研究系统。 01 Firecrawl ▸ → 把乱七八糟的网页变成模型真正能用的干净数据 02 Browser Use ▸ → 让智能体点击、滚动、搜索，直接操作网页 03 Crawl4AI ▸ → 整站爬取，把有用的内容结构化 04 Qdrant ▸ → 存下智能体找到的一切，方便之后检索…

        Read full on X, posted by @wanerfu

        @wanerfu · x.com

    8. Oct 82 posts

      • Stop spending $50/month on closed-source AI wrappers that break on real codebases. 12 free GitHub repos every AI builder should clone in 2026 (zero paywalls, 100% open-source). Direct links to all 12…

        Read full on X, posted by @ShadabLoveAi

        @ShadabLoveAi · x.com

      • 1. 把 PDF、Office 文档、HTML 和图片转成 markdown，直接喂给任何 AI。 微软内部先用，后来才对外开源。 从一堆脏文档开始，到最后变成 LLM 能读的数据。

        Read full on X, posted by @Techbruneth

        @Techbruneth · x.com

    9. Oct 71 post

      • 还是不敢撒手扒数据，这10个开源工具能把封锁成本给你压下去。 1️⃣ Firecrawl：URL丢进去，整站结构化输出，AI直接消费，JS渲染也不怕，14万★的GitHub Top 100项目。 🔗 2️⃣ Crawl4AI：把网页转成LLM爱吃的纯文本格式，完全免费不要key，16美元月费逼出来的爆款，7万★。 🔗 3️⃣…

        Read full on X, posted by @Manorgw

        @Manorgw · x.com

    You might also like

    Repositories close to unclecode/crawl4ai
    FeaturedAdvertise to 180K+ monthly visitors
    Bifrost

    Fastest Enterprise AI Gateway for Routing, Governing, and Securing all your AI Traffic

    GramosoftAI/GcrawlAI

    Python

    Turn any website into clean, LLM-ready data. Open-source web crawler with stealth mode, distributed crawling, real-time WebSocket progress & Markdown output. Power your AI apps with GcrawlAI.

    594

    0xMassi/webclaw

    Rust

    Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.

    2.4k271

    BrowserCash/teracrawl

    TypeScript

    High-performance web crawler API optimized for LLMs. Turn any search or website into clean Markdown using remote browsers. Firecrawl alternative

    28836

    Anakin-Inc/anakin

    Go

    Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up

    3.9k307

    lumpinif/deepcrawl

    TypeScript

    100% free and full open-source edge Firecrawl alternative with better links extraction for agents - that you can deploy to cloudflare or vercel by yourself.

    69285

    apify/actor-rag-web-browser

    TypeScript

    RAG Web Browser is an Apify Actor to feed your LLM applications and RAG pipelines with up-to-date text content scraped from the web.

    8123

    watercrawl/WaterCrawl

    TypeScript

    Transform Web Content into LLM-Ready Data

    2.2k286

    supermemoryai/markdowner

    TypeScript

    A fast tool to convert any website into LLM-ready markdown data. Built by https://supermemory.ai

    2k154

    fastcrw/crw

    Rust

    Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

    1.2k112