Trending repositories: crawler

3 tracked repositories tagged with crawler, ordered by stars. Use the topic filters below to narrow further.

Filter by topic

3 of 3 repositories

  • firecrawl/firecrawl

    The context API to search, scrape, and interact with the web at scale. 🔥

    AI summary: An API service that crawls websites and turns them into clean, LLM-ready markdown data.

    162,732dataTypeScriptAGPL-3.0

  • D4Vinci/Scrapling

    🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

    AI summary: A fast, undetectable web scraping framework for Python.

    72,975dataPythonBSD-3-Clause

  • NanmiCoder/MediaCrawler

    小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 ｜ 评论爬虫、微博帖子 ｜ 评论爬虫、百度贴吧帖子 ｜ 百度贴吧评论回复爬虫 | 知乎问答文章｜评论爬虫

    AI summary: A comprehensive crawler for extracting posts and comments from major Chinese social media platforms.

    60,171dataPythonOther