Trending repositories: web-scraping
14 tracked repositories tagged with web-scraping, ordered by stars. Use the topic filters below to narrow further.
14 of 14 repositories
firecrawl/firecrawl
The context API to search, scrape, and interact with the web at scale. 🔥
AI summary: An API service that crawls websites and turns them into clean, LLM-ready markdown data.
162,732dataTypeScriptAGPL-3.0browser-use/browser-use
🌐 Make websites accessible for AI agents. Automate tasks online with ease.
AI summary: Browser Use is an open-source library that enables AI agents to interact with and automate web browsers seamlessly.
108,175developer-toolsPythonMITD4Vinci/Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
AI summary: A fast, undetectable web scraping framework for Python.
72,975dataPythonBSD-3-ClauseItzCrazyKns/Vane
Vane is an AI-powered answering engine.
AI summary: Open-source AI-powered search engine alternative focusing on direct answers and real-time web synthesis.
35,896ai-mlTypeScriptMITlightpanda-io/browser
Lightpanda: the headless browser designed for AI and automation
AI summary: A headless browser built in Zig specifically designed for AI agents and web scraping.
33,558developer-toolsZigAGPL-3.0JCodesMore/ai-website-cloner-template
Clone any website with one command using AI coding agents
AI summary: One-command template to clone any website using AI coding agents.
31,118webTypeScriptMITCloakHQ/CloakBrowser
Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 30/30 tests passed.
AI summary: A privacy-first web browser designed to seamlessly bypass anti-bot protections and tracking.
29,675developer-toolsPythonMITh4ckf0r0day/obscura
The headless browser for AI agents and web scraping
AI summary: A Rust-based headless browser tailored for AI agents and automated scraping.
20,317developer-toolsRustApache-2.0browser-use/browser-harness
Browser Harness | Self-healing harness that enables LLMs to complete any task.
AI summary: Connects an LLM directly to a real Chrome browser via a thin, editable CDP harness for unconstrained web automation.
16,546developer-toolsPythonMITUsagi-org/ai-goofish-monitor
基于 Playwright 和AI实现的闲鱼多任务实时/定时监控与智能分析系统，配备了功能完善的后台管理UI。帮助用户从闲鱼海量商品中，找到心仪产品。
AI summary: A Playwright-driven, multi-task AI monitoring system for Xianyu, equipped with a comprehensive web dashboard.
14,117ai-mlPythonMITpinchtab/pinchtab
High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
AI summary: A keyboard-centric tab manager for power users with too many open tabs.
9,854productivityGoMITcitrolabs/ego-lite
The fastest browser for AI agents to run browser automation, built for sharing your logged-in browser state with your AI agents, like Codex or Claude Code, without disturbing you. Zero cost, zero config.
AI summary: A dedicated macOS browser for AI agents to run automated tasks in isolated spaces.
9,171productivityJavaScriptMITKnockOutEZ/wigolo
The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.
AI summary: Local-first web intelligence and search endpoint for AI agents.
4,369developer-toolsTypeScriptOthertnm/zclaw
Your personal AI assistant at all-in 888KiB (~35KB in app code). Running on an ESP32. GPIO, cron, custom tools, memory, and more.
AI summary: ZClaw is a highly concurrent, distributed web scraper built for massive data extraction.
2,203dataCMIT