Data & Analytics
Databases, ETL pipelines, analytics engines, and data infrastructure.
56 tracked repositories, ranked by stars.
56 of 56 repositories
firecrawl/firecrawl
Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.
AI summary: An API service that crawls websites and turns them into clean, LLM-ready markdown data.
205,346dataTypeScriptAGPL-3.0microsoft/markitdown
Python tool for converting files and office documents to Markdown.
AI summary: A lightweight Python utility for converting various file formats into clean Markdown.
197,266dataPythonMITGraphify-Labs/graphify
Turn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowledge graph. A /graphify skill for Claude Code, Cursor, Codex, and Gemini CLI: local deterministic AST parsing, every edge explained, no vector store.
AI summary: Data transformation engine for converting unstructured text into structured, queryable knowledge graphs automatically.
151,207dataPythonApache-2.0TauricResearch/TradingAgents
TradingAgents: Multi-Agents LLM Financial Trading Framework
AI summary: A robust research framework for developing, testing, and deploying autonomous AI agents for algorithmic financial tra...
123,303dataPythonApache-2.0koala73/worldmonitor
Real-time global intelligence dashboard. AI-powered news aggregation, geopolitical monitoring, and infrastructure tracking in a unified situational awareness interface
AI summary: A real-time global intelligence dashboard utilizing AI and dual map engines to track geopolitical and infrastructure...
90,935dataTypeScriptAGPL-3.0opendatalab/MinerU
Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.
AI summary: High-accuracy parsing engine that converts complex documents into LLM-ready Markdown and JSON.
84,006dataPythonOtherelastic/elasticsearch
Free and Open Source, Distributed, RESTful Search Engine
AI summary: Distributed, RESTful search and analytics engine optimized for speed and relevance on production-scale workloads.
78,658dataJavaOtheropenbq-org/OpenBB
Open Data Platform for analysts, quants and AI agents.
AI summary: An open-source investment research software platform offering extensive financial data access and analytics.
75,580dataPythonOtherZhuLinsen/daily_stock_analysis
LLM 驱动的多市场股票智能分析系统:多源行情、实时新闻、决策看板与自动推送,支持零成本定时运行。 LLM-powered multi-market stock analysis system with multi-source market data, real-time news, decision dashboard, automated notifications, and cost-free scheduled runs.
AI summary: An automated system for scraping, analyzing, and visualizing daily A-share stock market data.
73,136dataPythonMITNanmiCoder/MediaCrawler
小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 | 评论爬虫、微博帖子 | 评论爬虫、百度贴吧帖子 | 百度贴吧评论回复爬虫 | 知乎问答文章|评论爬虫
AI summary: An automated data mining script for scraping user profiles and posts from major Chinese social media platforms.
71,270dataPythonOtherbilawalsidhu/gods-eye-view
God's Eye View: a spy satellite simulator in your browser, except the data is real. Live open source spatial intelligence on a photorealistic 3D globe. Official site: godseyeview.ai
AI summary: Browser-based spy satellite simulator offering live open-source spatial intelligence on a photorealistic 3D globe.
65,706dataJavaScriptOthervirattt/ai-hedge-fund
An AI Hedge Fund Team
AI summary: An experimental framework for simulating algorithmic trading strategies using multi-agent LLM consensus.
64,776dataPythonMITsansan0/TrendRadar
⭐AI-driven public opinion & trend monitor with multi-platform aggregation, RSS, and smart alerts.🎯 告别信息过载,你的 AI 舆情监控助手与热点筛选工具!聚合多平台热点 + RSS 订阅,支持关键词精准筛选。AI 智能筛选新闻 + AI 翻译 + AI 分析简报直推手机,也支持接入 MCP 架构,赋能 AI 自然语言对话分析、情感洞察与趋势预测等。支持 Docker ,数据本地/云端自持。集成微信/飞书/钉钉/Telegram/邮件/ntfy/bark/slack 等渠道智能推送。
AI summary: An automated intelligence gathering tool that monitors and summarizes emerging tech trends across platforms.
63,747dataPythonGPL-3.0PostHog/posthog
:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
AI summary: A comprehensive, open-source product analytics suite providing tools for tracking, feature flags, and session recording.
40,856dataPythonOthergoogle/langextract
A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.
AI summary: AI-powered tool for extracting structured data from unstructured text using large language models.
39,340dataPythonApache-2.0Fincept-Corporation/FinceptTerminal
FinceptTerminal is a modern finance application offering advanced market analytics, investment research, and economic data tools, designed for interactive exploration and data-driven decision-making in a user-friendly environment.
AI summary: A native C++ financial intelligence platform combining institutional analytics with local AI agents.
34,013dataC++AGPL-3.0topoteretes/cognee
Cognee is the open-source AI memory platform for agents. Give your AI agents persistent long-term memory with small models for free
AI summary: An open-source AI memory platform that provides persistent long-term memory and knowledge graphs for agents.
33,432dataPythonApache-2.0nautechsystems/nautilus_trader
Production-grade Rust-native trading engine with deterministic event-driven architecture
AI summary: A highly deterministic, Rust-native algorithmic trading engine for multiple asset classes.
31,686dataRustLGPL-3.0supermemoryai/supermemory
Memory and context engine + app that is extremely fast, scalable, and can be run fully locally. The Memory API for the AI era.
AI summary: A scalable, local-first memory and context engine designed to serve as a personal or company brain for AI applications.
31,677dataTypeScriptMITt8y2/dbx
25 MB lightweight cross-platform database client for 100+ databases, including MySQL, PostgreSQL, SQLite, Redis, MongoDB, DuckDB, SQL Server, and Dameng. Built-in AI, MCP Server, CLI, desktop and Docker. | 轻量级跨平台数据库管理工具,支持 MySQL、PostgreSQL、SQLite、Redis、MongoDB、达梦等 100+ 数据库,提供桌面端、Docker、CLI、内置 AI 助手和 MCP。
AI summary: Lightweight desktop database client bundling 100+ database drivers, a built-in AI assistant, and an MCP Server in 25MB.
31,040dataRustApache-2.0TencentCloud/TencentDB-Agent-Memory
TencentDB Agent Memory is a team-level memory hub for AI Agents — turning conversations, docs, and code into four reusable memory assets (Chat Memory, Skill, LLM-Wiki, Code-Graph) that are governed, shared, and equipped across agents and frameworks.
AI summary: A team-level memory hub governing chat, skills, wikis, and code graphs for AI agents.
30,974dataTypeScriptOtheropendataloader-project/opendataloader-pdf
PDF Parser for AI-ready data. Automate PDF accessibility. Open-source.
AI summary: A high-accuracy PDF parser that extracts AI-ready Markdown, JSON, and HTML using deterministic and hybrid approaches.
30,347dataJavaApache-2.0clockworklabs/SpacetimeDB
Development at the speed of light
AI summary: A unified platform merging relational databases and application servers via WebAssembly for ultra-low latency apps.
25,566dataRustOthertursodatabase/turso
A SQL database in Rust: SQLite-compatible, now also speaking Postgres (experimental). The LLVM of databases.
AI summary: An edge-hosted, globally distributed SQL database built on libSQL, compatible with SQLite.
25,302dataRustMIThasaneyldrm/exercises-dataset
1,324-exercise fitness dataset — animation GIFs, 180×180 thumbnails, muscle-group & equipment data, and step-by-step instructions in 6 languages. The exercise data layer behind the LogPress app.
AI summary: A comprehensive JSON dataset detailing thousands of fitness exercises with categorical metadata.
23,877dataHTMLOtherfirecrawl/pdf-inspector
Fast Rust library for PDF inspection, classification, and text extraction. Intelligently detects scanned vs text-based PDFs to enable smart routing decisions.
AI summary: A Rust library for intelligent PDF classification, layout-aware text extraction, and clean Markdown conversion.
20,061dataRustMITmicrosoft/data-formulator
🪄 Data Formulator is an interactive AI-powered data analysis system makes it easy to connect, explore and visualize data.
AI summary: An interactive, AI-powered data analysis system that generates visualizations from raw datasets automatically.
17,691dataPythonMITRyanCodrai/turbovec
A vector index built on TurboQuant, written in Rust with Python bindings
AI summary: A memory-efficient Rust vector index with Python bindings, built on Google's TurboQuant algorithm.
17,498dataRustMITxbtlin/ai-berkshire
AI 时代的伯克希尔:基于 Claude Code / Codex 的价值投资研究框架。巴菲特·芒格·段永平·李录四大师方法论 + 多Agent并行研究。| AI-era Berkshire: a value investing research framework built for Claude Code / Codex. 4 masters' methodologies + multi-agent adversarial analysis.
AI summary: An institutional-grade, multi-agent investment research framework combining fundamental analysis with LLM automation.
17,196dataHTMLMITalibaba/zvec
A lightweight, lightning-fast, in-process vector database
AI summary: An open-source, in-process vector database by Alibaba built for lightweight, low-latency similarity search.
16,261dataC++Apache-2.0run-llama/liteparse
A fast, helpful, and open-source document parser
AI summary: Liteparse provides a lightweight utility for parsing complex document structures into clean text for LLM applications.
13,069dataRustApache-2.0datalab-to/chandra
OCR model that handles complex tables, forms, handwriting with full layout.
AI summary: A state-of-the-art document intelligence library for extracting structured data from complex documents using AI.
12,733dataPythonApache-2.0apache/cassandra
Open source transactional distributed database. Linear scalability and proven fault-tolerance on commodity hardware or cloud infrastructure without compromising performance.
AI summary: Highly scalable, partitioned row store database designed for zero single points of failure.
10,152dataJavaApache-2.0GoogleCloudPlatform/knowledge-catalog
Google Cloud Knowledge Catalog Tools and Samples
AI summary: An AI-powered data catalog providing a dynamic knowledge graph for context-aware enterprise AI agents.
9,696dataTypeScriptApache-2.0xberg-io/xberg
The fast, precise document-intelligence engine for every language: text, tables and metadata from 100+ formats, with OCR, transcription and code intelligence.
AI summary: A high-performance document intelligence framework for extracting structured data from over 98 file formats.
9,603dataRustMITshy3130/tick-stock-panel
TSP自托管、零运维的 A 股「自建策略 + 监控 + 回测」量化工作台 | LLM能力驱使策略定制+个股分析+复盘 | 自由接入第三方数据源与个性化扩展数据 | 个人开源
AI summary: Self-hosted quantitative trading workspace for Chinese stocks with AI assistance and comprehensive API capabilities.
8,105dataPythonMITwpzzz/blocked-sites-in-south-korea
AI summary: A maintained dataset and analysis of websites blocked by South Korean internet service providers.
8,058dataPythongoogle-deepmind/weathernext
AI summary: A highly accurate global, medium-range atmospheric forecasting model developed by Google DeepMind.
7,750dataPythonApache-2.0The-Swarm-Corporation/AutoHedge
Build your autonomous hedge fund in minutes. AutoHedge harnesses the power of swarm intelligence and AI agents to automate market analysis, risk management, and trade execution.
AI summary: An enterprise-grade autonomous hedge fund system powered by a multi-agent pipeline and swarm intelligence.
6,712dataPythonMITmalisper/pgrust
Postgres rewritten in Rust, now faster than Postgres and Clickhouse
AI summary: A high-performance Rust reimplementation of PostgreSQL 18 designed to be 300x faster for analytics.
5,937dataRustAGPL-3.0