Data & Analytics

Databases, ETL pipelines, analytics engines, and data infrastructure.

56 tracked repositories, ranked by stars.

Filter by topic

56 of 56 repositories

  • firecrawl/firecrawl

    Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

    AI summary: An API service that crawls websites and turns them into clean, LLM-ready markdown data.

    205,346dataTypeScriptAGPL-3.0
  • microsoft/markitdown

    Python tool for converting files and office documents to Markdown.

    AI summary: A lightweight Python utility for converting various file formats into clean Markdown.

    197,266dataPythonMIT
  • Graphify-Labs/graphify

    Turn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowledge graph. A /graphify skill for Claude Code, Cursor, Codex, and Gemini CLI: local deterministic AST parsing, every edge explained, no vector store.

    AI summary: Data transformation engine for converting unstructured text into structured, queryable knowledge graphs automatically.

    151,207dataPythonApache-2.0
  • TauricResearch/TradingAgents

    TradingAgents: Multi-Agents LLM Financial Trading Framework

    AI summary: A robust research framework for developing, testing, and deploying autonomous AI agents for algorithmic financial tra...

    123,303dataPythonApache-2.0
  • koala73/worldmonitor

    Real-time global intelligence dashboard. AI-powered news aggregation, geopolitical monitoring, and infrastructure tracking in a unified situational awareness interface

    AI summary: A real-time global intelligence dashboard utilizing AI and dual map engines to track geopolitical and infrastructure...

    90,935dataTypeScriptAGPL-3.0
  • opendatalab/MinerU

    Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.

    AI summary: High-accuracy parsing engine that converts complex documents into LLM-ready Markdown and JSON.

    84,006dataPythonOther
  • elastic/elasticsearch

    Free and Open Source, Distributed, RESTful Search Engine

    AI summary: Distributed, RESTful search and analytics engine optimized for speed and relevance on production-scale workloads.

    78,658dataJavaOther
  • openbq-org/OpenBB

    Open Data Platform for analysts, quants and AI agents.

    AI summary: An open-source investment research software platform offering extensive financial data access and analytics.

    75,580dataPythonOther
  • ZhuLinsen/daily_stock_analysis

    LLM 驱动的多市场股票智能分析系统:多源行情、实时新闻、决策看板与自动推送,支持零成本定时运行。 LLM-powered multi-market stock analysis system with multi-source market data, real-time news, decision dashboard, automated notifications, and cost-free scheduled runs.

    AI summary: An automated system for scraping, analyzing, and visualizing daily A-share stock market data.

    73,136dataPythonMIT
  • NanmiCoder/MediaCrawler

    小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 | 评论爬虫、微博帖子 | 评论爬虫、百度贴吧帖子 | 百度贴吧评论回复爬虫 | 知乎问答文章|评论爬虫

    AI summary: An automated data mining script for scraping user profiles and posts from major Chinese social media platforms.

    71,270dataPythonOther
  • bilawalsidhu/gods-eye-view

    God's Eye View: a spy satellite simulator in your browser, except the data is real. Live open source spatial intelligence on a photorealistic 3D globe. Official site: godseyeview.ai

    AI summary: Browser-based spy satellite simulator offering live open-source spatial intelligence on a photorealistic 3D globe.

    65,706dataJavaScriptOther
  • virattt/ai-hedge-fund

    An AI Hedge Fund Team

    AI summary: An experimental framework for simulating algorithmic trading strategies using multi-agent LLM consensus.

    64,776dataPythonMIT
  • sansan0/TrendRadar

    ⭐AI-driven public opinion & trend monitor with multi-platform aggregation, RSS, and smart alerts.🎯 告别信息过载,你的 AI 舆情监控助手与热点筛选工具!聚合多平台热点 + RSS 订阅,支持关键词精准筛选。AI 智能筛选新闻 + AI 翻译 + AI 分析简报直推手机,也支持接入 MCP 架构,赋能 AI 自然语言对话分析、情感洞察与趋势预测等。支持 Docker ,数据本地/云端自持。集成微信/飞书/钉钉/Telegram/邮件/ntfy/bark/slack 等渠道智能推送。

    AI summary: An automated intelligence gathering tool that monitors and summarizes emerging tech trends across platforms.

    63,747dataPythonGPL-3.0
  • PostHog/posthog

    :hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

    AI summary: A comprehensive, open-source product analytics suite providing tools for tracking, feature flags, and session recording.

    40,856dataPythonOther
  • google/langextract

    A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

    AI summary: AI-powered tool for extracting structured data from unstructured text using large language models.

    39,340dataPythonApache-2.0
  • Fincept-Corporation/FinceptTerminal

    FinceptTerminal is a modern finance application offering advanced market analytics, investment research, and economic data tools, designed for interactive exploration and data-driven decision-making in a user-friendly environment.

    AI summary: A native C++ financial intelligence platform combining institutional analytics with local AI agents.

    34,013dataC++AGPL-3.0
  • topoteretes/cognee

    Cognee is the open-source AI memory platform for agents. Give your AI agents persistent long-term memory with small models for free

    AI summary: An open-source AI memory platform that provides persistent long-term memory and knowledge graphs for agents.

    33,432dataPythonApache-2.0
  • nautechsystems/nautilus_trader

    Production-grade Rust-native trading engine with deterministic event-driven architecture

    AI summary: A highly deterministic, Rust-native algorithmic trading engine for multiple asset classes.

    31,686dataRustLGPL-3.0
  • supermemoryai/supermemory

    Memory and context engine + app that is extremely fast, scalable, and can be run fully locally. The Memory API for the AI era.

    AI summary: A scalable, local-first memory and context engine designed to serve as a personal or company brain for AI applications.

    31,677dataTypeScriptMIT
  • t8y2/dbx

    25 MB lightweight cross-platform database client for 100+ databases, including MySQL, PostgreSQL, SQLite, Redis, MongoDB, DuckDB, SQL Server, and Dameng. Built-in AI, MCP Server, CLI, desktop and Docker. | 轻量级跨平台数据库管理工具,支持 MySQL、PostgreSQL、SQLite、Redis、MongoDB、达梦等 100+ 数据库,提供桌面端、Docker、CLI、内置 AI 助手和 MCP。

    AI summary: Lightweight desktop database client bundling 100+ database drivers, a built-in AI assistant, and an MCP Server in 25MB.

    31,040dataRustApache-2.0
  • TencentCloud/TencentDB-Agent-Memory

    TencentDB Agent Memory is a team-level memory hub for AI Agents — turning conversations, docs, and code into four reusable memory assets (Chat Memory, Skill, LLM-Wiki, Code-Graph) that are governed, shared, and equipped across agents and frameworks.

    AI summary: A team-level memory hub governing chat, skills, wikis, and code graphs for AI agents.

    30,974dataTypeScriptOther
  • opendataloader-project/opendataloader-pdf

    PDF Parser for AI-ready data. Automate PDF accessibility. Open-source.

    AI summary: A high-accuracy PDF parser that extracts AI-ready Markdown, JSON, and HTML using deterministic and hybrid approaches.

    30,347dataJavaApache-2.0
  • clockworklabs/SpacetimeDB

    Development at the speed of light

    AI summary: A unified platform merging relational databases and application servers via WebAssembly for ultra-low latency apps.

    25,566dataRustOther
  • tursodatabase/turso

    A SQL database in Rust: SQLite-compatible, now also speaking Postgres (experimental). The LLVM of databases.

    AI summary: An edge-hosted, globally distributed SQL database built on libSQL, compatible with SQLite.

    25,302dataRustMIT
  • hasaneyldrm/exercises-dataset

    1,324-exercise fitness dataset — animation GIFs, 180×180 thumbnails, muscle-group & equipment data, and step-by-step instructions in 6 languages. The exercise data layer behind the LogPress app.

    AI summary: A comprehensive JSON dataset detailing thousands of fitness exercises with categorical metadata.

    23,877dataHTMLOther
  • firecrawl/pdf-inspector

    Fast Rust library for PDF inspection, classification, and text extraction. Intelligently detects scanned vs text-based PDFs to enable smart routing decisions.

    AI summary: A Rust library for intelligent PDF classification, layout-aware text extraction, and clean Markdown conversion.

    20,061dataRustMIT
  • microsoft/data-formulator

    🪄 Data Formulator is an interactive AI-powered data analysis system makes it easy to connect, explore and visualize data.

    AI summary: An interactive, AI-powered data analysis system that generates visualizations from raw datasets automatically.

    17,691dataPythonMIT
  • RyanCodrai/turbovec

    A vector index built on TurboQuant, written in Rust with Python bindings

    AI summary: A memory-efficient Rust vector index with Python bindings, built on Google's TurboQuant algorithm.

    17,498dataRustMIT
  • xbtlin/ai-berkshire

    AI 时代的伯克希尔:基于 Claude Code / Codex 的价值投资研究框架。巴菲特·芒格·段永平·李录四大师方法论 + 多Agent并行研究。| AI-era Berkshire: a value investing research framework built for Claude Code / Codex. 4 masters' methodologies + multi-agent adversarial analysis.

    AI summary: An institutional-grade, multi-agent investment research framework combining fundamental analysis with LLM automation.

    17,196dataHTMLMIT
  • alibaba/zvec

    A lightweight, lightning-fast, in-process vector database

    AI summary: An open-source, in-process vector database by Alibaba built for lightweight, low-latency similarity search.

    16,261dataC++Apache-2.0
  • run-llama/liteparse

    A fast, helpful, and open-source document parser

    AI summary: Liteparse provides a lightweight utility for parsing complex document structures into clean text for LLM applications.

    13,069dataRustApache-2.0
  • datalab-to/chandra

    OCR model that handles complex tables, forms, handwriting with full layout.

    AI summary: A state-of-the-art document intelligence library for extracting structured data from complex documents using AI.

    12,733dataPythonApache-2.0
  • apache/cassandra

    Open source transactional distributed database. Linear scalability and proven fault-tolerance on commodity hardware or cloud infrastructure without compromising performance.

    AI summary: Highly scalable, partitioned row store database designed for zero single points of failure.

    10,152dataJavaApache-2.0
  • GoogleCloudPlatform/knowledge-catalog

    Google Cloud Knowledge Catalog Tools and Samples

    AI summary: An AI-powered data catalog providing a dynamic knowledge graph for context-aware enterprise AI agents.

    9,696dataTypeScriptApache-2.0
  • xberg-io/xberg

    The fast, precise document-intelligence engine for every language: text, tables and metadata from 100+ formats, with OCR, transcription and code intelligence.

    AI summary: A high-performance document intelligence framework for extracting structured data from over 98 file formats.

    9,603dataRustMIT
  • shy3130/tick-stock-panel

    TSP自托管、零运维的 A 股「自建策略 + 监控 + 回测」量化工作台 | LLM能力驱使策略定制+个股分析+复盘 | 自由接入第三方数据源与个性化扩展数据 | 个人开源

    AI summary: Self-hosted quantitative trading workspace for Chinese stocks with AI assistance and comprehensive API capabilities.

    8,105dataPythonMIT
  • wpzzz/blocked-sites-in-south-korea

    AI summary: A maintained dataset and analysis of websites blocked by South Korean internet service providers.

    8,058dataPython
  • google-deepmind/weathernext

    AI summary: A highly accurate global, medium-range atmospheric forecasting model developed by Google DeepMind.

    7,750dataPythonApache-2.0
  • The-Swarm-Corporation/AutoHedge

    Build your autonomous hedge fund in minutes. AutoHedge harnesses the power of swarm intelligence and AI agents to automate market analysis, risk management, and trade execution.

    AI summary: An enterprise-grade autonomous hedge fund system powered by a multi-agent pipeline and swarm intelligence.

    6,712dataPythonMIT
  • malisper/pgrust

    Postgres rewritten in Rust, now faster than Postgres and Clickhouse

    AI summary: A high-performance Rust reimplementation of PostgreSQL 18 designed to be 300x faster for analytics.

    5,937dataRustAGPL-3.0