microsoft/onnxruntimePublic

• Widely adopted

  21,802 stars

• Battle-tested

  7 years of history

• Very active

  2,840 commits in 52 weeks

• Community-driven

  ~983 contributors

• Well documented

  High community health score

• Permissive license

  MIT

ONNX Runtime is a robust engine designed to execute machine learning models across diverse hardware platforms with maximum efficiency. It accelerates inference by natively supporting models trained in frameworks like PyTorch, TensorFlow, and scikit-learn via the Open Neural Network Exchange (ONNX) format. The runtime optimizes execution through advanced graph transformations and dynamically leverages hardware-specific execution providers, such as TensorRT for NVIDIA GPUs or CoreML for Apple Silicon. Beyond inferencing, it also provides tools to significantly accelerate model training times on multi-node GPU clusters.

Machine learning engineers, AI researchers, and production deployment teams looking to optimize and standardize model inference across varied hardware ecosystems.

• Hardware Acceleration: Utilize specialized execution providers (like CUDA, TensorRT, or DirectML) to maximize performance on specific hardware.
• Framework Agnostic: Seamlessly run models trained in PyTorch, TensorFlow/Keras, and classical ML libraries without framework lock-in.
• Graph Optimization: Automatically apply graph transforms and optimizations to reduce execution latency and memory footprint during inference.
• Cross-Platform Execution: Deploy the exact same optimized model across Windows, Linux, macOS, Android, and iOS environments.
• Training Acceleration: Speed up transformer model training on multi-node NVIDIA GPU setups with minimal changes to existing PyTorch scripts.

Getting started: Install via package managers (e.g., pip install onnxruntime) or refer to the official documentation for language-specific bindings.

1. ONNX Runtime v1.28.2v1.28.2Sep 3, 20261.4K downloads

   This is a patch release on top of [v1.28.1](https://github.com/microsoft/onnxruntime/releases/tag/v1.28.1), containing a targeted fix for Compile API model serialization. ## Highlights ### Bug Fixes - Fixed Compile API callback serialization to prevent duplicate graph nodes, inputs, outputs, and value information in emitted optimized models, including models with embedded or external initializers ([#32303](https://github.com/microsoft/onnxruntime/pull/32303)) ## Contributors Thanks to our contributor for this release! [@adrastogi](https://github.com/adrastogi) Full Changelog: [v1.28.1...v1.28.2](https://github.com/microsoft/onnxruntime/compare/v1.28.1...v1.28.2) > These release notes were drafted with assistance from GitHub Copilot.

2. ONNX Runtime WebGPU Plugin EP v0.3.0plugin-ep-webgpu/v0.3.0Aug 24, 2026

   ONNX Runtime WebGPU Plugin EP 0.3.0 expands model and data-type coverage, improves generative-model performance, and strengthens configuration, reliability, and release tooling. These release notes were drafted with AI assistance. ## Highlights ### Model and operator coverage - Added initial PagedAttention support, MRotaryEmbedding, GRU, DFT, PRelu, HardSwish, Trilu, Max and Min, and MatMulBnb4. ([#31611](https://github.com/microsoft/onnxruntime/pull/31611), [#31976](https://github.com/microsoft/onnxruntime/pull/31976), [#29840](https://github.com/microsoft/onnxruntime/pull/29840), [#29454](https://github.com/microsoft/onnxruntime/pull/29454), [#30512](https://github.com/microsoft/onnxruntime/pull/30512), [#29828](https://github.com/microsoft/onnxruntime/pull/29828), [#29845](https://github.com/microsoft/onnxruntime/pull/29845), [#29833](https://github.com/microsoft/onnxruntime/pull/29833), [#29587](https://github.com/microsoft/onnxruntime/pull/29587)) - Expanded integer support across common operators, including `int64` for Add, Cast, Clip, Concat, Equal, Gather, Min, Max, ReduceSum, Reshape, Sub, Tile, and Where; `uint8` for Cast, Expand, Gather, and Reshape; and `in

3. ONNX Runtime v1.28.1v1.28.1Aug 18, 20266.1K downloads

   This is a patch release on top of [v1.28.0](https://github.com/microsoft/onnxruntime/releases/tag/v1.28.0), containing support for device-free WebGPU compilation, improved compatibility with sandboxed Windows processes, and targeted graph-validation fixes. ## WebGPU EP - Added support for device-free compile-only sessions, enabling offline graph transformation and optimized-model serialization without access to GPU hardware ([#29681](https://github.com/microsoft/onnxruntime/pull/29681)) ## Bug Fixes - Prevented an access violation in Windows processes under Win32k lockdown by skipping DXGI device discovery ([#29755](https://github.com/microsoft/onnxruntime/pull/29755)) - Allowed zero-input `EPContext` nodes, aligning their schema with support for compiling zero-input models ([#29799](https://github.com/microsoft/onnxruntime/pull/29799)) - Hardened FastGelu fusion to skip malformed `Mul` and `Pow` patterns ([#32016](https://github.com/microsoft/onnxruntime/pull/32016)) - Added validation for in-memory external initializer references, rejecting unregistered or mismatched data before graph transformation ([#32042](https://github.com/microsoft/onnxruntime/pull/32042))

4. ONNX Runtime CUDA Plugin EP 0.1.0plugin-ep-cuda/v0.1.0Aug 17, 2026524 downloads

   This is the first release of ONNX Runtime CUDA Plugin EP, providing CUDA execution as a separately packaged plugin execution provider. These notes cover commits affecting CUDA Plugin EP core code, CMake integration, and its primary build and package pipeline. Please refer to [QUICK_START.md](https://github.com/microsoft/onnxruntime/blob/main/docs/cuda_plugin_ep/QUICK_START.md) for the usage. ## Highlights ### Plugin Runtime - Introduces the CUDA Plugin EP core and makes it the default CUDA provider implementation ([#27816](https://github.com/microsoft/onnxruntime/pull/27816), [#29544](https://github.com/microsoft/onnxruntime/pull/29544)). - Adds arena allocation, resource accounting, available-resource reporting, and IOBinding synchronization ([#27931](https://github.com/microsoft/onnxruntime/pull/27931), [#28028](https://github.com/microsoft/onnxruntime/pull/28028), [#28103](https://github.com/microsoft/onnxruntime/pull/28103), [#27919](https://github.com/microsoft/onnxruntime/pull/27919)). - Adds provider options for user compute streams, copy behavior, EP-level unified streams, and external allocators, with compatibility back to ONNX Runtime 1.24.4 through versi

5. ONNX Runtime v1.29.0v1.29.0Aug 12, 202665.1K downloads

   ## Announcements & Breaking Changes - onnxruntime-web has announced the deprecation of WebGL and JSEP. The native WebGPU EP is the recommended path going forward. See the deprecation and migration plans for details ([#29716](https://github.com/microsoft/onnxruntime/pull/29716), [#31683](https://github.com/microsoft/onnxruntime/pull/31683)). - POSIX telemetry is now available on Linux, macOS, Android, and iOS when ONNX Runtime is built with telemetry enabled. It does not change the public ABI, WebAssembly remains telemetry-free, and setting `ORT_DISABLE_TELEMETRY=1` before initialization disables non-Windows telemetry for the process ([#27379](https://github.com/microsoft/onnxruntime/pull/27379), [#29872](https://github.com/microsoft/onnxruntime/pull/29872)). - The unused internal `onnxruntime/python/tools/tensorrt` dashboard tooling was removed. This does not affect the TensorRT Execution Provider APIs ([#29395](https://github.com/microsoft/onnxruntime/pull/29395)). ## Security Fixes ### Path, bounds, and input validation - Fixed a path traversal vulnerability in TensorRT and NvTensorRTRTX engine refitting by making external-data path validation unconditional ([#29396

• f/prompts.chat

  f.k.a. Awesome ChatGPT Prompts. Share, discover, and collect prompts from the community. Free and open source — self-host for your organization with complete privacy.

  169.9K stars · HTML

• huggingface/transformers

  🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

  165.1K stars · Python

• Comfy-Org/ComfyUI

  The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.

  132.4K stars · Python

• ggml-org/llama.cpp

  LLM inference in C/C++

  127.7K stars · C++

• rasbt/LLMs-from-scratch

  Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

  104.7K stars · Jupyter Notebook

• Developer-Y/cs-video-courses

  List of Computer Science courses with video lectures.

  83.5K stars