Trending repositories: quantization

5 of 5 repositories

• microsoft/BitNet

  Official inference framework for 1-bit LLMs

  AI summary: An official inference framework optimized for running 1-bit Large Language Models efficiently on CPUs.

  39,816ai-mlC++MITllmquantizationinferencecpu

• RyanCodrai/turbovec

  A vector index built on TurboQuant, written in Rust with Python bindings

  AI summary: Rust vector index with Python bindings based on TurboQuant for extreme RAM compression.

  14,655dataRustMITannavx512embeddingembeddings

• TheTom/turboquant_plus

  AI summary: Advanced KV cache compression techniques merged into major LLM inference engines.

  7,002ai-mlPythonApache-2.0kv-cachellm-inferencequantizationvllm

• openai/parameter-golf

  Train the smallest LM you can that fits in 16MB. Best model wins!

  AI summary: A rigorous challenge optimizing language models for minimal parameter counts and rapid training times.

  5,174ai-mlPythonMITmachine-learningllm-optimizationchallengeparameter-efficiency

• PrismML-Eng/Bonsai-demo

  Bonsai Demo

  AI summary: Interactive demonstration of the Bonsai machine learning optimization platform.

  2,170ai-mlShellApache-2.0bonsaillamacppllmmlx