Andyyyy64/whichllmPublic

Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly.

AI summary: A CLI tool that aggressively profiles local hardware to recommend and rank the best-fitting local LLMs from HuggingFace.

Stars
6.8K
+9 today
Forks
379
Watchers
23
Open issues
6
Open PRs
2
Contributors
~30
Commits
459
Branches
67

PythonMITCreated Mar 4, 2026Last push 1d agoLatest release v0.6.8+15 stars this week+70 this month

Quick answers

What is whichllm?
A CLI tool that aggressively profiles local hardware to recommend and rank the best-fitting local LLMs from HuggingFace.
What does whichllm do?
whichllm is a Python command-line utility explicitly designed to eliminate the guesswork involved in selecting local Large Language Models. It actively profiles your specific machine's hardware, detecting exact GPU VRAM, system RAM bandwidth, and CPU capabilities. It then queries live HuggingFace databases and ranks the best available models based on constraints like partial RAM offloading and strict VRAM limits, entirely bypassing simplistic parameter count metrics. The tool also features powerful hardware simulation capabilities, allowing users to test hypothetical multi-GPU configurations before committing to expensive hardware purchases.
Who is whichllm for?
AI engineers, researchers, and hobbyists who run models locally and require an accurate, automated way to match HuggingFace models to strict hardware limits. Familiarity with Python is recommended.
How do I get started with whichllm?
uvx whichllm@latest
How popular is whichllm on GitHub?
Andyyyy64/whichllm has 6,847 stars and 379 forks on GitHub, and gained 15 stars in the last 7 days.
What license does whichllm use?
Andyyyy64/whichllm is released under the MIT license.

Star history

since Jul 28, 2026
02K4K6KJul 2026Sep 2026Oct 2026Nov 2026
6.8K stars as of Nov 20, 2026. Measured daily since Jul 28, 2026; GitHub no longer exposes earlier star timestamps.

Update history

3 recorded
  • Nov 13, 2026New release v0.6.8 (previously v0.5.23).
  • Oct 30, 2026New release v0.5.23 (previously v0.5.20).
  • Oct 6, 2026New release v0.5.20 (previously v0.5.19).

Contribution activity

commits per day, last 52 weeks
NovDecJanFebMarAprMayJunJulAugSepOctNovMonWedFri2025-11-16: 0 commits2025-11-17: 0 commits2025-11-18: 0 commits2025-11-19: 0 commits2025-11-20: 0 commits2025-11-21: 0 commits2025-11-22: 0 commits2025-11-23: 0 commits2025-11-24: 0 commits2025-11-25: 0 commits2025-11-26: 0 commits2025-11-27: 0 commits2025-11-28: 0 commits2025-11-29: 0 commits2025-11-30: 0 commits2025-12-01: 0 commits2025-12-02: 0 commits2025-12-03: 0 commits2025-12-04: 0 commits2025-12-05: 0 commits2025-12-06: 0 commits2025-12-07: 0 commits2025-12-08: 0 commits2025-12-09: 0 commits2025-12-10: 0 commits2025-12-11: 0 commits2025-12-12: 0 commits2025-12-13: 0 commits2025-12-14: 0 commits2025-12-15: 0 commits2025-12-16: 0 commits2025-12-17: 0 commits2025-12-18: 0 commits2025-12-19: 0 commits2025-12-20: 0 commits2025-12-21: 0 commits2025-12-22: 0 commits2025-12-23: 0 commits2025-12-24: 0 commits2025-12-25: 0 commits2025-12-26: 0 commits2025-12-27: 0 commits2025-12-28: 0 commits2025-12-29: 0 commits2025-12-30: 0 commits2025-12-31: 0 commits2026-01-01: 0 commits2026-01-02: 0 commits2026-01-03: 0 commits2026-01-04: 0 commits2026-01-05: 0 commits2026-01-06: 0 commits2026-01-07: 0 commits2026-01-08: 0 commits2026-01-09: 0 commits2026-01-10: 0 commits2026-01-11: 0 commits2026-01-12: 0 commits2026-01-13: 0 commits2026-01-14: 0 commits2026-01-15: 0 commits2026-01-16: 0 commits2026-01-17: 0 commits2026-01-18: 0 commits2026-01-19: 0 commits2026-01-20: 0 commits2026-01-21: 0 commits2026-01-22: 0 commits2026-01-23: 0 commits2026-01-24: 0 commits2026-01-25: 0 commits2026-01-26: 0 commits2026-01-27: 0 commits2026-01-28: 0 commits2026-01-29: 0 commits2026-01-30: 0 commits2026-01-31: 0 commits2026-02-01: 0 commits2026-02-02: 0 commits2026-02-03: 0 commits2026-02-04: 0 commits2026-02-05: 0 commits2026-02-06: 0 commits2026-02-07: 0 commits2026-02-08: 0 commits2026-02-09: 0 commits2026-02-10: 0 commits2026-02-11: 0 commits2026-02-12: 0 commits2026-02-13: 0 commits2026-02-14: 0 commits2026-02-15: 0 commits2026-02-16: 0 commits2026-02-17: 0 commits2026-02-18: 0 commits2026-02-19: 0 commits2026-02-20: 0 commits2026-02-21: 0 commits2026-02-22: 0 commits2026-02-23: 0 commits2026-02-24: 0 commits2026-02-25: 0 commits2026-02-26: 0 commits2026-02-27: 0 commits2026-02-28: 0 commits2026-03-01: 0 commits2026-03-02: 0 commits2026-03-03: 0 commits2026-03-04: 9 commits2026-03-05: 44 commits2026-03-06: 0 commits2026-03-07: 0 commits2026-03-08: 0 commits2026-03-09: 27 commits2026-03-10: 1 commit2026-03-11: 0 commits2026-03-12: 1 commit2026-03-13: 0 commits2026-03-14: 0 commits2026-03-15: 0 commits2026-03-16: 0 commits2026-03-17: 0 commits2026-03-18: 0 commits2026-03-19: 0 commits2026-03-20: 0 commits2026-03-21: 0 commits2026-03-22: 0 commits2026-03-23: 0 commits2026-03-24: 0 commits2026-03-25: 0 commits2026-03-26: 0 commits2026-03-27: 0 commits2026-03-28: 0 commits2026-03-29: 0 commits2026-03-30: 0 commits2026-03-31: 0 commits2026-04-01: 0 commits2026-04-02: 0 commits2026-04-03: 0 commits2026-04-04: 0 commits2026-04-05: 0 commits2026-04-06: 0 commits2026-04-07: 0 commits2026-04-08: 0 commits2026-04-09: 0 commits2026-04-10: 0 commits2026-04-11: 0 commits2026-04-12: 0 commits2026-04-13: 0 commits2026-04-14: 0 commits2026-04-15: 0 commits2026-04-16: 0 commits2026-04-17: 0 commits2026-04-18: 0 commits2026-04-19: 0 commits2026-04-20: 0 commits2026-04-21: 0 commits2026-04-22: 0 commits2026-04-23: 0 commits2026-04-24: 0 commits2026-04-25: 0 commits2026-04-26: 0 commits2026-04-27: 0 commits2026-04-28: 0 commits2026-04-29: 0 commits2026-04-30: 0 commits2026-05-01: 0 commits2026-05-02: 0 commits2026-05-03: 0 commits2026-05-04: 0 commits2026-05-05: 0 commits2026-05-06: 0 commits2026-05-07: 0 commits2026-05-08: 0 commits2026-05-09: 0 commits2026-05-10: 0 commits2026-05-11: 0 commits2026-05-12: 0 commits2026-05-13: 0 commits2026-05-14: 9 commits2026-05-15: 16 commits2026-05-16: 12 commits2026-05-17: 23 commits2026-05-18: 9 commits2026-05-19: 2 commits2026-05-20: 5 commits2026-05-21: 2 commits2026-05-22: 3 commits2026-05-23: 0 commits2026-05-24: 1 commit2026-05-25: 0 commits2026-05-26: 0 commits2026-05-27: 0 commits2026-05-28: 0 commits2026-05-29: 0 commits2026-05-30: 0 commits2026-05-31: 0 commits2026-06-01: 0 commits2026-06-02: 0 commits2026-06-03: 5 commits2026-06-04: 0 commits2026-06-05: 4 commits2026-06-06: 0 commits2026-06-07: 0 commits2026-06-08: 1 commit2026-06-09: 2 commits2026-06-10: 5 commits2026-06-11: 2 commits2026-06-12: 0 commits2026-06-13: 0 commits2026-06-14: 2 commits2026-06-15: 0 commits2026-06-16: 0 commits2026-06-17: 2 commits2026-06-18: 8 commits2026-06-19: 0 commits2026-06-20: 1 commit2026-06-21: 0 commits2026-06-22: 0 commits2026-06-23: 2 commits2026-06-24: 3 commits2026-06-25: 1 commit2026-06-26: 0 commits2026-06-27: 0 commits2026-06-28: 1 commit2026-06-29: 3 commits2026-06-30: 0 commits2026-07-01: 0 commits2026-07-02: 0 commits2026-07-03: 5 commits2026-07-04: 0 commits2026-07-05: 0 commits2026-07-06: 0 commits2026-07-07: 0 commits2026-07-08: 1 commit2026-07-09: 1 commit2026-07-10: 0 commits2026-07-11: 0 commits2026-07-12: 0 commits2026-07-13: 0 commits2026-07-14: 0 commits2026-07-15: 0 commits2026-07-16: 1 commit2026-07-17: 0 commits2026-07-18: 0 commits2026-07-19: 0 commits2026-07-20: 0 commits2026-07-21: 0 commits2026-07-22: 0 commits2026-07-23: 0 commits2026-07-24: 0 commits2026-07-25: 0 commits2026-07-26: 0 commits2026-07-27: 0 commits2026-07-28: 0 commits2026-07-29: 0 commits2026-07-30: 0 commits2026-07-31: 0 commits2026-08-01: 0 commits2026-08-02: 0 commits2026-08-03: 0 commits2026-08-04: 0 commits2026-08-05: 2 commits2026-08-06: 0 commits2026-08-07: 0 commits2026-08-08: 0 commits2026-08-09: 1 commit2026-08-10: 0 commits2026-08-11: 0 commits2026-08-12: 0 commits2026-08-13: 0 commits2026-08-14: 5 commits2026-08-15: 1 commit2026-08-16: 0 commits2026-08-17: 0 commits2026-08-18: 0 commits2026-08-19: 0 commits2026-08-20: 0 commits2026-08-21: 0 commits2026-08-22: 0 commits2026-08-23: 0 commits2026-08-24: 0 commits2026-08-25: 1 commit2026-08-26: 0 commits2026-08-27: 0 commits2026-08-28: 0 commits2026-08-29: 0 commits2026-08-30: 0 commits2026-08-31: 0 commits2026-09-01: 0 commits2026-09-02: 0 commits2026-09-03: 0 commits2026-09-04: 0 commits2026-09-05: 0 commits2026-09-06: 0 commits2026-09-07: 0 commits2026-09-08: 0 commits2026-09-09: 0 commits2026-09-10: 0 commits2026-09-11: 0 commits2026-09-12: 0 commits2026-09-13: 0 commits2026-09-14: 4 commits2026-09-15: 0 commits2026-09-16: 1 commit2026-09-17: 1 commit2026-09-18: 0 commits2026-09-19: 9 commits2026-09-20: 3 commits2026-09-21: 0 commits2026-09-22: 0 commits2026-09-23: 0 commits2026-09-24: 0 commits2026-09-25: 0 commits2026-09-26: 0 commits2026-09-27: 0 commits2026-09-28: 0 commits2026-09-29: 0 commits2026-09-30: 0 commits2026-10-01: 0 commits2026-10-02: 0 commits2026-10-03: 9 commits2026-10-04: 0 commits2026-10-05: 0 commits2026-10-06: 0 commits2026-10-07: 0 commits2026-10-08: 0 commits2026-10-09: 0 commits2026-10-10: 0 commits2026-10-11: 0 commits2026-10-12: 0 commits2026-10-13: 0 commits2026-10-14: 0 commits2026-10-15: 0 commits2026-10-16: 0 commits2026-10-17: 0 commits2026-10-18: 0 commits2026-10-19: 2 commits2026-10-20: 0 commits2026-10-21: 0 commits2026-10-22: 0 commits2026-10-23: 0 commits2026-10-24: 0 commits2026-10-25: 0 commits2026-10-26: 0 commits2026-10-27: 0 commits2026-10-28: 4 commits2026-10-29: 28 commits2026-10-30: 6 commits2026-10-31: 15 commits2026-11-01: 11 commits2026-11-02: 8 commits2026-11-03: 3 commits2026-11-04: 17 commits2026-11-05: 3 commits2026-11-06: 0 commits2026-11-07: 0 commits2026-11-08: 0 commits2026-11-09: 0 commits2026-11-10: 2 commits2026-11-11: 3 commits2026-11-12: 5 commits2026-11-13: 3 commits2026-11-14: 0 commits
361 commits in the last yearLessMore

Signals and awards

derived from tracked data
  • Actively maintained

    Pushed within 48 hours

  • Well documented

    High community health score

  • Permissive license

    MIT

  • Continuous integration

    Automated checks passing

What whichllm does

whichllm is a Python command-line utility explicitly designed to eliminate the guesswork involved in selecting local Large Language Models. It actively profiles your specific machine's hardware, detecting exact GPU VRAM, system RAM bandwidth, and CPU capabilities. It then queries live HuggingFace databases and ranks the best available models based on constraints like partial RAM offloading and strict VRAM limits, entirely bypassing simplistic parameter count metrics. The tool also features powerful hardware simulation capabilities, allowing users to test hypothetical multi-GPU configurations before committing to expensive hardware purchases.

AI engineers, researchers, and hobbyists who run models locally and require an accurate, automated way to match HuggingFace models to strict hardware limits. Familiarity with Python is recommended.

  • Hardware profiling: Instantly and accurately identifies your machine's exact available GPU VRAM, system RAM bandwidth, and unified memory specs.
  • Recency-aware benchmarking: Ranks HuggingFace models using the most up-to-date performance metrics to prioritize truly usable inference speed.
  • Accurate partial offload calculation: Estimates the viability of running massive models by mathematically modeling partial layer offloads to system RAM.
  • Advanced hardware simulation: Allows users to override detected specifications with flags to accurately test hypothetical multi-GPU setups.
  • Configurable VRAM headroom: Lets researchers specify strict memory buffers to prevent out-of-memory errors during intensive context scaling.
  • Evidence-graded scoring: Rejects fabricated uploader claims and highly penalizes cross-family score inheritance to ensure true benchmark integrity.

Where teams use it

Local Inference Optimization

Developers setting up a local AI environment execute the tool to instantly identify the most capable model their current hardware can reliably support.

Hardware Purchase Planning

AI enthusiasts rigorously simulate running specific multi-GPU configurations to see exactly which models they will be able to run before buying expensive hardware.

Safe Model Selection

Researchers enable strict VRAM headroom flags to guarantee the specifically selected model will run stably without catastrophic out-of-memory crashes.

Automated Pipeline Integration

Engineers leverage the tool's JSON output directly in bash scripts to dynamically map the best HuggingFace IDs to their local Ollama inference configurations.

Getting started: uvx whichllm@latest

README

main branch

whichllm

PyPI version Python 3.11+ License: MIT Tests Sponsor

Andyyyy64%2Fwhichllm | Trendshift

The smartest model your PC can actually run, including ones you didn't know it could.

whichllm reads your GPU, RAM and memory bandwidth, works out which open models fit and how fast each one will generate, and ranks them by benchmark results. Then it prints the command that runs the top pick the way it was estimated.

日本語版はこちら

Quick start

uvx whichllm@latest

Shopping for a GPU? Simulate one:

uvx whichllm@latest --gpu "RTX 4090"

To keep it around:

uv tool install whichllm
brew install andyyyy64/whichllm/whichllm
pip install whichllm

Example

A desktop with an 8 GB RTX 4060 and 32 GB of DDR4 (output trimmed):

$ whichllm

 #  Model                         Quant   Fit       Speed     Score
 1  Qwen/Qwen3.6-35B-A3B          Q6_K    GPU+RAM   16 tok/s   68.8
 2  unsloth/gemma-4-12b-it-GGUF   IQ4_XS  Full GPU  37 tok/s   68.1
 3  google/gemma-4-26B-A4B-it     Q4_K_M  GPU+RAM   11 tok/s   64.5
 4  Qwen/Qwen3.5-9B               Q5_K_M  Full GPU  36 tok/s   64.2
 5  zai-org/GLM-4.7-Flash         Q5_K_M  GPU+RAM   15 tok/s   62.3

  #1 Qwen3.6-35B-A3B also stands in for older generations of its line:
  Qwen3.5-35B-A3B, Qwen3-30B-A3B, Qwen_Qwen3-30B-A3B-Instruct-2507-GGUF

  Run #1 with llama.cpp:
    llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:Q6_K -c 4096 -ngl 99 --n-cpu-moe 35 -fa on

The top pick is a 35B model on an 8 GB card. It is a mixture-of-experts model, so each token only uses about 3B of its parameters. whichllm keeps attention, the shared weights and the KV cache in VRAM and leaves the experts of 35 of its 40 layers in system RAM, which is what llama.cpp's --n-cpu-moe does. The speed estimate counts only what each token reads from each place. A fits-in-VRAM check would have skipped the model, and treating it as a plain partial offload would have rated it far too slow.

On a 24 GB RTX 4090 the same logic picks Qwen3.8-27B, although Qwen3-32B also fits there at Q4_K_M. The 27B is the newer generation of the same model line at about the same size and scores higher on current benchmarks, so the 32B is not listed separately; the note under the table names it.

Speeds are estimates from memory bandwidth, not measurements of your machine. The table marks how confident each estimate is.

Here is one checked on real hardware. On a 16 GB RTX 5080 with DDR5-5600, whichllm estimates 70 tok/s for Qwen3.6-35B-A3B at Q4_K_M, with this command:

llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M -c 4096 -ngl 99 --n-cpu-moe 15 -fa on

It measured 79 tok/s on llama.cpp b11977, from a 22 GB file on a 16 GB card. whichllm bench runs the same comparison on your machine.

How it ranks

Quality comes from Artificial Analysis and LiveBench, put on one scale and averaged. --profile coding, math and vision rank on benchmarks for that task instead, such as SciCode and LiveCodeBench for coding. The Open LLM Leaderboard and Chatbot Arena are frozen, so they only score models nothing current covers, and older generations are demoted. Each match records how it was found (the exact repo, its base model, a same-family interpolation, or the uploader's own claim) and weaker matches count for less. whichllm never recommends pretrain-only base checkpoints.

To check the fit, memory is counted as weights plus a KV cache computed layer by layer from the model's config.json (grouped-query and latent attention, sliding windows, recurrent layers), plus runtime buffers. When a model does not fit in VRAM, a mixture-of-experts model first moves experts to RAM; any other model spills whole layers.

Speed is bound by memory bandwidth: the GPU reads its share of the weights at VRAM speed and the CPU reads the rest at RAM speed. Both sides are calibrated against llama.cpp runs. Every placement is held to 8 tok/s: below that, a model's quality is scaled down, so a stronger model that would answer at 3 tok/s does not outrank one that answers at 25.

The details are in Scoring and How it works.

How we test recommendations

A wrong recommendation is a bug, so the rankings have regression tests: tests/golden/.

  • 29 hardware archetypes, from a 4 GB GTX 1650 to a 128 GB M4 Max and a DGX Spark, with AMD, Intel and mixed GPU sets, CPU-only boxes and 32K/128K-context setups, ranked against a frozen copy of the index: 1,430 models, most GGUF repos with their real file sizes.
  • 30 constraints describe what a good answer looks like instead of pinning an exact order: no pretrain-only checkpoints, draft heads or diffusion models, no outdated generation in the top 5, the top 3 stay within 1 tok/s of the 8 tok/s bar whenever the machine has that many options that fast, full-GPU rows keep their VRAM headroom and stay inside Metal's working set, run can launch the top rows, the newest dense 27B makes the top 3 on 24 GB+, a big MoE with experts in RAM makes the top 5 on a 16 GB card with 64 GB of RAM, and coding picks are not limited to models named "coder".
  • Speed and memory estimates stay within set bounds of real runs: an RTX 5080 and its CPU, plus 379 published llama.cpp results on 194 GPUs and 255 LocalScore results on 138 CPUs in tests/data/. The estimator cannot drift without a test failing.
  • The top picks for every archetype are committed in top_picks.md, so any ranking change shows up as a diff in review.
  • A constraint that is known to fail is recorded with its reason as a strict expected failure. Fixing the bug turns it into a test failure until the record is removed.

If whichllm picks something you would not run on your hardware, open an issue with the output. The 6 GB and RTX 3060 archetypes started as reports like that (#76, #104).

Commands

whichllm                          # best models for this machine
whichllm --gpu "RTX 5060 Ti"      # simulate a GPU (add --vram 16 for a variant)
whichllm --gpu "2x RTX 3090"      # or several
whichllm --profile coding         # general, coding, vision, math, any
whichllm --context-length 64k     # long context changes what fits
whichllm --gpu-only               # only models that fit entirely in VRAM
whichllm --speed usable           # hide anything under 8 tok/s
whichllm --markdown               # pasteable table for issues and chats
whichllm --json                   # rows include placement and the run recipe

whichllm hardware                 # what was detected
whichllm engines                  # which inference engines run here
whichllm plan "qwen3 32b"         # what GPU a model needs
whichllm upgrade "RTX 4090" "RTX 5090"   # compare against candidate GPUs
whichllm run                      # download the top pick and chat with it
whichllm bench                    # measure the top pick on this machine
whichllm report                   # share that measurement in a GitHub issue
whichllm snippet                  # print Python that loads the top pick

Every option is in the CLI reference.

If LM Studio complains that a pick is slightly too large, ask for a more conservative fit:

whichllm --gpu-only --speed usable --vram-headroom 1.5GB

If an integrated GPU or unified memory is detected with the wrong limits, override them with --vram and --ram-bandwidth (see Hardware).

Run and snippet

whichllm run downloads the model and opens a chat. Without a model name it takes the top pick.

run demo

whichllm run
whichllm run "qwen 2.5 1.5b gguf"
whichllm run "phi 3 mini gguf" --cpu-only

With llama.cpp installed (brew install llama.cpp, winget install ggml.llamacpp), a GGUF model runs in its llama-cli with the flags of the printed recipe, so the chat uses the GPU and keeps MoE experts in RAM the way the ranking estimated. A named model is left to llama.cpp's automatic fit.

A GGUF model already installed in LM Studio is opened from that library instead of being downloaded again; whichllm recommend marks those rows Installed. Pass --lm-studio-path (repeatable) for a legacy or custom library:

whichllm run "qwen 2.5 1.5b gguf" --lm-studio-path /mnt/models/lm-studio

Without llama-cli, run creates a throwaway environment with uv and uses llama-cpp-python for GGUF models and transformers for the rest. PyPI only ships llama-cpp-python as source, so the first run compiles it, which took 22 minutes on a Core Ultra 9, and that build runs on the CPU only. See Run and snippet.

Where the data comes from

whichllm reads a prebuilt index: the Hugging Face model list, each family's config.json and the merged benchmark scores, rebuilt daily on GitHub Actions and published only if it passes quality checks. A cold start takes about a second and makes no Hugging Face API calls. --live fetches everything directly instead, and HF_TOKEN is used when set.

Documentation

Development

git clone https://github.com/Andyyyy64/whichllm.git
cd whichllm
uv sync --locked --dev
uv run whichllm
uv run pytest

Ranking changes need a look at the golden snapshot: uv run python scripts/update_golden.py report. See CONTRIBUTING.md.

Support

If whichllm saved you a bad download or a bad hardware purchase, sponsoring helps keep it maintained: hardware reports, benchmark updates, packaging and support for more machines. It stays open source either way.

A star helps other people find it. I'd also like to hear what it picked for your machine, so feel free to drop it in Issues.

Star History

Star History Chart

License

MIT

View on GitHub

Recent activity

commits and pull requests

Recent open issues

view all

Discussions

all 4

Releases and announcements

32 total
  1. v0.6.8v0.6.8Nov 13, 2026

    # whichllm 0.6.8 vLLM and SGLang are in. A row of a repository's own Hugging Face weights, which whichllm ranks on Linux with an NVIDIA GPU and under WSL, now names the server that runs it and the command. That holds when the weights split over several GPUs too, and when an MoE pages its experts in from RAM. ## vLLM and SGLang AWQ, GPTQ, FP8, NVFP4, MXFP4, INT8 and BF16 repos had no engine and no run line before. Now such a row names vLLM with `vllm serve <repo> --max-model-len auto`, or SGLang with `sglang serve <repo>` when SGLang is the one running or the only one installed. The cards are written from vLLM 0.33.0 and SGLang 0.5.23. On NVIDIA both need Linux, compute capability 7.5 or newer and a driver for CUDA 13, and both run under WSL. (#310) These rows used to take llama.cpp's fit and speed, with MoE experts or whole layers in RAM, and neither server runs them that way. Their RAM offload streams the weights on every token, 2.9 to 16.6 tok/s on a 16 GB card in SGLang's own measurements. So a row of Hugging Face weights now shows only where vLLM or SGLang hold it, and the model's GGUF covers the rest. ## Several GPUs Weights too big for one GPU now split evenly over 2, 4

  2. v0.6.7v0.6.7Nov 12, 2026

    # whichllm 0.6.7 A fix for the pack whichllm names under a model. On some PCs it named a pack built for a PC one size bigger, with the speed measured on that bigger PC. ## What happened On my PC, an RTX 5080 with 16 GB and 64 GB of RAM, 0.6.6 named Abyss's 24G64G pack under Qwen3.8-Flash-Next, with the 69.9 tok/s its docs measured on a 24 GB GPU. `whichllm engines` counted 8 of Abyss's 15 packs where 6 fit. It only showed up once the disk had room for the bigger pack. Two things added up. - Abyss's card gave each pack's memory as the GPU plus the memory limit the author tests with, 58 GB for a 64 GB PC. whichllm compares that with all of a PC's RAM, so the PC started 5 GB ahead. - The 4% allowance from 0.6.5, for a PC that reports under the size it is sold as, was taken off what a pack needs. That also shrank needs that are measured figures and no PC size. The PC reports 15.9 + 63.3 = 79.2 GB, and the pack's 82 GB less 4% is 78.7. ## What changed - The allowance is now on the machine's side. RAM or VRAM within 4% under a size PCs are sold with counts as that size, so 63.3 GB is 64 and 15.9 is 16. A pack's need is compared as written. - Abyss's packs give the GPU and RAM of

  3. v0.6.6v0.6.6Nov 11, 2026

    # whichllm 0.6.6 A fix for a crash at start. If you installed or upgraded whichllm since November 9 and it stopped with the error below, this release is for you. ``` TypeError: Option() got an unexpected keyword argument 'click_type' ``` ## What happened typer, the library whichllm's command line is built on, released 0.29.0 on November 9 and removed an argument that `--context-length` used. whichllm asked for `typer>=0.9` with no upper bound, so a fresh `pip install whichllm` or `uvx whichllm` got the new typer and failed at import. That hit every whichllm release, not only the newest. Homebrew installs pin their dependencies and kept working, and so did any install from before November 9. I found it on November 12, running 0.6.5 on Windows from a clean environment. Nobody had reported it. My tests run against locked dependencies, with an older typer, so they kept passing. ## What changed - `--context-length` no longer uses the removed argument. It behaves as before. `64k` is 65536, and a bad value says why. - whichllm now asks for typer 0.15 or newer. typer 0.9, the old lower bound, doesn't work with current click anyway. - The tests now also run against the newest release

  4. v0.6.5v0.6.5Nov 10, 2026

    # whichllm 0.6.5 This release adds Abyss to the engines whichllm knows, stops refusing PCs that are exactly the size a pack asks for, and brings seven engine cards up to date. ## Abyss [Abyss](https://github.com/zephel01/Abyss) is a fork of Project Maya that runs DeepSeek V4 and V4.1 Flash, GLM-5.3-Flash, Qwen3.8-Flash-Next, Ornith-1.5 and Qwen3.5-122B-A10B on one GPU, with the experts spread over VRAM, RAM and the SSD. Its card lists the 15 packs its setup installs with one command. The author's packs are named for the PC they target, such as 16G64G for a 16 GB GPU with 64 GB of RAM, and each one fits in memory on a machine of that size. The two DeepSeek V4.1 packs are the exception, since that model's experts don't fit a 32 GB GPU with 96 GB of RAM and some are read from the SSD. On a 16 GB GPU with 64 GB of RAM, Abyss shows up under Qwen3.8-Flash-Next. ``` #2 also runs with Abyss (zephel01 IQ3_M 16G64G, 70.7 tok/s in its docs on RTX 5090 limited to 16 GB, 58 GB memory limit, with its MTP drafter): ./setup-deepseek.sh --model qwen38 ``` On a 12 GB GPU with 32 GB, where llama.cpp can't place the model at all, Abyss's 12G32G pack puts it in the ranking as a row of its o

  5. v0.6.4v0.6.4Nov 4, 2026

    # whichllm 0.6.4 A patch for 0.6.3. Once the index was rebuilt with 0.6.3's lookup-table fix, I ran whichllm on my own PC (RTX 5080 16 GB, 39 GB of RAM under WSL), and Qwen3.8-Flash-Next still came out wrong. It sat 11th through an MLX repo ranked as FP16 safetensors, which nothing on that machine can run, and none of its GGUFs showed up. This release fixes why. ## Qwen3.8-Flash-Next ranks with the file that fits On that PC the one build that fits is ISTA-DASLab's IQ3_XXS, which Strata's measurement puts on par with IQ4_XS. Two things kept it out. whichllm leaves out types below 3 bits unless you ask for one with `--quant`, and it checked the file's own type, not the IQ4_XS it's rated as. And the official repo's row only offers Q3_K_M to Q8_0, so even as a candidate the file could only rank as ISTA-DASLab's own repo, whose benchmark evidence is inherited and counts for less. Now the rated type decides, and the official row can take an upload's measured file as that exact file, with its rating and its recipe. This is that PC now. ``` │ 1 │ Qwen/Qwen3.8-27B │ Q3_K_M │ 12.5 GB │ Full GPU │ 59.0 tok/s │ 256k │ 2026-08-05 │ 86.2 │ │ │ 27.8B

Code frequency

additions and deletions
+20.6K-20.6KWeek of 2026-03-01: +13,395 linesWeek of 2026-03-01: -7,726 linesWeek of 2026-03-08: +2,243 linesWeek of 2026-03-08: -443 linesWeek of 2026-03-15: +0 linesWeek of 2026-03-15: -0 linesWeek of 2026-03-22: +0 linesWeek of 2026-03-22: -0 linesWeek of 2026-03-29: +0 linesWeek of 2026-03-29: -0 linesWeek of 2026-04-05: +0 linesWeek of 2026-04-05: -0 linesWeek of 2026-04-12: +0 linesWeek of 2026-04-12: -0 linesWeek of 2026-04-19: +0 linesWeek of 2026-04-19: -0 linesWeek of 2026-04-26: +0 linesWeek of 2026-04-26: -0 linesWeek of 2026-05-03: +0 linesWeek of 2026-05-03: -0 linesWeek of 2026-05-10: +4,720 linesWeek of 2026-05-10: -674 linesWeek of 2026-05-17: +4,756 linesWeek of 2026-05-17: -1,007 linesWeek of 2026-05-24: +2 linesWeek of 2026-05-24: -0 linesWeek of 2026-05-31: +1,402 linesWeek of 2026-05-31: -478 linesWeek of 2026-06-07: +1,520 linesWeek of 2026-06-07: -126 linesWeek of 2026-06-14: +3,389 linesWeek of 2026-06-14: -1,069 linesWeek of 2026-06-21: +835 linesWeek of 2026-06-21: -38 linesWeek of 2026-06-28: +2,359 linesWeek of 2026-06-28: -1,303 linesWeek of 2026-07-05: +1,629 linesWeek of 2026-07-05: -1,473 linesWeek of 2026-07-12: +7 linesWeek of 2026-07-12: -1 linesWeek of 2026-07-19: +0 linesWeek of 2026-07-19: -0 linesWeek of 2026-07-26: +0 linesWeek of 2026-07-26: -0 linesWeek of 2026-08-02: +130 linesWeek of 2026-08-02: -40 linesWeek of 2026-08-09: +2,218 linesWeek of 2026-08-09: -1,288 linesWeek of 2026-08-16: +0 linesWeek of 2026-08-16: -0 linesWeek of 2026-08-23: +639 linesWeek of 2026-08-23: -4 linesWeek of 2026-08-30: +0 linesWeek of 2026-08-30: -0 linesWeek of 2026-09-06: +0 linesWeek of 2026-09-06: -0 linesWeek of 2026-09-13: +1,099 linesWeek of 2026-09-13: -253 linesWeek of 2026-09-20: +491 linesWeek of 2026-09-20: -14 linesWeek of 2026-09-27: +1,526 linesWeek of 2026-09-27: -1,137 linesWeek of 2026-10-04: +0 linesWeek of 2026-10-04: -0 linesWeek of 2026-10-11: +0 linesWeek of 2026-10-11: -0 linesWeek of 2026-10-18: +25 linesWeek of 2026-10-18: -7 linesWeek of 2026-10-25: +20,644 linesWeek of 2026-10-25: -3,963 linesWeek of 2026-11-01: +12,595 linesWeek of 2026-11-01: -754 linesWeek of 2026-11-08: +2,103 linesWeek of 2026-11-08: -240 linesMar 1, 2026Nov 8, 2026
+77.7K lines added, -22K removed over the last year.

Commits per week

last 52 weeks
530Week of 2025-11-16: 0 commitsWeek of 2025-11-23: 0 commitsWeek of 2025-11-30: 0 commitsWeek of 2025-12-07: 0 commitsWeek of 2025-12-14: 0 commitsWeek of 2025-12-21: 0 commitsWeek of 2025-12-28: 0 commitsWeek of 2026-01-04: 0 commitsWeek of 2026-01-11: 0 commitsWeek of 2026-01-18: 0 commitsWeek of 2026-01-25: 0 commitsWeek of 2026-02-01: 0 commitsWeek of 2026-02-08: 0 commitsWeek of 2026-02-15: 0 commitsWeek of 2026-02-22: 0 commitsWeek of 2026-03-01: 53 commitsWeek of 2026-03-08: 29 commitsWeek of 2026-03-15: 0 commitsWeek of 2026-03-22: 0 commitsWeek of 2026-03-29: 0 commitsWeek of 2026-04-05: 0 commitsWeek of 2026-04-12: 0 commitsWeek of 2026-04-19: 0 commitsWeek of 2026-04-26: 0 commitsWeek of 2026-05-03: 0 commitsWeek of 2026-05-10: 37 commitsWeek of 2026-05-17: 44 commitsWeek of 2026-05-24: 1 commitsWeek of 2026-05-31: 9 commitsWeek of 2026-06-07: 10 commitsWeek of 2026-06-14: 13 commitsWeek of 2026-06-21: 6 commitsWeek of 2026-06-28: 9 commitsWeek of 2026-07-05: 2 commitsWeek of 2026-07-12: 1 commitsWeek of 2026-07-19: 0 commitsWeek of 2026-07-26: 0 commitsWeek of 2026-08-02: 2 commitsWeek of 2026-08-09: 7 commitsWeek of 2026-08-16: 0 commitsWeek of 2026-08-23: 1 commitsWeek of 2026-08-30: 0 commitsWeek of 2026-09-06: 0 commitsWeek of 2026-09-13: 15 commitsWeek of 2026-09-20: 3 commitsWeek of 2026-09-27: 9 commitsWeek of 2026-10-04: 0 commitsWeek of 2026-10-11: 0 commitsWeek of 2026-10-18: 2 commitsWeek of 2026-10-25: 53 commitsWeek of 2026-11-01: 42 commitsWeek of 2026-11-08: 13 commitsNov 16, 2025Nov 8, 2026
361 commits in the last 52 weeks.

When work happens

weekday and hour
SunMonTueWedThuFriSat036912151821Sun 0:00 — 1 commitsSun 1:00 — 2 commitsSun 2:00 — 3 commitsSun 3:00 — 2 commitsSun 4:00 — 1 commitsSun 5:00 — 9 commitsSun 6:00 — 4 commitsSun 7:00 — 2 commitsSun 8:00 — 0 commitsSun 9:00 — 0 commitsSun 10:00 — 2 commitsSun 11:00 — 0 commitsSun 12:00 — 0 commitsSun 13:00 — 0 commitsSun 14:00 — 0 commitsSun 15:00 — 1 commitsSun 16:00 — 2 commitsSun 17:00 — 2 commitsSun 18:00 — 1 commitsSun 19:00 — 1 commitsSun 20:00 — 3 commitsSun 21:00 — 4 commitsSun 22:00 — 1 commitsSun 23:00 — 1 commitsMon 0:00 — 1 commitsMon 1:00 — 0 commitsMon 2:00 — 0 commitsMon 3:00 — 7 commitsMon 4:00 — 1 commitsMon 5:00 — 0 commitsMon 6:00 — 0 commitsMon 7:00 — 0 commitsMon 8:00 — 0 commitsMon 9:00 — 1 commitsMon 10:00 — 0 commitsMon 11:00 — 1 commitsMon 12:00 — 0 commitsMon 13:00 — 3 commitsMon 14:00 — 0 commitsMon 15:00 — 13 commitsMon 16:00 — 6 commitsMon 17:00 — 0 commitsMon 18:00 — 6 commitsMon 19:00 — 7 commitsMon 20:00 — 4 commitsMon 21:00 — 1 commitsMon 22:00 — 1 commitsMon 23:00 — 2 commitsTue 0:00 — 2 commitsTue 1:00 — 0 commitsTue 2:00 — 0 commitsTue 3:00 — 0 commitsTue 4:00 — 0 commitsTue 5:00 — 0 commitsTue 6:00 — 1 commitsTue 7:00 — 0 commitsTue 8:00 — 2 commitsTue 9:00 — 0 commitsTue 10:00 — 0 commitsTue 11:00 — 1 commitsTue 12:00 — 1 commitsTue 13:00 — 0 commitsTue 14:00 — 1 commitsTue 15:00 — 1 commitsTue 16:00 — 0 commitsTue 17:00 — 2 commitsTue 18:00 — 0 commitsTue 19:00 — 0 commitsTue 20:00 — 1 commitsTue 21:00 — 0 commitsTue 22:00 — 0 commitsTue 23:00 — 1 commitsWed 0:00 — 5 commitsWed 1:00 — 0 commitsWed 2:00 — 1 commitsWed 3:00 — 5 commitsWed 4:00 — 5 commitsWed 5:00 — 1 commitsWed 6:00 — 0 commitsWed 7:00 — 2 commitsWed 8:00 — 0 commitsWed 9:00 — 0 commitsWed 10:00 — 1 commitsWed 11:00 — 0 commitsWed 12:00 — 2 commitsWed 13:00 — 0 commitsWed 14:00 — 4 commitsWed 15:00 — 0 commitsWed 16:00 — 2 commitsWed 17:00 — 1 commitsWed 18:00 — 0 commitsWed 19:00 — 5 commitsWed 20:00 — 4 commitsWed 21:00 — 2 commitsWed 22:00 — 2 commitsWed 23:00 — 15 commitsThu 0:00 — 3 commitsThu 1:00 — 12 commitsThu 2:00 — 4 commitsThu 3:00 — 8 commitsThu 4:00 — 3 commitsThu 5:00 — 0 commitsThu 6:00 — 0 commitsThu 7:00 — 9 commitsThu 8:00 — 0 commitsThu 9:00 — 0 commitsThu 10:00 — 2 commitsThu 11:00 — 0 commitsThu 12:00 — 1 commitsThu 13:00 — 6 commitsThu 14:00 — 1 commitsThu 15:00 — 6 commitsThu 16:00 — 11 commitsThu 17:00 — 11 commitsThu 18:00 — 6 commitsThu 19:00 — 0 commitsThu 20:00 — 4 commitsThu 21:00 — 16 commitsThu 22:00 — 2 commitsThu 23:00 — 1 commitsFri 0:00 — 1 commitsFri 1:00 — 2 commitsFri 2:00 — 4 commitsFri 3:00 — 0 commitsFri 4:00 — 0 commitsFri 5:00 — 0 commitsFri 6:00 — 0 commitsFri 7:00 — 0 commitsFri 8:00 — 0 commitsFri 9:00 — 0 commitsFri 10:00 — 0 commitsFri 11:00 — 2 commitsFri 12:00 — 2 commitsFri 13:00 — 2 commitsFri 14:00 — 1 commitsFri 15:00 — 5 commitsFri 16:00 — 9 commitsFri 17:00 — 3 commitsFri 18:00 — 1 commitsFri 19:00 — 4 commitsFri 20:00 — 0 commitsFri 21:00 — 1 commitsFri 22:00 — 2 commitsFri 23:00 — 3 commitsSat 0:00 — 2 commitsSat 1:00 — 4 commitsSat 2:00 — 0 commitsSat 3:00 — 0 commitsSat 4:00 — 0 commitsSat 5:00 — 0 commitsSat 6:00 — 1 commitsSat 7:00 — 1 commitsSat 8:00 — 0 commitsSat 9:00 — 0 commitsSat 10:00 — 1 commitsSat 11:00 — 0 commitsSat 12:00 — 0 commitsSat 13:00 — 1 commitsSat 14:00 — 1 commitsSat 15:00 — 4 commitsSat 16:00 — 3 commitsSat 17:00 — 2 commitsSat 18:00 — 2 commitsSat 19:00 — 3 commitsSat 20:00 — 7 commitsSat 21:00 — 6 commitsSat 22:00 — 5 commitsSat 23:00 — 4 commits
Commit volume by weekday and hour (UTC). Larger dots mean more commits.

Who is committing

last 52 weeks
Maintainer commits377 (83%)
Community commits78 (17%)

455 commits in total over the last year.

DateListRankStars gained
Jun 9, 2026daily#23+17