Trending repositories: inference-server

1 tracked repository tagged with inference-server, ordered by stars. Use the topic filters below to narrow further.

Filter by topic

1 of 1 repository

  • jundot/omlx

    LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

    AI summary: A highly optimized LLM inference server for Apple Silicon, controllable directly from the macOS menu bar.

    18,510ai-mlPythonApache-2.0