OpenLLM runs open-source language models as cloud API endpoints compatible with OpenAI clients.
Open source
21 alternativesRapid-MLX
For Apple Silicon, Rapid-MLX offers an open-source LLM inference server and Mac app compatible with OpenAI and Anthropic APIs.
About Rapid-MLX
Rapid-MLX is an Apache-2.0 licensed LLM inference server built on MLX for Apple Silicon, with a macOS desktop app. It provides OpenAI-compatible chat completions and responses endpoints and an Anthropic-compatible Messages endpoint, with tool-call parsing for coding agents.
It uses continuous batching, a radix prefix cache saved to disk on shutdown and restored at startup, and speculative decoding. In the project's stated test with Qwen3.5-9B 4-bit on a 48 GB M4 Pro Mac mini, decode results versus mlx-lm ranged from 1.28x to 4.27x with a 1.50x median, and one long-prompt agent turn was 1.05x.
Key features
- OpenAI-compatible chat completions and responses endpoints
- Anthropic Messages compatibility at /v1/messages
- Tool-call parsing with 27 parser modules and auto-detect fallback
- Continuous batching for concurrent requests
- Radix prefix cache saved to disk and restored on startup
- macOS desktop app for local inference
Best for: Developers running coding agents locally on Apple Silicon Macs who need OpenAI- or Anthropic-compatible endpoints.
Good for
- Apache-2.0 license
- Supports both OpenAI and Anthropic API formats
- Publishes per-task results, method and raw data for speed tests
Watch out for
- Requires Apple Silicon Macs
- Reported speed gain varies by task, including 1.05x on one long-prompt turn
Rapid-MLX at a glance
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Not stated
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
Rapid-MLX alternatives
Tools and sites you could use instead, ranked by how closely they match.
NobodyWho runs language models locally through an inference engine designed for efficient operation across devices.
Open source
22 alternatives
GPUStack manages GPU clusters for serving AI models and provides on demand GPU instances accessible through SSH.
Open source
21 alternatives
LLMKube runs self hosted LLM inference across heterogeneous GPU fleets using Kubernetes and several model runtimes.
Open source
22 alternatives
PowerInfer supports serving large language models for local deployment and inference.
Open source
23 alternatives
NVIDIA Triton Inference Server provides an inference solution for deploying models in cloud and edge environments.
Open source
23 alternativesPopular in Model Serving & Inference
Well-known listings from the same topic.
MAI Playgroundplayground.microsoft.ai/chatBigModelbigmodel.cnkimi-k2.6platform.kimi.ai/docs/pricing/chat-k26Jinagithub.com/jina-ai/jinaFluxgithub.com/black-forest-labs/fluxprivateGPTgithub.com/imartinez/privateGPT
Rapid-MLX FAQ
What hardware does Rapid-MLX require?
It is built on MLX for Apple Silicon; the cited speed test used a 48 GB M4 Pro Mac mini.
Which API endpoints are supported?
OpenAI /v1/chat/completions and /v1/responses, and Anthropic Messages at /v1/messages.
How was the speed comparison measured?
Rapid-MLX 0.15.6 against mlx-lm 0.32.0 on the same Qwen3.5-9B 4-bit weights with greedy decoding, both at defaults, measured 2026-10-04.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.