FedML supports distributed model training, model serving, and federated learning, with a scheduler for AI jobs across cloud and local clusters.
Open source
20 alternativesPowerInfer
PowerInfer supports serving large language models for local deployment and inference.
About PowerInfer
PowerInfer is an open source CPU/GPU inference engine for running large language models locally on a PC with a single consumer-grade GPU. It uses activation locality, supports Windows with GPU inference and AMD devices with ROCm, and includes PowerInfer-2 for smartphones. The online demo is described as experimental and without warranties.
Key features
- CPU/GPU inference engine for local LLM deployment
- Runs on PC with single consumer-grade GPU
- Supports Windows with GPU inference
- Supports AMD devices with ROCm
- Includes PowerInfer-2 framework for smartphones
- Provides Gradio server demo with Falcon(ReLU)-40B-FP16
Best for: People who want to run large language models locally on a personal computer with a consumer GPU.
Good for
- Open source repository
- Designed for local use with single consumer-grade GPU
- Supports Windows GPU inference and AMD devices with ROCm
Watch out for
- Project notes demo is experimental and without warranties
PowerInfer at a glance
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 6, 2026 · Online when checked
- Listed
- Oct 5, 2026
PowerInfer alternatives
Tools and sites you could use instead, ranked by how closely they match.
bitsandbytes enables large language models in PyTorch using k-bit quantization.
Open source
21 alternatives
Deploy generative and predictive AI models on Kubernetes with KServe’s distributed inference platform.
Open source
21 alternatives
KServe provides distributed inference for generative and predictive AI models on Kubernetes.
Open source
21 alternatives
ANEE wraps pretrained transformer inference with an energy budget and early exits inside model execution.
Open source
21 alternatives
GPUStack manages GPU clusters for serving AI models and provides on demand GPU instances accessible through SSH.
Open source
21 alternativesPopular in Model Serving & Inference
Well-known listings from the same topic.
MAI Playgroundplayground.microsoft.ai/chatBigModelbigmodel.cnkimi-k2.6platform.kimi.ai/docs/pricing/chat-k26Jinagithub.com/jina-ai/jinaFluxgithub.com/black-forest-labs/fluxprivateGPTgithub.com/imartinez/privateGPT
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.