Petals runs large language models across distributed computers and supports fine-tuning and inference.
Open source
24 alternativesPetals
Petals runs large language models at home through a BitTorrent-style approach and supports text generation and fine-tuning.
About Petals
Petals is a system for running large language models in a BitTorrent-style setup: you load part of a model on your own machine, then join a network of other people serving the remaining parts. It supports text generation with Llama 3.1 (up to 405B), Mixtral (8x22B), Falcon (40B+) and BLOOM (176B), and fine-tuning of those models, using a consumer-grade GPU or Google Colab.
Beyond standard LLM APIs, the project states that you can use your own fine-tuning and sampling methods, execute custom paths through the model, or inspect its hidden states, combining API-like use with PyTorch and Transformers. Reported single-batch inference speeds are up to 6 tokens/sec for Llama 2 (70B) and up to 4 tokens/sec for Falcon (180B). Pricing is not stated, and the listing does not mark the project as open source.
Key features
- Loads part of a model locally; other parts served by a network
- Text generation with Llama 3.1, Mixtral, Falcon and BLOOM
- Fine-tuning of supported models for custom tasks
- Runs on a consumer-grade GPU or Google Colab
- Custom fine-tuning and sampling methods, custom model paths, hidden states
- Pairs an API-style interface with PyTorch and Transformers
Best for: Developers and researchers with a consumer-grade GPU who want to generate text with or fine-tune large models without hosting the full model themselves.
Good for
- Runs large models on consumer-grade hardware by splitting them across a network
- Supports very large models, including Llama 3.1 up to 405B and BLOOM 176B
- Allows custom sampling, fine-tuning methods and inspection of hidden states
Watch out for
- Single-batch inference is limited to about 6 tokens/sec for Llama 2 70B and 4 tokens/sec for Falcon 180B
- Depends on other people serving parts of the model; the site shows the network status can fail to load
- No pricing information is provided, and the listing does not mark the project as open source
Petals at a glance
- Website
- petals.dev
- Type
- Online tool
- Pricing
- Not confirmed. Check the site
- Platforms
- Web
- Open source
- Not stated
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
Petals alternatives
Tools and sites you could use instead, ranked by how closely they match.
LLM-API-Key-Proxy provides a gateway for multiple language model APIs, with compatible endpoints and provider translation.
This bilingual chat language model is described in English and Chinese on the ChatGLM2-6B repository page.
LocalAI runs text, voice, vision, image, video, and agent workloads through an open-source AI engine.
Open source
24 alternatives
SillyTavern is a frontend for users working with large language models.
Open source
24 alternatives
alphaXiv helps users discover, read, and discuss arXiv papers, with research answers grounded in the underlying work.
Popular in AI Chatbots & Assistants
Well-known listings from the same topic.
Lumolumo.proton.meMicrosoft Copilotcopilot.microsoft.comLearn Aboutlearning.google.com/experiments/learn-aboutEidoloneidolonai.comFlowHuntflowhunt.ioPine AI19pine.ai
Petals FAQ
What hardware do I need to use Petals?
The project says you can run it with a consumer-grade GPU or Google Colab, loading part of the model on your own machine.
How fast is text generation?
The site reports single-batch inference of up to 6 tokens/sec for Llama 2 (70B) and up to 4 tokens/sec for Falcon (180B), which it describes as enough for chatbots and interactive apps.
Can I fine-tune models with Petals?
Yes. The project states you can fine-tune supported models for your tasks and employ your own fine-tuning and sampling methods.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.