PEFT provides parameter-efficient fine-tuning methods for adapting pretrained machine learning models.
Open source
24 alternativesDeepSpeed
DeepSpeed provides optimization tools that make distributed deep learning training more efficient.
About DeepSpeed
DeepSpeed is a deep learning optimization library for distributed training. It includes system techniques such as ZeRO, 3D parallelism, DeepSpeed-MoE and ZeRO-Infinity, with recent work on offloading, long-sequence training and compiler optimization.
It has been used to train large-scale models including Megatron-Turing NLG and BLOOM, and is listed as integrating with Transformers, Accelerate, Lightning and MosaicML.
Key features
- ZeRO memory optimization for training large models
- 3D parallelism for distributed deep learning training
- Mixture-of-experts training and inference via DeepSpeed-MoE
- ZenFlow offloading engine for LLM training
- Long-sequence training for multi-million token sequences
- Compiler optimization for distributed training via DeepCompile
Best for: Teams training large deep learning or language models across distributed hardware.
Good for
- Used to train large-scale models including BLOOM and Megatron-Turing NLG
- Integrates with Transformers, Accelerate, Lightning and MosaicML
- Publishes ongoing updates on offloading, long-sequence and compiler work
DeepSpeed at a glance
- Website
- deepspeed.ai
- Type
- Online tool
- Pricing
- Not confirmed. Check the site
- Platforms
- Web
- Open source
- Not stated
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
DeepSpeed alternatives
Tools and sites you could use instead, ranked by how closely they match.
P-tuning v2 is a deep prompt tuning strategy designed as an alternative to full model fine-tuning.
Open source
24 alternatives
oumi supports fine-tuning, evaluation, and deployment of open-weight language models.
Open source
24 alternatives
DeepSpeed optimizes distributed training and inference for deep learning models.
Open source
24 alternatives
Enables single machine or distributed training and evaluation of deep learning models.
Open source
24 alternatives
Hugging Face Accelerate launches and trains PyTorch models across devices and distributed configurations.
Open source
24 alternativesPopular in ML Training & Optimization
Well-known listings from the same topic.
Edwardedwardlib.orgDataturedatature.ioAerosolveaerosolve.airbnb.techTogether AItogether.aiOptunaoptuna.orgCatBoostcatboost.ai
DeepSpeed FAQ
Which frameworks integrate with DeepSpeed?
Transformers, Accelerate, Lightning and MosaicML are listed as integrations.
Which large models have used DeepSpeed?
Listed examples include Megatron-Turing NLG, Jurassic-1, BLOOM, GLM, YaLM, GPT-NeoX, AlexaTM, Turing NLG and METRO-LM.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.