MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
Open source
24 alternativesGPT-NeoX
GPT-NeoX implements model parallel autoregressive transformers on GPUs using Megatron and DeepSpeed libraries.
About GPT-NeoX
GPT-NeoX is an open source repository from EleutherAI containing a library for training large-scale autoregressive language transformers on GPUs. Its framework is based on NVIDIA's Megatron Language Model and extends it with techniques from DeepSpeed and additional optimizations. The project states that it is intended as a central place to gather techniques for training large autoregressive language models.
The README says the library is for training models with billions of parameters from scratch. It states that for generic inference, Hugging Face's transformers library is recommended instead, and that it supports GPT-NeoX models. According to the README, the library has been used by more than 40 organizations to train more than 50 models, and has been run at scale on several NVIDIA and AMD GPU clusters.
Key features
- Distributed training with ZeRO and 3D parallelism
- Job launching through Slurm, MPI, and the IBM Job Step Manager
- Rotary and ALiBi positional embeddings, parallel feedforward attention layers, and flash attention
- Predefined configurations for Pythia, PaLM, Falcon, and LLaMA 1 and 2 architectures
- Curriculum learning support
- Experiment monitoring with Weights and Biases, Comet, or TensorBoard
Best for: Researchers and engineers who train large autoregressive language models from scratch on GPU clusters.
Good for
- The README states that it offers substantially increased usability compared with the Megatron-DeepSpeed library.
- The README reports use by more than 40 organizations to train more than 50 models.
- Integrates with Hugging Face tokenizers and transformers libraries and the Language Model Evaluation Harness.
Watch out for
- The README says it is likely the wrong library for users who do not plan to train models with billions of parameters from scratch.
- The README recommends Hugging Face transformers for generic inference needs.
- The codebase has primarily been developed and tested for Python 3.8 to 3.10.
GPT-NeoX at a glance
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 6, 2026 · Online when checked
- Listed
- Oct 5, 2026
GPT-NeoX alternatives
Tools and sites you could use instead, ranked by how closely they match.
DyNet provides tools for building dynamic neural networks and deep learning models.
Open source
24 alternatives
Extensible Toolkit for Finetuning and Inference of Large Foundation Models.
Open source
24 alternatives
The colibri-core NLP tool and library works with n-grams, skipgrams, and other linguistic patterns.
Open source
24 alternatives
Gymnasium provides a standard API for single agent reinforcement learning environments and related utilities.
Open source
24 alternatives
DEAP provides a Python framework for prototyping evolutionary computation algorithms and testing new ideas.
Open source
24 alternativesPopular in AI Infrastructure & MLOps
Well-known listings from the same topic.
Guild AIguild.aiFlow AIflow-ai.comsystemprompt.iosystemprompt.ioCoreWeavecoreweave.comKili Technologykili-technology.comAPI docsdocs.x.ai
GPT-NeoX FAQ
Is GPT-NeoX meant for running inference on existing models?
The README says that for generic inference needs, the Hugging Face transformers library is recommended instead, and that it supports GPT-NeoX models.
Which Python versions does the codebase target?
The README states that the codebase has primarily been developed and tested for Python 3.8 to 3.10.
Which hardware has GPT-NeoX been run on?
The README states it has been run at scale on NVIDIA and AMD GPUs on AWS, CoreWeave, ORNL Summit and Frontier, LUMI, and CINECA Leonardo.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.