FastChat offers an open platform for training, serving, and evaluating large language models, and hosts release repositories for Vicuna and Chatbot Arena.
Open source
24 alternativesModelz-LLM
OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others).
About Modelz-LLM
Modelz-LLM is an open source inference server for running open source large language models on local or cloud environments. It provides an OpenAI-compatible API for completions, chat completions, and embeddings, usable with the OpenAI Python SDK or LangChain by pointing api_base to the self-hosted server.
It supplies Docker images for models including FastChat-T5, Vicuna, LLaMA, ChatGLM, and BloomZ, with deployment options for Kubernetes and Modelz. Smaller BloomZ variants list CPU support, while larger models list specific Nvidia GPU requirements.
Key features
- OpenAI-compatible endpoints for completions, chat, and embeddings
- Use OpenAI Python SDK with custom api_base
- Integrate with LangChain via OpenAI wrapper
- Self-hosted server for local or cloud environments
- Docker images for specific LLMs for Kubernetes deployment
- Supports LLaMA, Vicuna, ChatGLM, FastChat-T5, BloomZ
Best for: Developers who want to self-host open source LLMs behind an OpenAI-compatible API.
Good for
- Open source code under Apache-2.0 license
- Works with existing OpenAI Python SDK and LangChain code
- Provides Docker images for Kubernetes and cloud deployment
Watch out for
- Larger listed models require specific Nvidia GPUs
- Model support is limited to the models listed in the repository
Modelz-LLM at a glance
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
Modelz-LLM alternatives
Tools and sites you could use instead, ranked by how closely they match.
File Parser optimised for LLM Ingestion with no loss. Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.
Open source
24 alternatives
Rockpool supports training spiking neural networks with PyTorch and JAX and deploying them to neuromorphic hardware.
Open source
24 alternatives
Torchvision provides datasets, transforms, and models for computer vision with PyTorch.
Open source
24 alternatives
The climin package provides optimization methods focused on machine learning scenarios in Python.
Open source
24 alternatives
ORMB packages machine learning models as OCI artifacts, in a workflow described as Docker for ML models.
Open source
24 alternativesPopular in AI Infrastructure & MLOps
Well-known listings from the same topic.
Guild AIguild.aiFlow AIflow-ai.comsystemprompt.iosystemprompt.ioCoreWeavecoreweave.comKili Technologykili-technology.comAPI docsdocs.x.ai
Modelz-LLM FAQ
How do I run a local server?
Install with pip install modelz-llm, then run e.g. modelz-llm -m bigscience/bloomz-560m --device cpu.
Which API endpoints are supported?
/completions, /chat/completions, /embeddings, /engines//embeddings, and /v1/ versions of completions, chat completions, and embeddings.
Can I use it with LangChain?
Yes, by setting api_base to the local server and using LangChain's OpenAI wrapper.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.