Looking for something like FastChat? Here are 24 alternatives, ranked by how closely they match it on purpose, features and platforms. Of these, 1 are free or have a free plan and 21 are open source.
FastChat offers an open platform for training, serving, and evaluating large language models, and hosts release repositories for Vicuna and Chatbot Arena.
| # | Alternative | Pricing | Open source | Platforms |
|---|---|---|---|---|
| 1 | TensorRT-LLM | — | Not stated | — |
| 2 | Sagify | — | Yes | — |
| 3 | deeplake | — | Yes | — |
| 4 | Starwhale | — | Yes | — |
| 5 | Nebullvm | — | Yes | — |
| 6 | traceAI | — | Yes | — |
| 7 | opensource | — | Yes | — |
| 8 | Upstage | — | Not stated | Web |
| 9 | Modelz-LLM | — | Yes | — |
| 10 | TFServing | — | Yes | — |
TensorRT-LLM provides Python and C++ runtimes with optimizations for language model inference on NVIDIA GPUs.
Sagify describes its project as a way to work with large language models and machine learning more easily.
DeepLake provides multimodal data infrastructure for AI agents, including retrieval and training.
Machine learning operations and large language model operations are the focus of Starwhale’s platform.
OptiMate is a collection of libraries for optimizing AI model performance; the repository is marked as a legacy project.
traceAI provides open source tracing for AI applications and frameworks using OpenTelemetry.
Helicone provides open-source observability for LLM applications, with tools to monitor, evaluate, and experiment.
Upstage builds large language models and document-processing engines for workplace workflows.
OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others).
Flexible, high-performance serving system for machine learning models.
Train and run models with OneFlow, a scalable deep learning framework for distributed machine learning workloads.
dstack coordinates AI compute and supports training and inference across GPU clouds, Kubernetes, VMs, and bare metal.
Forefront provides a platform for fine tuning and running inference with open source language models.
ORMB packages machine learning models as OCI artifacts, in a workflow described as Docker for ML models.
Cog packages machine learning models into containers to support deployment across environments.
DJL is a Java framework for running deep learning models across different engines.
ONNX-MLIR compiles ONNX models using MLIR compiler infrastructure for model deployment workflows.
Manifest connects AI agents and tools to multiple model providers through an LLM gateway.
ModelDB tracks machine learning model versions, metadata, and experiments.
Model deployment for AI applications is supported by nndeploy, a framework described as easy to use and high performance.
MMDeploy provides a model deployment framework from the OpenMMLab project.
Kitaru provides agent traces that teams can run and inspect during application workflows.
The Hydrosphere repository describes an MLOps platform for machine learning operations.
File Parser optimised for LLM Ingestion with no loss. Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.
Alternatives are suggested from the purpose and features described in our listings. Different pages for the same product appear once. Read each description to check whether it fits your task. How we choose alternatives · Suggest an alternative
