OneCompression is a Python package for compressing large language models.
Open source
24 alternativesTensorRT-LLM
vsGPTCache
TensorRT-LLM vs GPTCache
How do TensorRT-LLM and GPTCache compare? Here are the facts side by side: pricing, platforms, open-source status and what each one is for.
| Directory detail | TensorRT-LLM | GPTCache |
|---|---|---|
| What it is | TensorRT-LLM provides Python and C++ runtimes with optimizations for language model inference on NVIDIA GPUs. | GPTCache provides semantic caching for large language models and integrates with LangChain and LlamaIndex. |
| Website | ||
| Type | Open-source project | Open-source project |
| Topic | ||
| Pricing | Not confirmed | Not confirmed |
| Platforms | Not recorded | Not recorded |
| Open source | Not stated | Yes |
| Language | English | English |
| Alternatives |
Other options to consider
Sites similar to both TensorRT-LLM and GPTCache.
OpenLLMetry provides OpenTelemetry-based observability for generative AI and large language model applications.
Open source
24 alternatives
Extensible Toolkit for Finetuning and Inference of Large Foundation Models.
Open source
24 alternativesMosec serves machine learning models with dynamic batching and CPU or GPU pipelines for inference workloads.
Open source
24 alternatives
GPT-NeoX implements model parallel autoregressive transformers on GPUs using Megatron and DeepSpeed libraries.
Open source
24 alternatives
BentoML helps build and serve AI apps and models through inference APIs, job queues, and multi-model pipelines.
Open source
24 alternatives