The Best WhichLLM Alternatives in 2026

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

LLMeBench provides a flexible framework for benchmarking large language models.

AwaDB provides an AI-focused embedding vector database with real-time search features.

Aider LLM Leaderboards compares language models on code editing benchmarks.

skill-optimizer benchmarks, evaluates, and optimizes skills to check their performance across large language models.

Turbostream analyzes real time streaming data with large language models.

This repository catalogs AI models across providers with pricing, context, modality, and capability data.

Giskard provides open source evaluation and testing tools for LLM agents.

FastDatasets helps create training datasets for large language models.

Automatic "Differentiation" via Text, using large language models to backpropagate textual gradients.

A free downloadable guide covering strategies for testing large language models and analyzing their behavior.

HPOBench is a collection of benchmark problems for hyperparameter optimization.

NALP covers Natural Adversarial Language Processing through a dedicated library.

Indico benchmarks language models on extractive, classification, and predictive tasks.

Deploybase compares GPU cloud and language model API prices, performance benchmarks, and provider market data.

OpenLM Arena compares open language models through leaderboards and benchmark results.

This article discusses challenges in evaluating language models and interpreting benchmark results.

Canaryarena ranks AI models through chat, comparison, and community voting.

pytorch tabular provides a unified framework for deep learning models on tabular data.

Captum explains model behavior and supports interpretability analysis for PyTorch systems.

PyTorch Frame provides a library for applying deep learning models to tabular data.

A Python package to assess and improve fairness of machine learning models.

AIF360 provides fairness metrics and explanations for datasets and models, along with algorithms to mitigate bias.

This repository provides segmentation models with pretrained backbones for Keras and TensorFlow Keras.