The Best openai/evals Alternatives in 2026

WhichLLM tests local language models on a user’s hardware and ranks them using recent benchmarks.

AwaDB provides an AI-focused embedding vector database with real-time search features.

LLMeBench provides a flexible framework for benchmarking large language models.

Giskard provides open source evaluation and testing tools for LLM agents.

nilearn provides Python tools for machine learning and neuroimaging analysis.

skill-optimizer benchmarks, evaluates, and optimizes skills to check their performance across large language models.

Open-RAG-Eval evaluates retrieval-augmented generation without requiring golden answers.

Aider LLM Leaderboards compares language models on code editing benchmarks.

Cornac provides a framework for comparing multimodal recommender systems and recommendation algorithms.

The LightGBM framework uses gradient boosting for ranking, classification, and other machine learning tasks.

Captum explains model behavior and supports interpretability analysis for PyTorch systems.

An open source Python library for drift detection in machine learning systems.

This repository catalogs AI models across providers with pricing, context, modality, and capability data.

Turbostream analyzes real time streaming data with large language models.

neuropredict evaluates predictive performance using neuroimaging features.

Adala automates data labeling through an autonomous agent framework for organizing labeled datasets.

pytorch tabular provides a unified framework for deep learning models on tabular data.

modAL provides a modular active learning framework for Python built on top of scikit-learn.

Spektral provides a Python framework for creating graph neural networks with Keras and TensorFlow.

HPOBench is a collection of benchmark problems for hyperparameter optimization.

Omnigraph provides a graph engine for lakehouse data with Git-style workflows.

DVCLive logs and tracks machine learning metrics, parameters, and models with Git or DVC.

Pyro brings probabilistic programming capabilities to Python projects using PyTorch.

This Swift library builds machine learning predictions using linear regression.