WhichLLM tests local language models on a user’s hardware and ranks them using recent benchmarks.
Open source
24 alternativesopenai/evals
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
About openai/evals
Evals is a framework for evaluating large language models and systems built with them, with a registry of evals for testing dimensions of OpenAI models. For builders working with LLMs, it supports running existing evals, writing custom evals for specific use cases, and building private evals from own data without public exposure. Use requires an OpenAI API key and Python 3.9+, with registry data fetched via Git LFS.
Pricing is not clearly stated.
Key features
- Configure and run evals directly in the OpenAI Dashboard
- Install the package locally with pip install evals
- Run and create evals using Weights & Biases
- Log eval results to a Snowflake database
Watch out for
- Does not currently accept evals with custom code
openai/evals at a glance
- Website
- github.com/openai/evals
- Category
- AI & Machine Learning › AI Data & Analytics
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Details verified
- Oct 22, 2026 · Official source
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
openai/evals alternatives
Tools and sites you could use instead, ranked by how closely they match.
AwaDB provides an AI-focused embedding vector database with real-time search features.
Open source
24 alternatives
LLMeBench provides a flexible framework for benchmarking large language models.
Giskard provides open source evaluation and testing tools for LLM agents.
Open source
24 alternatives
nilearn provides Python tools for machine learning and neuroimaging analysis.
Open source
24 alternatives
skill-optimizer benchmarks, evaluates, and optimizes skills to check their performance across large language models.
Open source
24 alternativesPopular in AI Data & Analytics
Well-known listings from the same topic.
Dotgetdot.aiForvioforvio.comFluidAI Medicalfluidai.mdApache Zeppelinzeppelin.apache.orgSola Securitysola.securityQuadraticquadratichq.com
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.