OpenRouter publishes language model rankings based on real usage through its API.
Vals AI
vsAgentBench: Evaluating LLMs as Agents
Vals AI vs AgentBench: Evaluating LLMs as Agents
How do Vals AI and AgentBench: Evaluating LLMs as Agents compare? Here are the facts side by side: pricing, platforms, open-source status and what each one is for.
| Directory detail | Vals AI | AgentBench: Evaluating LLMs as Agents |
|---|---|---|
| What it is | Vals AI evaluates models on real-world tasks across fields including finance, software, cybersecurity, and healthcare. | AgentBench evaluates large language models acting as agents through a research paper and discussion page. |
| Website | ||
| Type | Website | Website |
| Topic | ||
| Pricing | Not confirmed | Not confirmed |
| Platforms | Web | Web |
| Open source | Not stated | Not stated |
| Language | English | — |
| Alternatives |
Other options to consider
Sites similar to both Vals AI and AgentBench: Evaluating LLMs as Agents.
PricePerToken compares API costs for language models from multiple providers.
LifeArchitect publishes a reference table of language models and their specifications and data points.
SimpleBench evaluates everyday reasoning by comparing language models with human performance.
Context Arena provides a leaderboard for evaluating language models on long-context tasks.
Indico benchmarks language models on extractive, classification, and predictive tasks.