skill-optimizer benchmarks, evaluates, and optimizes skills to check their performance across large language models.
Open source
24 alternativesAgentBench: Evaluating LLMs as Agents
AgentBench evaluates large language models acting as agents through a research paper and discussion page.
About AgentBench: Evaluating LLMs as Agents
This is the Hugging Face paper page for AgentBench, a research paper published on August 7, 2023, that introduces a multi-dimensional benchmark for evaluating large language models acting as autonomous agents. The benchmark consists of 8 distinct interactive environments designed to assess reasoning and decision-making in multi-turn, open-ended generation settings. The paper reports tests across 25 LLMs, including commercial API models and open-source models, and notes a performance disparity between the strongest commercial models and open-source competitors.
Key features
- Benchmark spanning 8 distinct interactive environments
- Assesses agent reasoning and decision-making in multi-turn open-ended tasks
- Reported tests across 25 LLMs, including APIs and open-source models
- Datasets, environments and an integrated evaluation package released on GitHub
- Paper page hosts community comments, upvotes and citation links
Best for: Researchers and practitioners who want a published benchmark and shared evaluation setup for comparing how LLMs perform as agents.
Good for
- Datasets, environments and an integrated evaluation package are released for reuse
- Compares a broad set of commercial and open-source models under one benchmark
- Linked arXiv entry and GitHub repository provide the underlying paper and code
Watch out for
- The evaluation package, datasets and environments are hosted externally on GitHub rather than on this page
- Reported results reflect the 25 models tested at the time of publication
- No pricing information is stated for this page
AgentBench: Evaluating LLMs as Agents at a glance
- Category
- AI & Machine Learning › AI Data & Analytics
- Type
- Website
- Pricing
- Not confirmed. Check the site
- Platforms
- Web
- Open source
- Not stated
- Last checked
- Oct 6, 2026 · Online when checked
- Listed
- Oct 5, 2026
AgentBench: Evaluating LLMs as Agents alternatives
Tools and sites you could use instead, ranked by how closely they match.
SimpleBench evaluates everyday reasoning by comparing language models with human performance.
A free downloadable guide covering strategies for testing large language models and analyzing their behavior.
This article discusses challenges in evaluating language models and interpreting benchmark results.
Context Arena provides a leaderboard for evaluating language models on long-context tasks.
Turbostream analyzes real time streaming data with large language models.
Open source
24 alternativesPopular in AI Data & Analytics
Well-known listings from the same topic.
Dotgetdot.aiForvioforvio.comFluidAI Medicalfluidai.mdApache Zeppelinzeppelin.apache.orgSola Securitysola.securityQuadraticquadratichq.com
AgentBench: Evaluating LLMs as Agents FAQ
What does AgentBench evaluate?
It evaluates large language models acting as autonomous agents, focusing on reasoning and decision-making in multi-turn, open-ended generation across interactive environments.
How many environments and models are covered?
The benchmark described in the paper consists of 8 distinct environments, and the reported tests cover 25 LLMs, including API-based and open-source models.
Where can I find the datasets and evaluation package?
The paper states that datasets, environments and an integrated evaluation package for AgentBench were released at github.com/THUDM/AgentBench.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.