The Madrigal Database manages and serves archival and real-time data from upper-atmosphere science instruments in multiple formats.
vectara/hallucination-leaderboard
"leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents".
About vectara/hallucination-leaderboard
The vectara/hallucination-leaderboard is an open-source GitHub repository that hosts a public leaderboard ranking large language models (LLMs) by their rate of hallucination when summarizing short documents. It uses Vectara's Hallucination Evaluation Model (HHEM) to generate standardized, comparable metrics for each tested model, with planned regular updates as both the evaluation tool and the set of evaluated LLMs are updated over time.
The leaderboard reports four core metrics for every listed model: hallucination rate, factual consistency rate, answer rate, and average summary length in words. An interactive version of the leaderboard is hosted on Hugging Face, and archived versions of previous leaderboard iterations are available for users who want to reference older data. The repository includes supporting files including a citation guide, open-source license, and README for users seeking to access or reuse the dataset.
Key features
- Ranks LLMs by hallucination rate when summarizing short documents
- Uses Vectara's HHEM for standardized hallucination evaluation
- Tracks hallucination rate, factual consistency, answer rate, and summary length per model
- Interactive version hosted on Hugging Face for easier browsing
- Archived previous leaderboard versions available for reference
- Open-source repository with citation guide and license included
Best for: It is most useful for AI researchers, developers, and teams evaluating LLM performance for short-document summarization tasks.
Good for
- Open-source and freely accessible to all users
- Uses a standardized evaluation model for consistent cross-model comparisons
- Includes multiple relevant performance metrics per tested LLM
- Planned regular updates to reflect new models and evaluation improvements
Watch out for
- Only evaluates LLM performance on short-document summarization tasks, not other use cases
- Results are tied to Vectara's proprietary HHEM evaluation methodology
vectara/hallucination-leaderboard at a glance
- Category
- Research & Data › Open Data & Datasets
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
vectara/hallucination-leaderboard alternatives
Tools and sites you could use instead, ranked by how closely they match.
Search CHIANTI, an atomic database for spectroscopic diagnostics of astrophysical plasmas.
Linked COVID-19 Data transforms public datasets into RDF and provides demonstration dashboards through a SPARQL endpoint.
RDatasets.jl gives Julia users access to many datasets distributed with the R ecosystem.
Open source
24 alternatives
The IQB Research Data Centre supports use and provision of research data, including free access to national school performance study data.
Free
24 alternatives
The UK Office for National Statistics produces official statistics and serves as the country’s national statistical institute.
Popular in Open Data & Datasets
Well-known listings from the same topic.
data.worlddata.worldNASA APIapi.nasa.govState of Hawaiidata.hawaii.govCosmoSimcosmosim.orgdata.meudon.frdata.meudon.frESS-DIVEess-dive.lbl.gov
vectara/hallucination-leaderboard FAQ
What is the vectara/hallucination-leaderboard repository?
It is an open-source GitHub repository hosting a public leaderboard that ranks LLMs by their hallucination rate when summarizing short documents, using Vectara's HHEM evaluation model.
What metrics are included in the hallucination leaderboard?
The leaderboard reports four metrics per tested model: hallucination rate, factual consistency rate, answer rate, and average summary length in words.
Is the hallucination leaderboard free to access?
Yes, the repository and all its associated data are open-source and freely available to all users with no associated costs.
Are previous versions of the leaderboard available?
Yes, archived versions of earlier leaderboard iterations are linked in the repository for users who want to reference older data.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.