PromptSite provides prompt version management for tracking, experimentation, and debugging LLM prompts.
Open source
24 alternativesconfident-ai/deepeval
DeepEval is an LLM evaluation framework, according to its repository description.
About confident-ai/deepeval
DeepEval is an open-source framework for evaluating large-language model systems, described as similar to Pytest but for unit testing LLM applications. It supports testing of agents, RAG pipelines, and chatbots at the level of end-to-end behavior, full agent trajectories, and individual steps such as LLM calls, tool use, and retrieval.
It includes ready-to-use metrics for custom, agentic, RAG, and multi-turn evaluation that can use a selected LLM or NLP models that run locally. Comparing iterations, sharing reports, and production monitoring are linked to Confident AI, a separate enterprise platform.
Key features
- Evaluate LLM apps end-to-end as black boxes
- Evaluate complete agent trajectories across decisions and actions
- Evaluate individual steps such as LLM calls, tool use, retrieval
- RAG metrics for answer relevancy, faithfulness, and retrieval context
- Agentic metrics for task completion, tool correctness, and plan adherence
- Custom metrics including G-Eval, DAG builder, and JevEval
Best for: Developers building and testing LLM agents, RAG pipelines, or chatbots, including those using LangChain or OpenAI.
Good for
- Open-source framework for unit testing LLM applications
- Evaluates end-to-end behavior, full trajectories, and individual agent steps
- Metrics can use a chosen LLM or NLP models that run locally
Watch out for
- Comparing iterations, sharing reports, and production monitoring require the separate Confident AI platform
confident-ai/deepeval at a glance
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
confident-ai/deepeval alternatives
Tools and sites you could use instead, ranked by how closely they match.
OneCompression is a Python package for compressing large language models.
Open source
24 alternatives
This repository contains utilities intended for use with Llama models.
OpenLIT provides observability and evaluation for AI agents and coding agents, tracing models, tools, prompts, costs, and workflows.
Open source
24 alternatives
Semantic Cache Router presents a distributed semantic cache and stateful routing system.
Notte provides cloud browser infrastructure and web automation for AI and coding agents.
Popular in AI Infrastructure & MLOps
Well-known listings from the same topic.
Guild AIguild.aiFlow AIflow-ai.comsystemprompt.iosystemprompt.ioCoreWeavecoreweave.comKili Technologykili-technology.comAPI docsdocs.x.ai
confident-ai/deepeval FAQ
What can DeepEval evaluate?
End-to-end LLM apps as black boxes, complete agent trajectories, and individual steps such as LLM calls, tool use, retrieval, and sub-agent handoffs.
What kinds of metrics are included?
Custom metrics such as G-Eval, DAG, and JevEval, plus agentic, RAG, and multi-turn metrics such as task completion, faithfulness, and knowledge retention.
Does it work with LangChain or OpenAI implementations?
The repository states it supports agents, RAG pipelines, and chatbots implemented via LangChain or OpenAI.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.