LangWatch provides evaluations for language models and tests for AI agent applications.
Open source
16 alternativespromptfoo
promptfoo tests prompts, agents, and retrieval systems with evaluations and security testing.
About promptfoo
promptfoo is an open-source CLI and library for evaluating prompts, agents, and RAG systems. It supports automated evaluations, red teaming and vulnerability scanning, side-by-side comparison of models from providers such as OpenAI, Anthropic, Azure, Bedrock, and Ollama, and checks in CI/CD using declarative configs.
It is installed via npm, brew, or pip, or run with npx, and includes commands to run evals and view results. Most LLM providers require an API key set as an environment variable, so prompts are sent to those external providers during testing.
Key features
- Automated evaluations for prompts, agents, and RAG systems
- Red teaming and vulnerability scanning for LLM apps
- Side-by-side comparison across multiple LLM providers
- Declarative configs with command line and CI/CD integration
- Pull request code scanning for LLM security issues
- Install via npm, brew, pip, or run with npx
Best for: Developers who build LLM apps and need command-line evaluation and vulnerability testing.
Good for
- Open source under MIT license
- Works with multiple LLM APIs and programming languages
- Installable via npm, brew, pip, or npx
Watch out for
- Most LLM providers require an API key set as an environment variable
promptfoo at a glance
- Website
- github.com/typpo/promptfoo
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
promptfoo alternatives
Tools and sites you could use instead, ranked by how closely they match.
LangKit monitors language model prompts and responses and extracts signals related to safety and security.
Open source
ReviewCerberus analyzes Git branch differences and generates code review reports using AWS Bedrock or Anthropic API.
Open source
15 alternatives
Qodo reviews code with codebase context and supports workflows for IDEs, pull requests, and security.
rehydra scans LLM prompts for accidental personal information before they reach a model.
Open source
17 alternatives
EvalView regression tests AI agents by snapshotting behavior and comparing tool calls in CI.
Open source
17 alternativesPopular in AI Code Review & Testing
Well-known listings from the same topic.
Prediction Guardpredictionguard.comGreptilegreptile.comGitAutogitauto.aipromptfoopromptfoo.devRefractionrefraction.devKatalonkatalon.com
promptfoo FAQ
How do I install promptfoo?
Install with npm install -g promptfoo, brew install promptfoo, or pip install promptfoo, or run it with npx promptfoo@latest without installing.
How do I run an evaluation?
Set your provider API key as an environment variable, then run promptfoo eval and promptfoo view in the project directory.
Is promptfoo open source?
Yes, it remains open source and MIT licensed.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.