Sign in Submit site

Voice Lab

github.com/saharmor/voice-lab Developer Tools Open-source projectOpen source

Voice Lab provides a framework for testing and evaluating voice agents.

Visit Voice Lab 24 alternatives
github.com/saharmor/voice-lab
Voice Lab homepage screenshot

About Voice Lab

Voice Lab is an open-source Python framework for testing and evaluating LLM-powered agents, with a focus on voice agents. It is intended to help developers compare language models and prompts so that decisions, such as switching to a smaller or newer model to reduce cost, can be based on measured results rather than manual review of call logs.

The framework lets users define custom metrics in JSON and use LLM-as-a-Judge to score them, track performance across configurations, and generate comparison tables across models. Its current scope is limited: the source states that it supports only the text part of a voice agent, meaning the underlying language model and prompt, and not speech components. Setup requires cloning the repository, installing the requirements, and creating a .env file with an OpenAI API key.

Key features

  • Custom evaluation metrics defined in JSON and scored with LLM-as-a-Judge
  • Tracks performance metrics across different model and prompt configurations
  • Generates comparison tables across different language models
  • Supports switching between models, such as Claude Sonnet and GPT-4
  • Tests multiple prompt variations systematically
  • Configuration Editor for generating JSON test configuration files
Best for: Developers and teams building LLM-powered voice agents who need to compare models or prompts with repeatable tests.
Good for
  • Open source, with the repository published on GitHub under a LICENSE file
  • Supports custom metrics, so evaluation criteria can be defined for a specific agent
  • Applies to any LLM-powered agent evaluation, not only voice agents
Watch out for
  • Currently supports only the text part of a voice agent, namely the language model and prompt
  • Setup requires creating a .env file with an OPENAI_API_KEY
  • New test scenarios can be added by editing JSON files or through the Configuration Editor

Voice Lab at a glance

Type
Open-source project
Pricing
Not confirmed. Check the site
Open source
Yes · Source code
Language
English
Last checked
Oct 6, 2026 · Online when checked
Listed
Oct 5, 2026
open sourcepythonai agentsdeveloper tools

Voice Lab alternatives

Tools and sites you could use instead, ranked by how closely they match.

All 24 alternatives

Popular in Developer Tools

Well-known listings from the same topic.

All 423
AppDevToolsappdevtools.comCommand Not Foundcommand-not-found.comTestFlighttestflight.apple.com/join/A8GwCnq8Sourcegraphabout.sourcegraph.comProject Wallaceprojectwallace.comCarbon Calculatorwebsitecarbon.com
Related topicsWeb DevelopmentAPIs & DatabasesCode Editors & IDEsNetwork ToolsCloud & Web HostingGit & Code HostingProgramming LanguagesWeb Directories & Curated Lists

Voice Lab FAQ

Does Voice Lab test speech or audio components of a voice agent?
Not at present. The source states that the library supports only the text part of a voice agent, which means testing the underlying language model and prompt.
How are new test scenarios added?
They can be generated with the Voice Lab Configuration Editor or added by editing the test_details.json file in the llm_testing directory, following the template in the documentation.
What is needed to set up Voice Lab?
Clone the repository, create a Python virtual environment, install the packages in requirements.txt, and create a .env file in the project root containing an OPENAI_API_KEY.
Is something wrong or out of date?

Tell us and an editor will check it.

Own this website?

Claim the listing and suggest updates yourself. An editor checks every claim first.

Claim this listing

Listing maintained by Chris McClure and the WebIndexly editors · Updated Oct 12, 2026 · How we rank