Sola Security provides a shared security context layer that helps teams and agents answer questions about their environment.
ai-evaluation
agent-learning-kit provides an evaluation and simulation environment for AI-related workflows.
About ai-evaluation
Agent Learning Kit is a local-first SDK and CLI for testing, simulating, red-teaming, and optimizing AI agents. It provides simulate, evals, and optimize workflows, plus a conversational harness that reads an agent's source, builds a world its tools act on, and writes test scenarios.
It installs from PyPI and the documented quickstart runs offline without an API key or network. It includes CLI commands for eval, run, optimize, and redteam, example manifests, and a harness chat in the terminal or on localhost:8777. The TypeScript evaluation package is built from source pending npm publishing at v1 launch.
Key features
- Run local worlds, tasks, adapters, replays and regression artifacts
- Evaluate prompts, traces, memory, retrieval, safety and robustness evidence
- Search over prompts, agents, worlds, workflows and red-team scenarios
- Generate worlds and proved test scenarios from agent source via harness
- Run CLI commands for eval, simulation, optimization and red-team campaigns
- Operate offline without API key using bundled example manifests
Best for: Developers testing and improving AI agents with local simulation and evaluation workflows.
Good for
- Runs fully offline with no API key or network for quickstart
- Provides SDK and CLI for simulate, evaluate, and optimize loop
- Harness operates as conversation in terminal or local web page
Watch out for
- TypeScript SDK requires building from source until v1 npm publishing
ai-evaluation at a glance
- Category
- AI & Machine Learning › AI Data & Analytics
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Not stated
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
ai-evaluation alternatives
Tools and sites you could use instead, ranked by how closely they match.
An open-source visual programming environment for battle-testing prompts to LLMs.
Open source
24 alternatives
VeritasGraph combines knowledge graphs and GraphRAG for multi hop reasoning and attributed retrieval over user data.
Fiddler provides evaluation, monitoring, enforcement, governance, and cost controls for enterprise AI agents.
Zerve supports research and analytics workflows, with AI agents for data discovery, analysis, reports and production deployment.
Flexor prepares and retrieves unstructured data as context for AI agents, workflows, and enterprise decisions.
Popular in AI Data & Analytics
Well-known listings from the same topic.
Dotgetdot.aiForvioforvio.comFluidAI Medicalfluidai.mdApache Zeppelinzeppelin.apache.orgQuadraticquadratichq.comIntellectia AIintellectia.ai
ai-evaluation FAQ
Does the quickstart require an API key or network?
No. The source states everything in the quickstart runs fully offline with no API key and no network.
How is it installed for Python use?
Install from PyPI with pip install agent-learning-kit, with optional extras for livekit, nli, or all.
How do you evaluate a suite?
Run agent-learn eval examples/eval_suite.json --output artifacts/eval.json.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.