Deploybase compares GPU cloud and language model API prices, performance benchmarks, and provider market data.
skill-optimizer
skill-optimizer benchmarks, evaluates, and optimizes skills to check their performance across large language models.
About skill-optimizer
skill-optimizer is an open-source Docker workbench and agent skill for running deterministic evaluations against agent skills. It can be used in two ways: installed as a skill or plugin into an agent to help author and debug eval suites, or run as a local CLI that executes cases and suites in Docker against models accessed through OpenRouter. Graders act as the acceptance contract, inspecting workspace files, command logs, SQL, generated artifacts, answer files and trace output, while the agent phase sees only the /work directory and not the graders, hidden answers or results.
Key features
- Docker workbench runs deterministic evals against agent skills
- CLI runs single cases or full suites against OpenRouter models
- Graders check workspace files, logs, SQL, artifacts and traces
- Agent phase isolated to /work, separated from graders and answers
- Plugin installs for Claude Code, Codex, Cursor, OpenCode and Gemini CLI
- Skill-only install option through the open skills CLI
Best for: Developers and teams who want to check whether their agent skills behave reliably across multiple models and whose task success can be verified from files, logs or local state.
Good for
- Open source with a Docker-based, isolated workbench for evaluations
- Supports several agent platforms, with every plugin manifest pointing to the same skill file
- Uses deterministic graders as an explicit acceptance contract for each case
Watch out for
- Only openrouter/... model references are supported
- The local CLI requires Node.js 20+, Docker and an OPENROUTER_API_KEY for real model runs
- Designed for evals whose success can be verified from files, command logs, SQL, artifacts or other local state
skill-optimizer at a glance
- Category
- AI & Machine Learning › AI Data & Analytics
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 6, 2026 · Online when checked
- Listed
- Oct 5, 2026
skill-optimizer alternatives
Tools and sites you could use instead, ranked by how closely they match.
Uncover LLM evaluation's importance and explore methods for assessing its performance and impact across industries.
Enkrypt AI provides security and compliance tools to detect threats, remove vulnerabilities, and monitor AI system performance.
This Hugging Face Space displays interactive leaderboards comparing AI performance across categories and methods.
Turbostream analyzes real time streaming data with large language models.
Open source
24 alternatives
WhichLLM tests local language models on a user’s hardware and ranks them using recent benchmarks.
Open source
24 alternativesPopular in AI Data & Analytics
Well-known listings from the same topic.
Dotgetdot.aiForvioforvio.comFluidAI Medicalfluidai.mdApache Zeppelinzeppelin.apache.orgSola Securitysola.securityQuadraticquadratichq.com
skill-optimizer FAQ
What do I need to run the local CLI?
Node.js 20 or newer, Docker, and an OPENROUTER_API_KEY for real model runs. Installation is done with npm install and npm run build.
Which models can be used?
Only openrouter/... model references are supported. Commands such as run-case and run-suite take OpenRouter model references.
Which agents can the skill be installed into?
Claude Code, OpenAI Codex CLI, the Codex app, Cursor, OpenCode and Gemini CLI each have their own install steps. A skill-only install is also available through the open skills CLI.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.