DeepSeek Coder is a language model project for code generation and programming tasks.
Open source
24 alternativesDeep Swe
DeepSWE benchmarks coding agents on original, long-horizon software engineering tasks.
About Deep Swe
DeepSWE is a benchmark for measuring coding agents on original, long-horizon software engineering tasks drawn from active open-source repositories. It includes 113 tasks across TypeScript, Go, Python, JavaScript, and Rust, with isolated environments and program-based verifiers that check observable behavior.
Tasks use the Harbor format with metadata, instructions, a Dockerfile, held-out tests, and a reference solution that is not used for grading. Evaluations run through Pier, with the agent working in an isolated environment and grading applied as a patch in a separate pristine container, producing structured scores and test logs.
Key features
- 113 tasks across TypeScript, Go, Python, JavaScript, and Rust
- Isolated agent and verifier environments with program-based checks
- Behavior-based grading accepting any correct observable behavior
- Structured run outputs including reward.json, CTRF report, and logs
- Run full corpus, deterministic subsets, or single tasks via Pier
- Supports mini-swe-agent, claude-code, codex, gemini-cli, and opencode
Best for: Researchers and engineers evaluating coding agents on multi-language, long-horizon software tasks.
Good for
- Open source repository under Apache-2.0 license.
- Reference solutions are held out from grading for offline review.
- Each run produces structured scores, test reports, and raw logs.
Watch out for
- Version 1.1 grading requires Pier newer than 0.3.0.
- Running the benchmark requires installing Pier and configuring external model API keys.
Deep Swe at a glance
- Category
- AI & Machine Learning › Models & ML Research
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 6, 2026 · Online when checked
- Listed
- Oct 5, 2026
Deep Swe alternatives
Tools and sites you could use instead, ranked by how closely they match.
OpenVLA provides an open-source vision-language-action model for robotic manipulation tasks.
Open source
24 alternatives
MobiLlama is a small language model designed for edge devices, according to its repository description.
Open source
24 alternatives
Janus combines multimodal understanding and image generation models in a unified model series.
Open source
24 alternatives
Point·E is a research project that generates 3D point clouds using diffusion models.
Open source
24 alternatives
CodeGen provides open-source models for program synthesis and code generation.
Open source
24 alternativesPopular in Models & ML Research
Well-known listings from the same topic.
Falcon LLMfalconllm.tii.aeERNIEernie.baidu.comTokenVersetoken-verse.github.iowTAObittensor.comAlpacacrfm.stanford.edu/2023/03/13/alpaca.htmlSmolVLAhuggingface.co/blog/smolvla
Deep Swe FAQ
How do I run DeepSWE?
Clone the repository, install datacurve-pier, set a model API key, then run pier run with mini-swe-agent and a model.
Can I run only part of the benchmark?
Yes. Use --n-tasks with --sample-seed for a deterministic subset, or point pier run at a single task-id directory.
What output does grading produce?
reward.json with scores, ctrf.json test report, test-stdout.txt, run.log, and framework-native files in reports/.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.