ORMB packages machine learning models as OCI artifacts, in a workflow described as Docker for ML models.
Open source
24 alternativesHarbor
Harbor sets up a preconfigured LLM stack with one command and includes hundreds of services to explore.
About Harbor
Harbor is an open-source CLI and companion app for running a preconfigured local LLM stack. A single harbor up command starts a chosen backend such as Ollama, llama.cpp, or vLLM together with a frontend like Open WebUI, while Harbor handles Docker Compose orchestration, configuration, and connectivity between the services. Additional components can be added in the same command, including SearXNG for web search, Speaches for voice chat, and ComfyUI for image generation.
Beyond starting services, Harbor can point installed coding and agent CLIs at its OpenAI-compatible backends with harbor launch, and it can store per-service arguments, such as llama.cpp flags. Platform-specific options include host-native Metal/GPU inference backends for macOS, a PrismML llama.cpp backend that runs Ternary Bonsai 2 GGUFs on CPU, NVIDIA, and ROCm, and a Halogen Flash Server backend for AMD Strix Halo. The repository states hundreds of services are available to explore; no pricing information is given in the source.
Key features
- Single 'harbor up' command starts a preconfigured local LLM stack
- Backends include Ollama, llama.cpp, and vLLM
- Services pre-wired together, including Open WebUI frontend
- Optional services such as SearXNG search, Speaches voice, ComfyUI images
- Harbor handles Docker Compose orchestration and cross-service connectivity
- Configures coding and agent CLIs via 'harbor launch'
Best for: Developers and self-hosters who want a working local LLM stack without manually wiring backends, frontends, and supporting services together.
Good for
- Open source and designed for self-hosted local use
- Reduces manual configuration by starting pre-connected services in one command
- Covers backends, frontends, and supporting tools such as search, voice, and image generation
Watch out for
- Some capabilities are platform-specific, such as macOS Metal/GPU backends and the AMD Strix Halo backend
- The source provides no pricing information
Harbor at a glance
- Website
- github.com/av/harbor
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 6, 2026 · Online when checked
- Listed
- Oct 5, 2026
Harbor alternatives
Tools and sites you could use instead, ranked by how closely they match.
Cog packages machine learning models into containers to support deployment across environments.
Open source
24 alternatives
DJL is a Java framework for running deep learning models across different engines.
Open source
24 alternatives
Kitaru provides agent traces that teams can run and inspect during application workflows.
Open source
24 alternatives
Machine learning operations and large language model operations are the focus of Starwhale’s platform.
Open source
24 alternatives
ONNX-MLIR compiles ONNX models using MLIR compiler infrastructure for model deployment workflows.
Open source
24 alternativesPopular in AI Infrastructure & MLOps
Well-known listings from the same topic.
Guild AIguild.aiFlow AIflow-ai.comsystemprompt.iosystemprompt.ioCoreWeavecoreweave.comKili Technologykili-technology.comAPI docsdocs.x.ai
Harbor FAQ
What happens when I run 'harbor up'?
It starts the services you choose, such as a backend plus Open WebUI, and Harbor handles the Docker Compose orchestration, configuration, and cross-service connections.
Can Harbor be used with coding assistants and agent CLIs?
Yes. 'harbor launch' starts or detects a Harbor OpenAI-compatible endpoint so installed coding and agent CLIs can use it without hand-editing each tool's provider configuration.
Which model backends does Harbor support?
The source names Ollama, llama.cpp, vLLM, macOS host-native Metal/GPU options (dmr, mlx, omlx), a PrismML llama.cpp backend, and a Halogen Flash Server backend for AMD Strix Halo.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.