A Rust inference engine for GGUF models, Shimmy uses WebGPU and supports the OpenAI API.
Open source
22 alternativeslibonnx
A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.
About libonnx
libonnx is an open source inference engine written in pure C99 for running ONNX machine learning models. The library is designed to be lightweight and portable and is aimed at embedded devices. Its source files can be added directly to a C project and compiled with it. A context is created from a model file, input and output tensors are located by name, and inference is run into the output tensor.
It supports hardware acceleration through an array of resolver structures passed when the context is allocated. The README does not name which accelerators are supported. It is based on ONNX v1.17.0 with opset 24 support, but some operators are not implemented. The supported operator list is in the documents directory. The listed platform is Linux, and the README includes a cross-compilation example for arm64. Running the bundled tests on model folders other than the example set may not succeed.
Key features
- Pure C99 source files that can be compiled directly into a project
- Loads ONNX models from a file or from a compiled byte array
- Accepts an array of resolver structures for hardware acceleration
- Makefile builds a static library, examples, and tests
- Cross-compilation example for arm64 using a toolchain prefix
- Based on ONNX v1.17.0 with opset 24 support; operator list in documents directory
Best for: C developers who need to run ONNX models inside embedded or bare-metal projects.
Good for
- Released under the MIT license and publicly available as open source
- Source files can be dropped into an existing C project without a separate runtime
- Documents a cross-compilation workflow for arm64 targets
Watch out for
- Some ONNX operators have not been implemented
- Tests run on model folders other than the bundled examples may not succeed
- The mnist example's graphical interface requires SDL2 and SDL2 GFX to be installed
libonnx at a glance
- Website
- github.com/xboot/libonnx
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Platforms
- Linux
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 6, 2026 · Online when checked
- Listed
- Oct 5, 2026
libonnx alternatives
Tools and sites you could use instead, ranked by how closely they match.
NVIDIA Triton Inference Server provides an inference solution for deploying models in cloud and edge environments.
Open source
23 alternatives
Sonar provides large-scale inference for language models through the project hosted in this repository.
Open source
23 alternatives
Deploy generative and predictive AI models on Kubernetes with KServe’s distributed inference platform.
Open source
21 alternatives
KServe provides distributed inference for generative and predictive AI models on Kubernetes.
Open source
21 alternatives
Jina Serve helps developers build multimodal AI applications with cloud-native tools.
Open source
21 alternativesPopular in Model Serving & Inference
Well-known listings from the same topic.
MAI Playgroundplayground.microsoft.ai/chatBigModelbigmodel.cnkimi-k2.6platform.kimi.ai/docs/pricing/chat-k26Fluxgithub.com/black-forest-labs/fluxprivateGPTgithub.com/imartinez/privateGPTTNNgithub.com/Tencent/TNN
libonnx FAQ
How can a model be used without loading it from a file?
The README suggests converting the .onnx file into a byte array with xxd -i on Linux, then using onnx_context_alloc with that array. The hello and mnist examples load models this way.
Which operators are supported?
The supported operator table is in the documents directory of the repository. The README notes that some operators have not been implemented.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.