Mosec serves machine learning models with dynamic batching and CPU or GPU pipelines for inference workloads.
Open source
24 alternativesautobatcher
autobatcher batches asynchronous OpenAI-compatible API requests through a drop-in replacement library.
About autobatcher
autobatcher is an open-source library for Python and TypeScript that provides AsyncOpenAI and BatchOpenAI clients with the same interface as the OpenAI SDK. By default, chat-completion and Responses calls are submitted to /v1/responses with flex and background options, then polled until results return as standard ChatCompletion or Response objects. It handles response IDs, polling, JSONL files, uploads, and result matching internally.
Explicit 24-hour batch mode is available by setting the completion window to "24h"; embeddings always use 24-hour batches because they have no flex tier. The library is designed for the Doubleword Inference API, with support for OpenAI's batch API or other compatible APIs described as best effort.
Key features
- Drop-in AsyncOpenAI and BatchOpenAI clients for Python and TypeScript
- Default flex async polling for chat completions and Responses
- Explicit 24-hour batch mode via completion window setting
- Internal handling of polling, JSONL, uploads, result matching
- Configurable batch size, window, poll interval and concurrency
- Embeddings processed through 24-hour batch lifecycle
Best for: Developers using the Doubleword Inference API who want OpenAI-compatible async and batch calls without managing polling infrastructure.
Good for
- Retains OpenAI SDK interface while managing polling and batch lifecycles internally
- Supports both Python and TypeScript clients
- Returns standard ChatCompletion and Response objects for async calls
Watch out for
- Designed for Doubleword Inference API; other compatible APIs are best effort
- Embeddings always use 24-hour batches with no flex tier option
- Default flex HTTP concurrency is limited to 20 operations per client
autobatcher at a glance
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
autobatcher alternatives
Tools and sites you could use instead, ranked by how closely they match.
Venice AI generates text, images, characters and video, and offers access to AI models through private or anonymized options.
Baidu Qianfan lists pricing and capabilities for models and services on the Baidu AI Cloud platform.
OmniRoute routes requests across hundreds of providers through an OpenAI-compatible endpoint with automatic fallback.
Open source
24 alternatives
The colibri-core NLP tool and library works with n-grams, skipgrams, and other linguistic patterns.
Open source
24 alternatives
LiteLLM provides a way to call more than 100 language models through an OpenAI-compatible interface.
Popular in AI Infrastructure & MLOps
Well-known listings from the same topic.
Guild AIguild.aiFlow AIflow-ai.comsystemprompt.iosystemprompt.ioKili Technologykili-technology.comAPI docsdocs.x.aiEyePop.aieyepop.ai
autobatcher FAQ
How do I select 24-hour batch inference instead of flex?
Set completion_window="24h" in Python or completionWindow: "24h" in TypeScript. Embeddings always use 24-hour batches.
Which APIs is autobatcher designed for?
It is designed for the Doubleword Inference API. Support for OpenAI's batch API or other compatible APIs is best effort.
How do I install autobatcher?
For Python run pip install autobatcher. For TypeScript run npm install autobatcher.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.