colibri-core

The colibri-core NLP tool and library works with n-grams, skipgrams, and other linguistic patterns.

github.com/proycon/colibri-core
colibri-core homepage screenshot

About colibri-core

Colibri core is an open-source NLP tool and C++ and Python library for counting and extracting patterns from large corpus data. It handles n-grams (n consecutive words), skipgrams (patterns with gaps of a specific size), and flexgrams (patterns with gaps of variable size). Users can compute statistics on the extracted patterns and relations between them. The software is licensed under GPLv3.

The core command-line tool, colibri-patternmodeller, builds pattern models, generates statistical reports, queries patterns and relations, and manipulates models. The software reduces memory use by storing word types as compact binary classes and by discarding patterns that fall below an occurrence threshold during iterative counting. Skipgram and flexgram extraction are more computationally demanding than n-gram extraction.

Key features

  • Extracts n-grams, skipgrams, and flexgrams from large corpus data
  • Stores word types in a compact binary representation to reduce memory use
  • Iteratively counts patterns and discards those below an occurrence threshold
  • Command-line tool colibri-patternmodeller for building and querying pattern models
  • Generates statistical reports on extracted patterns
  • Computes relations between extracted patterns, available as C++ and Python library
Best for: Researchers and developers doing corpus-based linguistic analysis who need to count and query n-gram, skipgram, or flexgram patterns in large text collections.
Good for
  • Designed to limit memory and disk requirements when processing large corpora
  • Covers n-grams, skipgrams, and flexgrams within one toolset
  • Open source under the GPLv3 license
Watch out for
  • Skipgram and flexgram extraction are computationally more demanding than n-gram extraction

colibri-core at a glance

Type
Open-source project
Pricing
Not confirmed. Check the site
Open source
Yes · Source code
Language
English
Last checked
Oct 6, 2026 · Online when checked
Listed
Oct 5, 2026

colibri-core alternatives

Tools and sites you could use instead, ranked by how closely they match.

All 24 alternatives

Popular in AI Infrastructure & MLOps

Well-known listings from the same topic.

All 341

colibri-core FAQ

What kinds of patterns does Colibri core extract?
It extracts n-grams (n consecutive words), skipgrams (abstract patterns of fixed length with one or more gaps of specific size), and flexgrams (abstract patterns with one or more gaps of variable size).
How does Colibri core keep memory use low?
It assigns each word type a numeric class encoded in a compact binary format, always uses this representation on disk and in memory, and iteratively processes the corpus in several passes, discarding patterns that will not reach the desired occurrence threshold.
Is something wrong or out of date?

Tell us and an editor will check it.

Own this website?

Claim the listing and suggest updates yourself. An editor checks every claim first.

Claim this listing