Polyglot provides multilingual text processing tools for natural language processing.
sentences
The sentences command line tool tokenizes text into sentences in multiple languages using Go.
About sentences
sentences is an open-source command line tool and Go library that splits a block of text into a list of sentences. It is a port of NLTK's punkt tokenizer, using unsupervised multilingual sentence boundary detection: the tokenizer is trained on text in a given language to estimate the likelihood of abbreviations, collocations and sentence starters, which allows it to decide where sentences end.
The project states support for English, Czech, Dutch, Estonian, Finnish, German, Greek, Italian, Norwegian, Polish, Portuguese, Slovene and Turkish, and describes itself as zero-dependency and extendable, with most major components able to be customized. The command line utility loads English by default, and training data for a language must be downloaded from the repository. The author notes that the tokenizer has not been tested in any language other than English.
Key features
- Splits a blob of text into a list of sentences
- Advertised support for 13 languages, including English and German
- Go library with a command line utility
- Port of NLTK's punkt tokenizer using training data
- Zero dependencies and extendable, composable components
- Installable via Homebrew, pre-built binaries, or go get
Best for: Go developers and script users who need sentence segmentation in an application or text pipeline.
Good for
- Open source with pre-built binaries, Homebrew packages and Go installation options
- Multilingual goal with training data included in the repository
- Built around composability, so major components can be extended
Watch out for
- Author states the tokenizer has not been tested in languages other than English
- Training data for a language must be downloaded and loaded separately
- Contributions and maintenance help are requested, indicating limited maintainer capacity
sentences at a glance
- Category
- Development › Programming Languages
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
sentences alternatives
Tools and sites you could use instead, ranked by how closely they match.
The Twitter Text Java library tokenizes and parses text according to platform text requirements.
Open source
24 alternatives
JEnv for Windows switches the active Java version with a short command.
Open source
24 alternatives
Launch Recaf 4 or later with Recaf Launcher, which documents GUI and command-line usage.
Explore GitHub Awesome lists through AwesomeExplorer, a web app for finding repositories and resources.
Open source
24 alternatives
Frog combines memory based NLP modules for Dutch, including tagging, lemmatization, morphology analysis, and dependency parsing.
Open source
24 alternativesPopular in Programming Languages
Well-known listings from the same topic.
Language Benchmarksprogramming-language-benchmarks.vercel.appJavaDecompilerjavadecompilers.comOpenNLPopennlp.apache.orgStandardJSstandardjs.comactnektosact.comRustPythonrustpython.github.io
sentences FAQ
Which languages does it support?
The project lists English, Czech, Dutch, Estonian, Finnish, German, Greek, Italian, Norwegian, Polish, Portuguese, Slovene and Turkish.
How is it installed?
Options listed are Homebrew taps for macOS, pre-built binaries from the GitHub releases page, or go get and go install with Go.
What is required to tokenize text?
The tokenizer is trained on language text, so training data files from the repository's data folder must be downloaded and loaded; the command line utility loads English by default.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.