TextCL
TextCL provides preprocessing tools for cleaning text data used in natural language processing tasks.
About TextCL
TextCL is an open-source Python package that cleans text data for use in natural language processing tasks, including text analysis and predictive, classification, or text generation models. It splits texts into sentences, removes sentences not in the target language, removes linguistically disconnected sentences such as OCR noise using perplexity filtering, removes duplicate sentences using Jaccard similarity, and provides unsupervised outlier detection for off-topic texts.
It requires Python >= 3.8 with listed dependencies including pytorch_pretrained_bert, langdetect, numpy, pandas, lxml, protobuf, and nltk. It can be installed from PyPI or from source and is distributed under the MIT License.
Key features
- Split texts into sentences
- Filter sentences not in target language
- Remove disconnected sentences via perplexity filtering
- Remove duplicate sentences using Jaccard similarity
- Detect off-topic texts with unsupervised outlier detection
- Offer TONMF, RPCA, and SVD outlier detection methods
Best for: Python developers preparing noisy or mixed-topic text data for NLP models.
Good for
- Open source under MIT License
- Installable from PyPI and from source with pip
- Combines sentence splitting, language, perplexity, duplicate, and outlier filtering
Watch out for
- Requires Python 3.8 or newer with listed dependencies
TextCL at a glance
- Category
- Development › Developer Tools
- Type
- Open-source project
- Pricing
- Not confirmed. Check the site
- Open source
- Yes · Source code
- Language
- English
- Last checked
- Oct 6, 2026 · Online when checked
- Listed
- Oct 5, 2026
Guides for Developer Tools
TextCL alternatives
Tools and sites you could use instead, ranked by how closely they match.
Segment Chinese text with jieba, a Python package for natural language processing.
Open source
24 alternatives
torchaudio provides data manipulation and transformation tools for audio signal processing with PyTorch.
Open source
24 alternatives
spaCy provides Python natural language processing tools for entity recognition, tagging, parsing, and word vectors.
Free
24 alternatives
LlamaIndex provides a document processing platform for AI applications.
Open source
24 alternatives
The LlamaIndex repository describes a document processing platform for AI applications.
Open source
24 alternatives
Gitingest turns a GitHub repository URL into a text digest suited to language model prompts.
Open source
24 alternativesPopular in Developer Tools
Well-known listings from the same topic.
AppDevToolsappdevtools.comCommand Not Foundcommand-not-found.comTestFlighttestflight.apple.com/join/A8GwCnq8Sourcegraphabout.sourcegraph.comProject Wallaceprojectwallace.comCarbon Calculatorwebsitecarbon.com
TextCL FAQ
How do I install TextCL?
From PyPI with pip install textcl, or from source with pip install git+https://github.com/alinapetukhova/textcl.git#egg=textcl.
What is required to use TextCL?
Python >= 3.8 with pytorch_pretrained_bert, langdetect, numpy, pandas, lxml, protobuf, and nltk.
What license does TextCL use?
MIT License.
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.