The Pile dataset combines 22 smaller datasets for language modeling research.
Dataset
RedPajama-Data-v2 by Together AI
An open dataset with 30 trillion tokens for training Large Language Models.
About RedPajama-Data-v2 by Together AI
RedPajama-Data-v2 is an open dataset that provides a large pool of web text for LLM pre-training research and development. It combines crawled documents with pre-computed quality annotations that developers can use to filter, weight and assemble their own training mixtures. Use requires applying those signals to select suitable subsets.
Pricing is not clearly stated.
Key features
- Over 100 billion text documents with 100+ trillion raw tokens from 84 CommonCrawl dumps
- 30 trillion filtered and deduplicated tokens subset
- Five languages: English, French, Spanish, German, and Italian
- 40+ pre-computed quality annotations for filtering and weighting
- Processing scripts on GitHub with data on HuggingFace
Best for: LLM developers building pre-training datasets
RedPajama-Data-v2 by Together AI at a glance
- Official website
- together.ai/blog/redpajama-data-v2
- Category
- AI & Machine Learning › AI Data & Analytics
- Type
- Dataset
- Platforms
- Web
- Details verified
- Oct 22, 2026 · Official source
- Language
- English
- Last checked
- Oct 5, 2026 · Online when checked
- Listed
- Oct 5, 2026
Similar data sources
Other listings in AI Data & Analytics with the most in common.
Popular in AI Data & Analytics
Well-known listings from the same topic.
Dotgetdot.aiForvioforvio.comFluidAI Medicalfluidai.mdApache Zeppelinzeppelin.apache.orgSola Securitysola.securityQuadraticquadratichq.com
Is something wrong or out of date?
Tell us and an editor will check it.
Own this website?
Claim this listing
Claim the listing and suggest updates yourself. An editor checks every claim first.