Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation Pipeline

Native-script data for 22 languages, romanized data for twelve, and a reusable GPU-accelerated pipeline for translation, quality assessment, and repair.