By Shadrack John (Shadrackovsky), Antera Group.
Ask ChatGPT, Gemini, or Claude to write a poem in English, and you'll get something fluent, idiomatic, and structurally sound. Ask the same model to write a poem in Swahili-a language spoken by well over 200 million people across East Africa-and the cracks start to show: stilted grammar, mistranslated idioms, and a noticeable drift toward English sentence structure dressed up in Swahili words. This isn't a minor bug. It's a direct, measurable consequence of how large language models are built, and it points to a much bigger structural problem: the near-total absence of African languages from the data pipelines that power modern AI.

This post from ANTERA breaks down, technically, what "low-resource" actually means in NLP, why Swahili specifically struggles even though it's one of Africa's most digitally documented languages, and what the research community is doing to close the gap.

Table of Contents

1. The Scale of the Problem: 2,000 Languages, One Data Pipeline

2. What "Low-Resource" Actually Means in NLP

   • 2.1. Corpus Scarcity: The Common Crawl Problem

   • 2.2. The Tokenization Tax: Fertility and Subword Fragmentation

   • 2.3. Benchmark Blindness: We Can't Measure What We Don't Test

3. Why Swahili Specifically Struggles

   • 3.1. Agglutinative Morphology vs. Byte-Pair Encoding

   • 3.2. Code-Switching: KiswaEnglish and the Limits of Monolingual Training

   • 3.3. Script, Dialect, and Orthographic Inconsistency

4. The Downstream Consequences

   • 4.1. Higher Cost, Slower Inference

   • 4.2. Degraded Reasoning and Hallucination

   • 4.3. Cultural Flattening

5. What's Being Done: Community-Led Solutions

   • 5.1. Masakhane and the African Languages Hub

   • 5.2. Small, Specialized Models: Lelapa AI's InkubaLM

   • 5.3. Vocabulary Adaptation and Syllable Tokenization Research

6. Building for the Gap: A Technical Roadmap

7. Conclusion: Closing the Gap From the Inside

1. The Scale of the Problem: 2,000 Languages, One Data Pipeline

Africa is home to roughly a third of the world's living languages-more than 2,000 by most linguistic counts. Yet recent surveys of LLM language coverage found that only a small handful of African languages (commonly Swahili, Amharic, Afrikaans, and Malagasy) receive consistent support across major models, while the overwhelming majority-well over 95%-remain effectively unsupported in any systematic way. Of the world's most-used internet languages, none are African, despite the continent's linguistic scale and its rapidly growing base of internet users.

This isn't a coincidence of neglect; it's an artifact of how foundation models are trained. Pretraining corpora are built primarily by scraping the web, and the web itself reflects decades of unequal digitization: English, Chinese, and a handful of European languages dominate online text, while African languages-many historically oral, multilingual within single communities, and under-documented in formal writing-are a rounding error in the training mix.

2. What "Low-Resource" Actually Means in NLP

"Low-resource" gets used loosely, but in NLP it has a precise technical meaning: a language for which there isn't enough digitized, labeled, or naturally occurring text to train or fine-tune a model to the same standard as high-resource languages like English. This scarcity shows up at three distinct layers of the pipeline.

2.1. Corpus Scarcity: The Common Crawl Problem

Most foundation models are pretrained on some derivative of Common Crawl-a periodic snapshot of the public web. Because Common Crawl simply reflects what exists online, and what exists online is skewed heavily toward high-resource languages, Swahili and its neighbors end up contributing a tiny fraction of a percent of total training tokens, even in "multilingual" models that claim support for 100+ languages. A model might technically include Swahili in its training mix while still only ever seeing it a few hundred million times fewer than English-nowhere near enough to build robust grammatical or semantic representations.

2.2. The Tokenization Tax: Fertility and Subword Fragmentation

Before a model ever sees a word, it has to be broken into tokens-and this is where low-resource languages take their first real hit. Most LLMs use subword tokenizers (BPE, SentencePiece, WordPiece) trained on corpora dominated by high-resource languages. The result is a vocabulary that represents common English words as a single token, while morphologically rich or underrepresented languages get chopped into far more fragments per word-a phenomenon researchers call tokenizer "fertility." Studies measuring this effect have found tokenization penalties of several times more tokens per word for underrepresented languages compared to English, and in extreme cases, an order of magnitude more.

This isn't just an efficiency footnote. Higher fertility means longer sequences for the same sentence, which increases inference cost and latency, eats into the model's effective context window faster, and-critically-degrades generation quality, since the model has to reconstruct meaning across more, smaller, noisier fragments instead of clean semantic units.

2.3. Benchmark Blindness: We Can't Measure What We Don't Test

The third layer is arguably the most insidious: most standard LLM benchmarks (MMLU, Winogrande, HellaSwag) were built for English and have only recently been translated into a handful of African languages, often by small research teams working with limited budgets. This means that for most of the training and evaluation cycle, model developers have had almost no systematic way to measure how badly their models perform on Swahili, let alone Bambara, Igbo, or Setswana-so the gap persists unmeasured, and therefore unaddressed, release after release.

3. Why Swahili Specifically Struggles

Swahili is, on paper, one of the better-off African languages: it has an official Latin-script orthography, a long history of written literature, status as a lingua franca across Tanzania, Kenya, Uganda, and beyond, and dedicated efforts from major labs to include it in multilingual models. And yet it still underperforms dramatically compared to high-resource languages. The reasons are structural, not just about data volume.

3.1. Agglutinative Morphology vs. Byte-Pair Encoding

Swahili is a Bantu language with rich agglutinative morphology: a single verb can encode subject, tense, object, and negation all as prefixes and infixes around a root, producing word forms that don't exist as standalone units anywhere in a typical training corpus. Standard subword tokenizers, designed around the statistical patterns of English and other Indo-European languages, handle this poorly-they either over-fragment Swahili words into meaningless pieces or fail to generalize across the huge space of possible inflected forms. Recent research proposing syllable-based tokenization specifically for Swahili demonstrates that rethinking tokenization at the linguistic level-rather than just adding more Swahili text to a generic tokenizer-can meaningfully improve how well embeddings capture the language's structure.

3.2. Code-Switching: KiswaEnglish and the Limits of Monolingual Training

In real-world use, especially online and in messaging, Swahili speakers rarely write in "pure" Swahili. Code-switching between Swahili and English-sometimes called KiswaEnglish-is the norm in casual digital communication, mixing English loanwords, sentence fragments, and grammar mid-sentence. Almost no mainstream training corpus is built to represent this properly, because most data pipelines classify text by a single dominant language and either discard or mislabel mixed-language content. A model trained overwhelmingly on "clean" monolingual Swahili text (much of it, ironically, religious or governmental in origin) ends up poorly equipped for the actual, messy, code-switched language its users speak day to day.

3.3. Script, Dialect, and Orthographic Inconsistency

Even within "clean" Swahili text, there's meaningful variation: regional dialects, inconsistent spelling conventions, and loanword spellings that shift between Arabic-influenced and English-influenced forms depending on region and era. Without large, carefully curated corpora that capture this variation, models trained on whatever Swahili text happens to be scraped end up overfit to a narrow register-often formal, religious, or news-style text-while struggling with informal, conversational, or regionally distinct language.

4. The Downstream Consequences

4.1. Higher Cost, Slower Inference

Because of tokenizer fertility, the same sentence in Swahili can cost meaningfully more to process than its English equivalent on any token-billed API-a real economic barrier for African startups and users trying to build or use AI products in their own language.

4.2. Degraded Reasoning and Hallucination

Sparse training data combined with fragmented tokenization compounds into worse downstream performance on everything from factual recall to multi-step reasoning. Recent benchmarking work translating standard reasoning and knowledge tests into low-resource African languages has consistently found significant, previously unmeasured performance gaps between state-of-the-art models' English and African-language capabilities-gaps that don't show up at all if you only ever evaluate in English.

4.3. Cultural Flattening

Perhaps the least discussed consequence is cultural. Because most Swahili training data over-represents certain domains (religious texts, government communications, news translations), models risk reflecting a narrow, skewed slice of the culture rather than the full range of how the language is actually used and understood-and risk defaulting to Anglo-centric framing even when generating Swahili text, since the underlying reasoning is still happening in a representation space shaped overwhelmingly by English.

5. What's Being Done: Community-Led Solutions

The encouraging part of this story is that the response isn't coming primarily from Silicon Valley-it's coming from African researchers and grassroots organizations who understand the problem from the inside.

5.1. Masakhane and the African Languages Hub

Masakhane-whose name means "we build together" in isiZulu-is a grassroots, pan-African NLP research community explicitly organized around the principle that African language technology should be built by Africans, for Africans, rather than treating African communities as passive data sources for external labs. Its African Languages Hub has since evolved into a larger, better-funded effort: in 2026, Masakhane partnered with Microsoft's AI for Good Lab, Google.org, and the Gates Foundation to launch LINGUA Africa, an open call for proposals offering funding and cloud compute credits to projects building open datasets, models, and real-world applications across African languages, with the explicit goal of reaching a billion Africans with locally relevant AI tools by 2029.

5.2. Small, Specialized Models: Lelapa AI's InkubaLM

Rather than chasing ever-larger general-purpose models, South African AI company Lelapa AI took a different approach with InkubaLM: a small, compact language model purpose-built for a handful of African languages, paired with dedicated training datasets. The core bet is that a smaller model trained carefully on well-curated, language-specific data can match or exceed the performance of much larger multilingual models on the languages it targets-while being dramatically cheaper to train, fine-tune, and run, which matters enormously in resource-constrained African markets.

5.3. Vocabulary Adaptation and Syllable Tokenization Research

On the technical research side, multiple groups are attacking the tokenization problem directly. Work on vocabulary augmentation shows that adding even a modest number of language-specific tokens to an existing multilingual tokenizer can bring fertility for a low-resource language close to parity with English. Separately, researchers working specifically on Swahili have proposed syllable-based tokenization as an alternative to generic subword tokenizers, showing it produces embeddings that better capture the language's actual structure. Neither approach alone solves the underlying data scarcity problem, but together they represent a meaningful shift from "add more raw text and hope" to deliberately engineering tokenizers and training pipelines around the linguistic realities of the languages being served.

6. Building for the Gap: A Technical Roadmap

For developers and researchers-especially those building from within Tanzania and East Africa-closing this gap is less about waiting for big labs to prioritize African languages and more about a set of concrete, achievable technical moves:

• Custom tokenizers over generic ones. A tokenizer trained on or adapted to Swahili's morphology (whether via vocabulary augmentation or syllable-aware methods) will outperform reusing a generic multilingual tokenizer, even before touching the model weights.

• Curate for register diversity, not just volume. A smaller, deliberately diverse corpus spanning formal, conversational, and code-switched Swahili will generalize better than a larger corpus that's mostly religious or government text.

• Treat code-switching as a first-class case, not noise. Since KiswaEnglish is how people actually communicate, filtering it out of training data guarantees a model that fails in real deployment.

• Fine-tune smaller models rather than fighting for representation in giant ones. As InkubaLM demonstrates, a well-tuned smaller model on curated data can be more practical-and often more accurate-than trying to improve Swahili performance inside a massive general-purpose model you don't control.

• Build and share evaluation benchmarks. Without a way to measure Swahili performance specifically, it's impossible to know if any of the above interventions are actually working-benchmark contribution is as valuable as model-building.

7. Conclusion: Closing the Gap From the Inside

The underrepresentation of Swahili and other African languages in mainstream LLMs isn't a mysterious oversight-it's the predictable output of training pipelines built around web-scraped data, generic tokenizers, and benchmarks designed for high-resource languages. But the technical nature of the problem is also what makes it solvable: better tokenization, better-curated corpora, and purpose-built smaller models can close a meaningful share of the gap without needing to out-compete trillion-token frontier models at their own game.

For a continent with over a third of the world's languages and a fast-growing digital population, this isn't a niche research interest-it's foundational infrastructure. The organizations and researchers doing this work today, from Masakhane's community-driven data efforts to Lelapa AI's specialized models, are laying the groundwork for African languages to move from an afterthought in global AI to a first-class citizen in it.