imara learning library ← all courses
Voice AI · lightweight TTS · 6 modules

Local text-to-speech on hardware you can afford

Three open-source voices that run offline on cheap machines — Piper (the efficiency king), Kokoro (the quality-to-size sweet spot), and XTTS (voice cloning). Their architectures, how to tune them for Tanzanian Swahili, what hardware they need, and how to deploy them for next to nothing — with the licensing reality that decides which you can actually ship.

3
models: Piper, Kokoro, XTTS
~60MB
smallest deployable voice (Piper)
1
of the three you can't ship commercially
Pi
runs on a Raspberry Pi, no cloud
Module 01choosing a voice

The lightweight TTS landscape

The Voice AI course used Chatterbox. But when you need a voice that runs offline on a Raspberry Pi, a cheap CPU box, or inside a USSD/IVR gateway with no GPU, three smaller models dominate — and they sit at very different points on the speed–quality–cloning triangle.

The three at a glance

 Piper ⚡Kokoro 🎯XTTS v2 🎭
StrengthExtreme speed, tinyQuality-to-sizeVoice cloning, expressive
ArchitectureVITSStyleTTS2 (82M)Autoregressive codec LM (~0.5B)
Size~60–100 MB~310 MB~1.8 GB
Runs onCPU, Raspberry PiCPU, low-end GPUGPU (4–6 GB VRAM)
CloningNoLimited (blending)Zero-shot (~6 s)
LicenseGPL-3.0Apache-2.0CPML
Commercial useYes (copyleft)YesNo
Read this before you fall in love with XTTS

XTTS has the best cloning, and it's the one you cannot use in a commercial product. Its weights are under the Coqui Public Model License (CPML), which is non-commercial only — and because Coqui shut down in January 2024, there is no longer anyone to sell you a commercial licence. Any fine-tune you make inherits that restriction. For a lender shipping a product, that leaves Kokoro (Apache-2.0) and Piper (GPL-3.0) as the real choices. XTTS is for prototyping and internal demos only.

Pick the right one for your constraints

Toggle what matters for your use case and see which model survives:

Model chooser · click your requirements
"Commercial use" instantly eliminates XTTS. "CPU / Raspberry Pi only" favours Piper, then Kokoro. "Instant voice cloning" needs XTTS — which collides with commercial use, the tension at the heart of open TTS today.
Module 02the efficiency king

Piper

If the target is a Raspberry Pi, a USSD/IVR box, or any CPU-only machine, Piper is the default. It's tiny, instant, fully offline, and deterministic — the same text always produces the same audio. Voices sound a little flat, but they never surprise you.

Architecture: VITS

Piper is built on VITS — a single end-to-end network that goes straight from phonemes to a waveform, with no separate vocoder stage. That compactness is why a full voice is ~60–100 MB and runs faster than real time on a single CPU thread. It exports to ONNX, a portable runtime format, so inference needs no Python or GPU at all.

The phoneme front-end: espeak-ng

Before Piper synthesises anything, it converts text to phonemes (mouth-sound instructions) using espeak-ng. This matters hugely for us: espeak-ng already supports Swahili (sw), so the language front-end is largely solved out of the box — a real head start over Kokoro (Module 3).

The honest limits

Why Piper fits the inclusion mission

A ~60 MB voice that runs offline on a Raspberry Pi with no GPU and no cloud call is exactly right for low-connectivity, low-cost deployments — an IVR line that answers in Swahili with no per-minute cloud TTS bill. For many of your use cases this is the pragmatic winner.

Module 03the sweet spot

Kokoro

Kokoro is the model that made the open-source community stop and stare: at just 82 million parameters it produces audio that rivals cloud providers, under a clean Apache-2.0 licence. For a commercial product that wants to sound genuinely natural without a GPU bill, it's the strongest choice.

Architecture: StyleTTS2

Kokoro is built on the StyleTTS2 design: a PL-BERT text encoder, a prosody predictor that decides rhythm and intonation, and an ISTFTNet decoder that produces the waveform — all decoder-only, no diffusion. That prosody predictor is why it sounds so much less robotic than Piper: it actively models the music of speech, not just the phonemes. Remarkably, it was trained on under 100 hours of audio.

The phonemizer catch

Kokoro uses its own G2P (grapheme-to-phoneme) library, misaki, which is optimised mainly for English and Chinese and falls back to espeak-ng for other languages. This is the main hurdle for Swahili: you'll likely need to route Swahili through espeak-ng (which Piper already does well) or map Swahili letters directly to IPA phonemes. Swahili is highly phonetic — you say it as it's written — so a small, explicit mapping (ch→tʃ, sh→ʃ, ny→ɲ, clean five-vowel values) gets you most of the way. More on this in Module 5.

Tuning paths

The pick for a natural commercial Swahili voice

Apache-2.0 (ship it freely), 82M params (runs on CPU and low-end GPUs), and markedly more natural than Piper. If the voice is customer-facing and you want it to sound warm rather than robotic, Kokoro is the one to invest the phonemizer work in.

Module 04the cloner

XTTS v2

XTTS is the most capable of the three and the one with the catch. It clones a voice from about six seconds of reference audio, speaks 17 languages, and captures cadence and emotion — but its licence makes it a prototyping tool, not a product component, for anyone commercial.

Architecture: an autoregressive codec LM

XTTS works like a language model, but for audio. It predicts a sequence of audio codec tokens one at a time, conditioned on your text and a short reference clip — the same autoregressive idea behind the end-to-end speech models, applied just to synthesis. That's what gives it zero-shot cloning and expressiveness: it has learned how reference voices map to speech, so it can imitate a new one on the fly. It's also why it's heavier (~0.5B parameters, ~4–6 GB VRAM) and occasionally hallucinates — mumbling or repeating words — which a deterministic model like Piper never does.

The licence — the thing that decides it

Non-commercial, with no way to buy out

XTTS v2 is under the Coqui Public Model License (CPML): personal, research, and hobby use only. Coqui Inc. shut down in January 2024, so there is no entity to purchase a commercial licence from. Fine-tuned derivatives inherit the restriction. In plain terms: you may prototype and evaluate with XTTS, but you cannot ship it in a revenue-generating product. Maintained community forks (e.g. at Idiap) keep the code alive, but the weights' licence is unchanged.

Where it still earns its place

Module 05making it speak Tanzanian

Tuning for Tanzanian Swahili

Kiswahili cha Tanzania has its own prosody, rhythm and vowel colour, distinct from Kenyan or textbook Swahili. Teaching a model that comes down to three pillars — the data, the phonemizer, and the training run — plus one concept worth clearing up first.

"Full dataset retraining" vs zero-shot — what it means

XTTS clones a voice from a clip at runtime because it learned cloning. Piper and Kokoro can't guess: to make them speak with a specific voice or dialect, that identity must be baked into the network by fine-tuning. You don't train from scratch — you take an existing checkpoint and keep training it on your Swahili dataset until it morphs to the new voice and accent. A few hours on a GPU; the payoff is a permanent, tiny, offline voice.

Pillar A — the dataset

metadata.csv
audio_001.wav|Habari za asubuhi, unahitaji msaada gani leo?
audio_002.wav|Tanzania ni nchi nzuri yenye amani.
audio_003.wav|Umefanikiwa kulipa mkopo wako. Asante sana.

Pillar B — the phonemizer

Text becomes phonemes before the model ever sees it, and this is where the three models differ most. Try the step on a Swahili phrase:

Grapheme → phoneme · the Swahili front-end
Swahili is written as it's spoken, so the mapping is clean: ch→tʃ, sh→ʃ, ny→ɲ, ng'→ŋ, and five pure vowels. Piper's espeak-ng does this for free; for Kokoro you supply a small map like this.

Add rules for local terms the phonemizer won't know — acronyms like NIDA, TRA, CRDB, or place and product names — so they're pronounced correctly rather than spelled out.

Pillar C — the training run, per model

 PiperKokoroXTTS (non-commercial)
MethodFine-tune VITS checkpointkikiri-tts (StyleTTS2) recipeFine-tune / expand tokenizer
Data1–2 hr1–5 hrClip (clone) or hours (tune)
Train GPULow (Colab)Medium8–12 GB VRAM
Swahili front-endespeak-ng has swBuild a G2P mapExpand tokenizer
ResultClear, stable, flatNatural, expressiveExpressive, cloned
piper fine-tune — the shape
# 1. preprocess: pair text -> phonemes (espeak-ng, lang sw)
python -m piper_train.preprocess --language sw \
   --input-dir ./tz_swahili --output-dir ./dataset \
   --sample-rate 22050

# 2. fine-tune FROM an existing checkpoint (not from scratch)
python -m piper_train --dataset-dir ./dataset \
   --resume_from_checkpoint base_voice.ckpt \
   --max_epochs 2000     # ~1000+ epochs on top of the base

# 3. export the tiny deployable artifacts (~60 MB)
python -m piper_train.export_onnx model.ckpt tz_sw.onnx
# ship tz_sw.onnx + tz_sw.onnx.json to the Raspberry Pi
The no-regret asset

Whichever model you pick, the clean, transcribed Tanzanian Swahili recordings are the real investment — and the same dataset fine-tunes Piper and Kokoro, trains your Chatterbox voice from the Voice AI course, and even seeds an end-to-end model later. Record once, reuse everywhere.

Module 06running it for almost nothing

Deploying on inexpensive hardware

The whole point of these models: once trained, they run offline on hardware you already have — no GPU, no cloud, no per-minute bill. Here's how to size it, and the one number that tells you whether it'll keep up.

The number that matters: real-time factor (RTF)

RTF = processing time ÷ audio duration. RTF below 1.0 means the model generates speech faster than it plays — essential for a live phone call. Piper on a Raspberry Pi sits comfortably below 1; XTTS on a CPU does not. Pick a model and a box and check:

Will it run? · model × hardware → RTF
Illustrative. Green = comfortably real-time; amber = works but tight; red = not viable for live calls. Notice XTTS needs a GPU to be real-time at all.

Making it cheaper and faster

The recommendation for us

Where each lands

Piper for offline, CPU-only, high-volume info delivery (IVR, reminders) where a clear voice beats a warm one — ~60 MB, Raspberry-Pi-friendly, Swahili front-end already there. Kokoro when the voice is customer-facing and must sound natural, and you can do the phonemizer work — Apache-2.0, so safe to ship. XTTS only for internal prototypes, never in the product, because of the licence. And the dataset you build serves all of them.

You've seen the whole lightweight stack

From the three models and their architectures, through the licence that quietly decides which you can ship, how to tune each for Tanzanian Swahili, and how to run them on a Raspberry Pi or a cheap CPU box for next to nothing — you can now choose, adapt, and deploy a local voice that fits your hardware, your language, and your mission.