Kokoro text to speech

Kokoro-82M serves an OpenAI-compatible /v1/audio/speech endpoint on port 8004, reachable through the gateway as tts-1, tts-1-hd, or kokoro.

curl -X POST https://my-app.the inference gateway/v1/audio/speech \
  -H "Authorization: Bearer $GPU_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"tts-1","voice":"alloy","input":"Hello world"}' \
  -o speech.mp3

OpenAI’s six voice names map onto Kokoro voices (alloy to af_sarah, echo to am_adam, and so on), and Kokoro’s own names work directly if you want one of the other four dozen, including the Japanese, Chinese and French sets. Output is 24 kHz mono. The wav and pcm formats stream as they synthesise; mp3, opus, aac and flac go through ffmpeg and arrive as a single body.

Why this one was easy

Kokoro runs on CPU via onnxruntime. No CUDA, no GPU memory, no interaction with model residency at all. Where the embedding service needed a parity check and a Blackwell-specific torch build before it could move here, this one needed a virtualenv and a systemd unit.

That difference is the useful lesson rather than an incidental detail. When deciding where a service should live, the question that matters is what resource it contends for. A service that wants GPU memory has to be weighed against every model that wants the same memory. A service that wants CPU cycles on a machine whose CPU is mostly idle can be placed almost anywhere, and the placement decision stops being interesting.

Synthesis runs at roughly 3 to 5 times realtime, which is comfortably fast enough that nothing has needed tuning.

Running it

systemctl status kokoro
journalctl -u kokoro -f
curl -s http://localhost:8004/health

/health returns the model version, sample rate, and the full OpenAI voice mapping, which makes it a convenient way to confirm the voice aliases without reading the source.

The first start downloads the ONNX model and voice pack, around 350 MB, from GitHub releases. The unit allows a long start timeout for that reason, and because that download is rate limited along with everything else leaving this VLAN.

Dialect caveat

The Spanish voice is Castilian. For any deployment serving a Mexican-Spanish-speaking population this is the wrong dialect, and the mismatch is obvious to native speakers even though the synthesis quality is fine. Flag Spanish prompts for human recording rather than shipping them. The English voices are good enough to ship as-is.