iliasaz/chatterbox-nano-coreml

Chatterbox Nano — CoreML (ANE-optimized)

On-device Apple Silicon build of ResembleAI Chatterbox Nano for iOS 18+ / macOS 15+. Prefill + decode run fully on the Apple Neural Engine as one weight-shared multifunction CoreML model; the audio synth runs as three further CoreML packages.

Links

• Upstream model — ResembleAI/chatterbox-nano by Resemble AI.
• Runtime — iliasaz/chatterbox-coreml, the Swift package + demo app that loads this repo.
• Watermarking — iliasaz/perth-coreml (Swift) / weights. Every utterance the runtime generates is watermarked with Perth, as upstream chatterbox does.

What Nano is

Nano is Chatterbox Turbo with a smaller language model — upstream's tts_nano.py is byte-identical to tts_turbo.py except for the backbone (GPT2_small vs GPT2_medium) and the T3 checkpoint. Everything downstream is the same: the S3Gen MeanFlow synth (2 CFM timesteps), the tokenizer, the conditioning, and the sampler.

	Nano	Turbo
T3 backbone	GPT-2 small — 12 layers / 768 hidden / 12 heads	GPT-2 medium — 24 / 1024 / 16
T3 params	217 M	~430 M
T3LM.mlpackage	120 MB (8-bit palettized)	328 MB (8-bit palettized)
S3 synth stack	identical (bit-for-bit same upstream weights)	—

Artifacts

File	What	Backend
T3LM.mlpackage	T3 prefill + decode, multifunction (q=512 prefill, q=1 decode, shared MLState KV cache)	CoreML (ANE)
S3Encoder / S3CFM / S3Vocoder .mlpackage	speech-token → mel → 24 kHz waveform (MeanFlow CFM, 2 timesteps)	CoreML (ANE/GPU)
CAMPPlus / MatchaMel / VEMel / VELSTM / S3Tokenizer .mlpackage	voice-cloning encoders	CoreML
text_emb.npy, speech_emb.npy	embedding tables (host-side lookups), 768-wide	—
spkr_enc_weight.npy, spkr_enc_bias.npy	speaker conditioning linear (256→768)	—
default-conds.safetensors, tokenizer files	supporting artifacts	—

The four .npy tables are hidden-width-specific. The Swift runtime checks them against the loaded graph at load time and refuses to run on a mismatch — pairing Turbo's 1024-wide tables with this 768-wide graph would otherwise produce garbage audio silently.

On-device numbers (iPhone 17 Pro Max, iOS 26.5)

Measured on device against a Python-CoreML-CPU baseline, same phone and harness, Turbo shown for reference:

	Nano	Turbo
prefill predict (q=512, ANE)	22 ms	64 ms
prefill ANE parity (cos_sim vs CPU)	0.9996	0.9998

Both are 8-bit palettized (k-means, per_grouped_channel, group 16). Nano's wpe and speech_head stay fp16 — speech_head's 6563 output channels are prime, so no group size divides them.

Mac (M5, Release CLI, same sentence): Nano generates 4.16 s of audio in 0.65 s (≈6.4× realtime); Turbo 4.00 s in 1.02 s (≈3.9× realtime). The sampler is nondeterministic, so utterance length varies run to run — read these as a rough ratio, not an exact benchmark.

Pipeline

text → BPE → host-assemble inputs_embeds + position_ids + masks
     → T3LM "prefill" function (CoreML, ANE)  — writes shared KV state
     → T3LM "decode" function (CoreML, ANE)   — autoregressive, reads/writes same state
     → S3Encoder → S3CFM → S3Vocoder (CoreML) — 24 kHz waveform

All attention masks (attn_mask, write_mask, decode update_mask) are built on the Swift host and passed in, and the prefill's attention is hand-decomposed rather than fused — on iPhone the ANE's fused attention kernel silently drops the attn_mask for q_len >> 1.

License

MIT. These weights are a CoreML format conversion of ResembleAI/chatterbox-nano, which Resemble AI publishes under the MIT licence; the conversion inherits those terms, and copyright in the weights stays with Resemble AI — no claim of ownership is made over them here. Redistribute with attribution to Resemble AI.

The Swift runtime that loads them is separately MIT-licensed; see the LICENSE and NOTICE files in iliasaz/chatterbox-coreml.