hviske-v5-tiny

hviske-v5-tiny is a 263M-parameter Danish ASR model distilled from the syv-transcribe ensemble (~2.1B). About 8× smaller than its teachers while staying competitive with them, and roughly 2× faster than any 2B model on the Danish ASR leaderboard.

Audio is expected at 16 kHz mono, clips up to 35 s. Danish only, offline transcription (no timestamps, diarization, or streaming).

Try it live in your browser — live dictation with VAD-based streaming, running entirely client-side on WebGPU (onnxruntime-web + a transformers.js tokenizer). Nothing is uploaded; the model is downloaded once and cached.

Prefer not to run it yourself? The model is also available as a hosted API at platform.syv.ai.

Accuracy

Measured with the leaderboard harness, unmodified (--backend cohere-asr), on a single RTX 3090.

Dataset	WER	CER
CoRal conversation	26.07	15.97
CoRal read-aloud	14.76	6.00
Common Voice 17 (da)	9.89	3.57
FLEURS (da)	11.32	4.49
FTSpeech	7.15	3.84
Mean	13.84	6.77

FTSpeech 7.15 is the best score on the leaderboard at the time of submission, ahead of the 2.1B syv-transcribe ensemble (8.02).

Speed

RTFx = audio seconds transcribed per wall-clock second. "Single" is one clip at a time (the interactive case); "batched" is the best measured batched/concurrent configuration for that runtime. Where clip length matters the batched cell shows both: longer clips amortise per-request cost, so 30 s audio yields a higher RTFx than the 11 s FLEURS average.

Hardware	Runtime	Single	Batched	Batch config	WER
RTX 3090	custom CUDA kernels (cuda/)	320×	3833× (11 s) · 4341× (30 s)	batch 64	11.47
RTX 3090	vLLM 0.19.0 + CUDA kernels	182×	2646× (11 s) · 3587× (30 s)	512 concurrent	11.21
RTX 3090	vLLM 0.19.0 stock	176×	1783× (11 s) · 2936× (30 s)	256–384 concurrent	11.27
RTX 3090	PyTorch bf16 (leaderboard harness)	158×	323×	batch 8	11.32
Apple M4 (base)	MLX int4	87×	93×	batch 8	10.60†
Apple M4 (base)	MLX int8	63×	110×	batch 8	10.60†
Apple M4 (base)	MLX fp16	39×	78×	batch 8	10.46†
Apple M4 (base)	GGUF q4_k via CrispASR (Metal)	56×	—	sequential CLI	10.51†
Apple M4 (base)	GGUF q8_0 via CrispASR (Metal)	45×	—	sequential CLI	10.44†
Apple M4 (base)	ONNX Runtime CPU, 2 threads	31×	—	batch axis fixed at 1	10.55†
Apple M4 (base)	GGUF q4_k via CrispASR (CPU, 8 threads)	24×	—	sequential CLI	10.88†
Apple M4 (base)	PyTorch CPU fp32	11×	—		11.03†
x86 CPU, 4 threads	PyTorch fp32	3.2×	—		—

WER is FLEURS-da (lowercase, punctuation-stripped, same normaliser as the leaderboard). RTX rows are the full 930-clip test set; † rows are the 200-clip subset used in the per-build sections below, which skews ~0.8 lower because its clips are shorter — compare within a group, not across. The spread inside each group (±0.1–0.2) is quantisation/bf16 noise, not a real accuracy difference between runtimes.

Notes on the dashes: the CrispASR CLI processes files sequentially (passing many files in one invocation only amortises model load), and the published ONNX graphs were exported with the batch axis fixed at 1, so neither supports true batching today. Both are fixable — the ONNX one just needs a re-export with a dynamic batch axis.

Batching pays off very differently per backend: 12× on the 3090 with the custom kernels, 9–15× under vLLM, but only 1.1–2× on a base M4, where a single stream already keeps the GPU busy.

The Apple rows were measured back-to-back within each build, but the M4 had other work running and its throughput moves by roughly ±25% with machine load (the MLX int4 single figure measured as high as 139× on an idle machine, 73–87× loaded). Treat all Apple numbers as a band, not a point; per-build details are in the MLX, ONNX and GGUF sections below.

Usage

All paths below are self-hosted. For a managed option, the model is served through the hosted API at platform.syv.ai.

PyTorch

import soundfile as sf
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    "syvai/hviske-v5-tiny", trust_remote_code=True).eval()
processor = AutoProcessor.from_pretrained("syvai/hviske-v5-tiny", trust_remote_code=True)

audio, sr = sf.read("clip.wav", dtype="float32")
text = model.transcribe(processor=processor, language="da",
                        audio_arrays=[audio], sample_rates=[sr])[0]
print(text)

• Requires trust_remote_code=True and transformers 4.57.x — 5.x regressed remote-model loading for this architecture.
• Use bf16 on GPU, not fp16: the architecture's masking values overflow in fp16.

CUDA kernels (NVIDIA GPUs)

The fastest GPU path is the custom Triton kernel stack in cuda/: flash-style relative-position attention (no rel_shift copies or score-tensor materialization), single-query decode attention, fused depthwise-conv+BatchNorm+SiLU, channels-last subsampling convs, a GPU log-mel frontend, and a CUDA-graphed greedy decode loop. Same weights, same math: WER on the full FLEURS-da test set is 11.47 vs 11.45 through the reference path on identical audio. Speed is in the table above — the batched figure corresponds to about 48% MFU on the 3090, so the remaining headroom on this card is small.

cuda/hviske_enc_kernels.py also patches the encoder of vLLM 0.19.0 in place (instructions in cuda/README.md), lifting batched serving by 25–35% at unchanged WER (the two vLLM rows above). After the patch, vLLM is CPU-frontend-bound: use concurrency ≥512 to saturate the GPU.

Server (vLLM)

A bf16 checkpoint with the serving fixes already applied lives in vllm/. Tested with vLLM 0.19.0.

pip install vllm==0.19.0
huggingface-cli download syvai/hviske-v5-tiny --include 'vllm/*' --local-dir ./hviske

vllm serve ./hviske/vllm --served-model-name hviske-v5-tiny \
  --trust-remote-code --dtype bfloat16 --gpu-memory-utilization 0.90 \
  --max-model-len 1024 --max-num-seqs 512 --api-server-count 4 --port 18010

curl http://127.0.0.1:18010/v1/audio/transcriptions \
  -F file=@clip.wav -F model=hviske-v5-tiny -F language=da

Single-request p50 is 62 ms on an RTX 3090; throughput and WER are in the Speed table above. See vllm/README.md for tuning notes and the reasons that directory differs from the root checkpoint.

Apple silicon (MLX)

Native MLX builds live in mlx/ — fp16, int8 and int4 weights plus a small pure-MLX runtime, so the model runs on a Mac with no PyTorch installed.

pip install mlx numpy sentencepiece soundfile huggingface_hub

import sys
import mlx.core as mx
import soundfile as sf
from huggingface_hub import snapshot_download

path = snapshot_download("syvai/hviske-v5-tiny", allow_patterns=["mlx/int4/*", "mlx/hviske_mlx/*"])
sys.path.insert(0, f"{path}/mlx")
from hviske_mlx.transcribe import Hviske

model = Hviske(f"{path}/mlx/int4")
audio, sr = sf.read("clip.wav", dtype="float32")   # 16 kHz mono
print(model.generate(mx.array(audio))["text"])

Batched greedy decoding is available via model.generate_batch([a1, a2, ...]).

Build sizes: fp16 526 MB, int8 300 MB, int4 179 MB; speed and WER are in the Speed table above. int4 is the recommended default — it matches int8 on accuracy while being 41% smaller and faster; fp16 tracks the PyTorch model most closely. The port is verified against PyTorch stage by stage (encoder output agrees to 1.5e-06; 38 of 40 clips decode byte-identically) — see mlx/README.md for details, limitations, and the alternative mlx-speech runtime.

CPU (ONNX Runtime)

ONNX graphs live in onnx/ — portable CPU inference with no PyTorch, on macOS, Linux or Windows.

pip install onnxruntime numpy sentencepiece soundfile huggingface_hub

import sys
import soundfile as sf
from huggingface_hub import snapshot_download

path = snapshot_download("syvai/hviske-v5-tiny", allow_patterns=["onnx/*"])
sys.path.insert(0, f"{path}/onnx")
from hviske_onnx.runtime import HviskeOnnx

model = HviskeOnnx(f"{path}/onnx", encoder_int8=False, decoder_int8=True, threads=2)
audio, sr = sf.read("clip.wav", dtype="float32")   # 16 kHz mono
print(model.generate(audio)["text"])

Speed and WER are in the Speed table above (2.8× faster than the PyTorch CPU path at equal accuracy). The encoder is fp32 and the decoder int8: dynamic int8 helps the decoder (3× faster, no WER cost) but hurts the conv-heavy encoder on ARM (2× slower, +1.9 WER). Use 2 threads — the graphs are too small to profit from more. See onnx/README.md.

GGUF (CrispASR)

GGUF builds for the CrispASR C++/ggml runtime live in gguf/ — no Python at inference, with Metal/CUDA/Vulkan/CPU backends.

./build/bin/crispasr --backend cohere -m hviske-v5-tiny-q4_k.gguf -f clip.wav -l da -t 4

Build	Size	RTFx (Metal)	WER
q4_k	160 MB	56.4×	10.51
q5_0	190 MB	52.5×	10.57
q6_k	243 MB	50.5×	10.53
q8_0	281 MB	45.0×	10.44
f16	527 MB	39.1×	10.51

q4_k is the recommended build — smallest, fastest, and no measurable accuracy cost against f16 (ggml's per-block k-quants cost nothing here, unlike ORT's dynamic int8). The CPU-only figure is in the Speed table above. See gguf/README.md.

Limitations

• Danish only, despite the multilingual tokenizer inherited from the teacher.
• Conversational/spontaneous speech is the weakest domain (CoRal conversation 26.07 WER), consistent with the teacher family.
• Distilled from teacher pseudo-labels, so it inherits the teacher's biases and cannot exceed it on material where the teacher is wrong.
• No timestamps, diarization, or streaming.

License

CC BY-NC 4.0, inherited from the teacher syvai/hviske-v5.3.