Add verified FP32 ONNX Runtime export

Adds dynamic-length ONNX graphs, a torch-free runner, reproducible exporter, provenance, parity reports, checksums, and updated documentation.

CHANGELOG.md

@@ -1,5 +1,12 @@
-# Changelog
-
+# Changelog
+
+## 2.1.0 - 2026-07-25
+
+- Add verified FP32 ONNX graphs with dynamic text and waveform lengths.
+- Add a torch-free ONNX Runtime API and CLI with CPU, CUDA, and DirectML provider selection.
+- Add a reproducible exporter, checkpoint provenance, parity reports, and SHA-256 checksums.
+- Credit Robert Bak's original ONNX conversion and browser implementation.
+
 ## 2.0.0 - 2026-07-24
 
 - Release Inflect-Micro-v2 as a complete 24 kHz English text-to-waveform package.

README.md

@@ -14,6 +14,8 @@ tags:
   - edge-ai
   - small-model
   - pytorch
+  - onnx
+  - onnxruntime
   - vits
   - 24khz
 thumbnail: assets/inflect-v2-repository-hero.png
@@ -171,7 +173,9 @@ equal-work pooled across their two tested voices.
 Because Inflect uses the larger 100-prompt steady-state run while comparator
 rows use the shorter 50-prompt confirmation pass, this table is deployment
 context rather than a perfectly matched speed leaderboard. Several comparators
-also use optimized ONNX runtimes while this Inflect release uses PyTorch.
+also use optimized ONNX runtimes, while the published Inflect benchmark above
+uses the canonical PyTorch runtime. The separately released Inflect ONNX path
+has not been substituted into those benchmark numbers.
 
 </details>
 
@@ -257,6 +261,28 @@ sample_rate, waveform = tts.synthesize("The complete model runs locally.")
 
 The result is a 24 kHz mono `float32` waveform. Long input is split at punctuation-aware boundaries, synthesized chunk by chunk, and joined with controlled pauses.
 
+### ONNX Runtime
+
+A verified FP32 ONNX export is included under [`onnx/`](https://huggingface.co/owensong/Inflect-Micro-v2/tree/main/onnx).
+It supports dynamic lengths, CPU/CUDA/DirectML provider selection, deterministic
+seeds, and the same long-text wrapper without importing PyTorch:
+
+```bash
+python -m pip install -r onnx/requirements.txt
+python onnx/inference_onnx.py \
+  --text "The complete model now runs through ONNX Runtime." \
+  --output sample-onnx.wav \
+  --provider cpu \
+  --seed 7
+```
+
+The neural model is split into `duration.onnx` and `decode.onnx`; together they
+contain the complete learned text-to-waveform path. The English eSpeak-ng
+frontend remains CPU-side code. See the
+[`ONNX guide`](https://huggingface.co/owensong/Inflect-Micro-v2/blob/main/onnx/README.md)
+for graph contracts, provenance, parity measurements, browser deployment, and
+re-export instructions.
+
 ## Release profile
 
 | **Local runtime** | **Long-text handling** |
@@ -317,6 +343,7 @@ This release is inference-first. New-voice and new-language adaptation are **not
 | `inference.py` | Public Python API and CLI |
 | `inflect_vits_frontend.py` | English normalization, phonemization, and punctuation frontend |
 | `runtime/` | Self-contained model implementation |
+| `onnx/` | Verified FP32 ONNX graphs, torch-free runner, parity report, checksums, and exporter |
 | `samples/` | Held-out example generations |
 | `evaluation/final/` | Frozen benchmark prompts, reports, and protocol artifacts |
 | `docs/` | API, deployment, evaluation, adaptation, and export documentation |

docs/EXPORTS.md

@@ -6,19 +6,38 @@
 | --- | --- | --- |
 | PyTorch FP32 | Supported | Canonical weights and the fully tested runtime. |
 | PyTorch FP16/BF16 | Not released | Smaller in memory, but not validated for waveform quality or CPU compatibility. |
-| ONNX | Not released | Dynamic text length, stochastic latent sampling, and custom model operations require a dedicated export/runtime path. |
+| ONNX FP32 | Supported | Verified dynamic-length graphs, torch-free Python runner, and reproducible exporter under `onnx/`. |
 | INT8 / INT4 | Not released | Naive weight quantization can audibly damage the integrated waveform decoder. |
 | GGUF | Not applicable today | GGUF targets transformer/LLM runtimes and is not a drop-in container for this VITS-family convolutional waveform model. |
 | Core ML / TFLite | Not released | Requires separate conversion and target-device validation. |
 
-## Why only FP32 ships initially
+## Supported ONNX package
+
+The official ONNX package contains:
+
+- `onnx/duration.onnx`: token sequence to aligned acoustic distribution;
+- `onnx/decode.onnx`: aligned distribution and seeded noise to waveform;
+- `onnx/inference_onnx.py`: ONNX Runtime API and CLI;
+- `onnx/export_onnx.py`: reproducible export from the canonical checkpoint;
+- machine-readable provenance, parity results, and SHA-256 checksums.
+
+Dynamic text and waveform lengths are supported. The public wrapper implements
+seeded latent sampling, speed and variation controls, punctuation-aware
+long-text chunking, and CPU/CUDA/DirectML provider selection. eSpeak-ng remains
+the non-neural English frontend and is not embedded in the ONNX graphs.
+
+See [`onnx/README.md`](https://huggingface.co/owensong/Inflect-Micro-v2/blob/main/onnx/README.md)
+for installation and verified parity measurements.
+
+## Why only verified FP32 ships
 
 The model is already small: 37.53 MB for Micro and 15.97 MB for Nano. A format
 is useful only if it is smaller or faster **and** preserves pronunciation,
 stability, and waveform quality. An export that loads but produces degraded
 speech is not release-ready.
 
-The canonical `model.pth` remains the source of truth. Future exports must:
+The canonical `model.pth` remains the source of truth. Future quantized or
+platform-specific exports must:
 
 1. reproduce deterministic fixed-seed outputs within a declared tolerance;
 2. pass matched semantic-WER and signal-diagnostic gates;
@@ -28,7 +47,8 @@ The canonical `model.pth` remains the source of truth. Future exports must:
 
 ## Practical deployment today
 
-Use the FP32 package and keep one `InflectTTS` instance loaded. Nano is the
-recommended footprint-first deployment; Micro is the quality-first deployment.
-Install the CPU or CUDA PyTorch build appropriate for the target platform before
-installing the remaining requirements.
+Use either the canonical PyTorch runtime or the verified FP32 ONNX Runtime
+package and keep one engine instance loaded. Nano is the recommended
+footprint-first deployment; Micro is the quality-first deployment. Choose ONNX
+for C/C++ bindings, browser work, DirectML, or other ONNX Runtime execution
+providers. Choose PyTorch when integrating with the original Python model code.

onnx/README.md

@@ -0,0 +1,120 @@
+# Inflect-Micro-v2 ONNX
+
+Official, verified FP32 ONNX Runtime export of
+[Inflect-Micro-v2](https://huggingface.co/owensong/Inflect-Micro-v2).
+The neural model is split into two graphs:
+
+| File | Purpose | Size |
+| --- | --- | ---: |
+| `duration.onnx` | Text tokens to aligned acoustic distribution | 7.32 MB |
+| `decode.onnx` | Acoustic distribution and seeded noise to 24 kHz waveform | 30.43 MB |
+
+The complete ONNX neural weights are about **37.75 MB**. The original FP32
+PyTorch checkpoint remains the canonical source of truth.
+
+## Install
+
+```bash
+python -m pip install -r onnx/requirements.txt
+```
+
+ONNX Runtime CPU is installed by default. For another execution provider, use
+the appropriate ONNX Runtime package for the target platform.
+
+## Generate speech
+
+From the repository root:
+
+```bash
+python onnx/inference_onnx.py \
+  --text "A compact voice can still sound clear and expressive." \
+  --output sample.wav \
+  --provider cpu \
+  --speed 1.0 \
+  --variation 0.667 \
+  --seed 7
+```
+
+Python:
+
+```python
+import sys
+
+sys.path.insert(0, "onnx")
+from inference_onnx import InflectONNX
+
+tts = InflectONNX(".", provider="cpu")
+sample_rate, waveform = tts.synthesize(
+    "Inflect runs through ONNX Runtime.",
+    speed=1.0,
+    variation=0.667,
+    seed=7,
+)
+```
+
+Accepted provider aliases are `cpu`, `cuda`, and `directml`. The corresponding
+provider must exist in the installed ONNX Runtime build.
+
+## What the export includes
+
+- Dynamic text and waveform lengths
+- The duration predictor, acoustic generator, flow, and waveform decoder
+- Seeded stochastic latent sampling in the Python wrapper
+- The same punctuation-aware long-text chunking used by the PyTorch package
+- 24 kHz mono `float32` waveform output
+
+The English normalization and phonemization frontend remains Python code and
+uses eSpeak-ng on the CPU. The ONNX files contain the learned neural graphs,
+not eSpeak-ng itself.
+
+## Verification
+
+The packaged graphs pass:
+
+- ONNX checker validation at opset 17
+- Dynamic-length inference tests
+- Short- and long-form end-to-end synthesis
+- Same-seed determinism and different-seed variation checks
+- Direct numerical comparison with the canonical PyTorch checkpoint
+- Independent re-export comparison from the published `model.pth`
+
+The direct PyTorch comparison measured a maximum absolute waveform error of
+`1.7583e-05`, mean absolute error of `2.2595e-07`, and correlation above
+`0.9999999999`. An independent re-export matched this package within
+`1.0617e-07` maximum waveform error.
+
+See [`parity_report.json`](parity_report.json), [`SOURCE.json`](SOURCE.json),
+and [`checksums.sha256`](checksums.sha256) for machine-readable details.
+
+## Re-export
+
+The export script is included for reproducibility:
+
+```bash
+python -m pip install torch==2.4.1 onnx==1.19.1
+python onnx/export_onnx.py \
+  --model-dir . \
+  --output-dir onnx-reexport \
+  --model-name Inflect-Micro-v2 \
+  --source-repository owensong/Inflect-Micro-v2 \
+  --source-revision 3e8c567f9bce1a309eea12d38ca9fbf7050df72e
+```
+
+Serialization hashes can differ across exporter versions even when numerical
+outputs match. Validate outputs rather than expecting byte-identical files.
+
+## Browser implementation and credit
+
+Robert Bak produced the original ONNX conversion and a complete browser runner:
+[robertbak/webtts-inflect](https://github.com/robertbak/webtts-inflect).
+The official package preserves that conversion credit and adds checkpoint
+provenance, metadata, reproducible export tooling, and Python ONNX Runtime
+validation. The browser project is Apache-2.0 and runs synthesis locally with
+ONNX Runtime Web and an eSpeak-ng WASM frontend.
+
+## Quantization status
+
+Only verified FP32 ONNX graphs are released. INT8 and INT4 are not included:
+naive quantization of the flow and waveform decoder can produce audible
+degradation, and no quantized build has passed matched listening and
+intelligibility validation yet.

onnx/SOURCE.json

@@ -0,0 +1,16 @@
+{
+  "format": "inflect_v2_onnx_source_v1",
+  "model": "Inflect-Micro-v2",
+  "source_repository": "owensong/Inflect-Micro-v2",
+  "source_revision": "3e8c567f9bce1a309eea12d38ca9fbf7050df72e",
+  "source_model_file": "model.pth",
+  "source_model_sha256": "3eede065c9ccfa88ade0a5a9a5c23de34afcbbb32213e59aad44d5cf100fdee8",
+  "conversion_repository": "https://github.com/robertbak/webtts-inflect",
+  "conversion_commit": "60832b1978115e4ed7fdddce4d9b9fd9161f9fe7",
+  "conversion_author": "Robert Bak",
+  "official_packaging": "Owen Song",
+  "precision": "FP32",
+  "onnx_opset": 17,
+  "sample_rate_hz": 24000,
+  "license": "Apache-2.0"
+}

onnx/checksums.sha256

@@ -0,0 +1,8 @@
+7940923add86f76e7fa78d910b0632ca1779f8cc9a2ca2b49236381a9ca77183  decode.onnx
+b728ca2564b9e5b7d6cf5e446f65e02a6fe2f1880ba281466fec93a667dd2388  duration.onnx
+77bd27277861c8cae670247e3c9e3b976134a4886506fb069331aa6828752be6  export_onnx.py
+cf48c4812f68e8315d80568e93650c64a7ad3fdfb9a0051c61a0d1d0746df274  inference_onnx.py
+10c88564d24fe65ea730dc6ec98aa19ec74d3590efa427595a3c122c4c0f7039  parity_report.json
+ecfaf6a3edca6ffc3152a61ccc6c27240f42daaa0ddf6efc08e1f9b277295734  README.md
+43abd2c128e4296f1f0171f12946f3fc7dee3f8c8e2bb1858bcae6b43b62b3c9  requirements.txt
+e950bf44034b011a64bba3773b388e386418ef952c3cd17ceefda5bfa94a3aaa  SOURCE.json

onnx/decode.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7940923add86f76e7fa78d910b0632ca1779f8cc9a2ca2b49236381a9ca77183
+size 30427790

onnx/duration.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b728ca2564b9e5b7d6cf5e446f65e02a6fe2f1880ba281466fec93a667dd2388
+size 7322687

onnx/export_onnx.py

@@ -0,0 +1,261 @@
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import logging
+import sys
+from pathlib import Path
+
+import onnx
+import torch
+from torch import nn
+
+
+PACKAGE_ROOT = Path(__file__).resolve().parents[1]
+commons = None
+
+
+class DurationGraph(nn.Module):
+    def __init__(self, model: nn.Module) -> None:
+        super().__init__()
+        self.enc_p = model.enc_p
+        self.dp = model.dp
+
+    def forward(
+        self,
+        tokens: torch.Tensor,
+        lengths: torch.Tensor,
+        length_scale: torch.Tensor,
+    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
+        hidden, m_p, logs_p, x_mask = self.enc_p(tokens, lengths)
+        logw = self.dp(hidden, x_mask, g=None)
+        durations = torch.ceil(torch.exp(logw) * x_mask * length_scale)
+        y_lengths = torch.clamp_min(torch.sum(durations, [1, 2]), 1).long()
+        if commons is None:
+            raise RuntimeError("Runtime helpers were not initialized.")
+        y_mask = torch.unsqueeze(commons.sequence_mask(y_lengths, None), 1).to(
+            x_mask.dtype
+        )
+        attention_mask = torch.unsqueeze(x_mask, 2) * torch.unsqueeze(y_mask, -1)
+        attention = commons.generate_path(durations, attention_mask)
+        m_p_exp = torch.matmul(
+            attention.squeeze(1),
+            m_p.transpose(1, 2),
+        ).transpose(1, 2)
+        logs_p_exp = torch.matmul(
+            attention.squeeze(1),
+            logs_p.transpose(1, 2),
+        ).transpose(1, 2)
+        return m_p_exp, logs_p_exp, y_mask
+
+
+class DecodeGraph(nn.Module):
+    def __init__(self, model: nn.Module) -> None:
+        super().__init__()
+        self.flow = model.flow
+        self.decoder = model.dec
+
+    def forward(
+        self,
+        m_p_exp: torch.Tensor,
+        logs_p_exp: torch.Tensor,
+        y_mask: torch.Tensor,
+        zp_noise: torch.Tensor,
+        noise_scale: torch.Tensor,
+    ) -> torch.Tensor:
+        z_p = m_p_exp + zp_noise * torch.exp(logs_p_exp) * noise_scale
+        z = self.flow(z_p, y_mask, g=None, reverse=True)
+        return self.decoder(z * y_mask, g=None)
+
+
+def sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for block in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def load_model(root: Path) -> nn.Module:
+    global commons
+    runtime_root = root / "runtime"
+    sys.path.insert(0, str(runtime_root))
+    sys.path.insert(0, str(root))
+    import commons as runtime_commons
+    import utils
+    from models import SynthesizerTrn
+    from text.symbols import symbols
+
+    commons = runtime_commons
+    hps = utils.get_hparams_from_file(str(root / "config.json"))
+    model = SynthesizerTrn(
+        len(symbols),
+        hps.data.filter_length // 2 + 1,
+        hps.train.segment_size // hps.data.hop_length,
+        **hps.model,
+    ).cpu().eval()
+    logger = logging.getLogger()
+    previous_level = logger.level
+    try:
+        logger.setLevel(logging.WARNING)
+        utils.load_checkpoint(str(root / "model.pth"), model, None)
+    finally:
+        logger.setLevel(previous_level)
+    return model
+
+
+def attach_metadata(
+    path: Path,
+    *,
+    model_name: str,
+    source_repository: str,
+    source_revision: str,
+    source_model_sha256: str,
+) -> None:
+    graph = onnx.load(path)
+    metadata = {
+        "model_name": model_name,
+        "source_repository": source_repository,
+        "source_revision": source_revision,
+        "source_model_sha256": source_model_sha256,
+        "license": "Apache-2.0",
+        "sample_rate_hz": "24000",
+        "precision": "FP32",
+        "onnx_opset": "17",
+        "conversion_author": "Robert Bak (github.com/robertbak)",
+        "official_packaging": "Owen Song (huggingface.co/owensong)",
+    }
+    del graph.metadata_props[:]
+    for key, value in metadata.items():
+        item = graph.metadata_props.add()
+        item.key = key
+        item.value = value
+    onnx.checker.check_model(graph)
+    onnx.save(graph, path)
+
+
+def export(
+    root: Path,
+    output_dir: Path,
+    *,
+    model_name: str,
+    source_repository: str,
+    source_revision: str,
+) -> None:
+    torch.manual_seed(7)
+    model = load_model(root)
+    duration = DurationGraph(model).eval()
+    decode = DecodeGraph(model).eval()
+
+    output_dir.mkdir(parents=True, exist_ok=True)
+    duration_path = output_dir / "duration.onnx"
+    decode_path = output_dir / "decode.onnx"
+
+    tokens = torch.tensor(
+        [[0, 18, 0, 61, 0, 55, 0, 48, 0, 44, 0, 46, 0]],
+        dtype=torch.long,
+    )
+    lengths = torch.tensor([tokens.shape[1]], dtype=torch.long)
+    length_scale = torch.tensor(1.0, dtype=torch.float32)
+
+    torch.onnx.export(
+        duration,
+        (tokens, lengths, length_scale),
+        duration_path,
+        input_names=["tokens", "lengths", "length_scale"],
+        output_names=["m_p_exp", "logs_p_exp", "y_mask"],
+        dynamic_axes={
+            "tokens": {1: "text_len"},
+            "m_p_exp": {2: "mel_len"},
+            "logs_p_exp": {2: "mel_len"},
+            "y_mask": {2: "mel_len"},
+        },
+        opset_version=17,
+        do_constant_folding=True,
+        dynamo=False,
+    )
+
+    with torch.no_grad():
+        m_p_exp, logs_p_exp, y_mask = duration(tokens, lengths, length_scale)
+    zp_noise = torch.randn_like(m_p_exp)
+    noise_scale = torch.tensor(0.667, dtype=torch.float32)
+    torch.onnx.export(
+        decode,
+        (m_p_exp, logs_p_exp, y_mask, zp_noise, noise_scale),
+        decode_path,
+        input_names=[
+            "m_p_exp",
+            "logs_p_exp",
+            "y_mask",
+            "zp_noise",
+            "noise_scale",
+        ],
+        output_names=["waveform"],
+        dynamic_axes={
+            "m_p_exp": {2: "mel_len"},
+            "logs_p_exp": {2: "mel_len"},
+            "y_mask": {2: "mel_len"},
+            "zp_noise": {2: "mel_len"},
+            "waveform": {2: "wav_len"},
+        },
+        opset_version=17,
+        do_constant_folding=True,
+        dynamo=False,
+    )
+
+    source_hash = sha256(root / "model.pth")
+    for path in (duration_path, decode_path):
+        attach_metadata(
+            path,
+            model_name=model_name,
+            source_repository=source_repository,
+            source_revision=source_revision,
+            source_model_sha256=source_hash,
+        )
+
+    manifest = {
+        "format": "inflect_v2_onnx_export_v1",
+        "model_name": model_name,
+        "source_repository": source_repository,
+        "source_revision": source_revision,
+        "source_model_sha256": source_hash,
+        "torch_version": torch.__version__,
+        "onnx_version": onnx.__version__,
+        "opset": 17,
+        "files": {
+            duration_path.name: {
+                "bytes": duration_path.stat().st_size,
+                "sha256": sha256(duration_path),
+            },
+            decode_path.name: {
+                "bytes": decode_path.stat().st_size,
+                "sha256": sha256(decode_path),
+            },
+        },
+    }
+    (output_dir / "export_manifest.json").write_text(
+        json.dumps(manifest, indent=2) + "\n",
+        encoding="utf-8",
+    )
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Export Inflect v2 to ONNX.")
+    parser.add_argument("--model-dir", type=Path, default=PACKAGE_ROOT)
+    parser.add_argument("--output-dir", type=Path, required=True)
+    parser.add_argument("--model-name", required=True)
+    parser.add_argument("--source-repository", required=True)
+    parser.add_argument("--source-revision", required=True)
+    args = parser.parse_args()
+    export(
+        args.model_dir.resolve(),
+        args.output_dir.resolve(),
+        model_name=args.model_name,
+        source_repository=args.source_repository,
+        source_revision=args.source_revision,
+    )
+
+
+if __name__ == "__main__":
+    main()

onnx/inference_onnx.py

@@ -0,0 +1,229 @@
+from __future__ import annotations
+
+import argparse
+import re
+import sys
+from pathlib import Path
+
+import numpy as np
+import onnxruntime as ort
+import soundfile as sf
+
+
+PACKAGE_ROOT = Path(__file__).resolve().parents[1]
+RUNTIME_ROOT = PACKAGE_ROOT / "runtime"
+sys.path.insert(0, str(RUNTIME_ROOT))
+sys.path.insert(0, str(PACKAGE_ROOT))
+
+from inflect_vits_frontend import run_vits_frontend  # noqa: E402
+from text.symbols import symbols  # noqa: E402
+
+
+SAMPLE_RATE = 24_000
+SYMBOL_TO_ID = {symbol: index for index, symbol in enumerate(symbols)}
+
+
+def split_text(text: str, limit: int = 280) -> list[str]:
+    normalized = " ".join(text.split())
+    sentences = [
+        part.strip()
+        for part in re.split(r"(?<=[.!?;:])\s+", normalized)
+        if part.strip()
+    ]
+    chunks: list[str] = []
+    for sentence in sentences or [normalized]:
+        while len(sentence) > limit:
+            search = sentence[: limit + 1]
+            punctuation = max(search.rfind(mark) for mark in (",", ";", ":"))
+            split_at = (
+                punctuation + 1
+                if punctuation >= limit // 2
+                else sentence.rfind(" ", 0, limit + 1)
+            )
+            if split_at < limit // 2:
+                split_at = limit
+            chunks.append(sentence[:split_at].strip())
+            sentence = sentence[split_at:].strip()
+        if sentence:
+            chunks.append(sentence)
+    return chunks
+
+
+def boundary_pause_seconds(chunk: str) -> float:
+    ending = chunk.rstrip()[-1:] if chunk.strip() else ""
+    return {
+        "?": 0.28,
+        "!": 0.24,
+        ".": 0.22,
+        ";": 0.16,
+        ":": 0.13,
+        ",": 0.09,
+    }.get(ending, 0.08)
+
+
+def edge_fade(
+    waveform: np.ndarray,
+    sample_rate: int = SAMPLE_RATE,
+    milliseconds: float = 5.0,
+) -> np.ndarray:
+    frames = min(round(sample_rate * milliseconds / 1000.0), waveform.size // 2)
+    if frames <= 0:
+        return waveform
+    output = waveform.copy()
+    ramp = np.linspace(0.0, 1.0, frames, endpoint=True, dtype=np.float32)
+    output[:frames] *= ramp
+    output[-frames:] *= ramp[::-1]
+    return output
+
+
+def phonemes_to_tokens(phoneme_text: str) -> np.ndarray:
+    sequence = [SYMBOL_TO_ID[symbol] for symbol in phoneme_text]
+    if not sequence:
+        raise ValueError("The text frontend produced no speakable tokens.")
+    with_blanks = np.zeros(len(sequence) * 2 + 1, dtype=np.int64)
+    with_blanks[1::2] = sequence
+    return with_blanks[None, :]
+
+
+def available_provider(name: str) -> str:
+    aliases = {
+        "cpu": "CPUExecutionProvider",
+        "cuda": "CUDAExecutionProvider",
+        "directml": "DmlExecutionProvider",
+    }
+    provider = aliases.get(name.lower(), name)
+    available = ort.get_available_providers()
+    if provider not in available:
+        raise ValueError(
+            f"Provider {provider!r} is unavailable. Installed providers: {available}"
+        )
+    return provider
+
+
+class InflectONNX:
+    """Torch-free Inflect v2 inference through ONNX Runtime."""
+
+    def __init__(
+        self,
+        model_dir: str | Path = PACKAGE_ROOT,
+        provider: str = "cpu",
+    ) -> None:
+        root = Path(model_dir).resolve()
+        onnx_dir = root / "onnx"
+        selected_provider = available_provider(provider)
+        providers = [selected_provider]
+        if selected_provider != "CPUExecutionProvider":
+            providers.append("CPUExecutionProvider")
+        self.duration = ort.InferenceSession(
+            str(onnx_dir / "duration.onnx"),
+            providers=providers,
+        )
+        self.decode = ort.InferenceSession(
+            str(onnx_dir / "decode.onnx"),
+            providers=providers,
+        )
+
+    def _synthesize_chunk(
+        self,
+        text: str,
+        *,
+        speed: float,
+        variation: float,
+        seed: int,
+    ) -> np.ndarray:
+        phoneme_text = run_vits_frontend(text).phoneme_text
+        tokens = phonemes_to_tokens(phoneme_text)
+        duration_outputs = self.duration.run(
+            ["m_p_exp", "logs_p_exp", "y_mask"],
+            {
+                "tokens": tokens,
+                "lengths": np.asarray([tokens.shape[1]], dtype=np.int64),
+                "length_scale": np.asarray(1.0 / speed, dtype=np.float32),
+            },
+        )
+        m_p_exp, logs_p_exp, y_mask = duration_outputs
+        rng = np.random.default_rng(seed)
+        latent_noise = rng.standard_normal(m_p_exp.shape, dtype=np.float32)
+        waveform = self.decode.run(
+            ["waveform"],
+            {
+                "m_p_exp": m_p_exp,
+                "logs_p_exp": logs_p_exp,
+                "y_mask": y_mask,
+                "zp_noise": latent_noise,
+                "noise_scale": np.asarray(variation, dtype=np.float32),
+            },
+        )[0]
+        return edge_fade(np.asarray(waveform, dtype=np.float32).reshape(-1))
+
+    def synthesize(
+        self,
+        text: str,
+        *,
+        speed: float = 1.0,
+        variation: float = 0.667,
+        seed: int = 0,
+    ) -> tuple[int, np.ndarray]:
+        normalized = " ".join(text.split())
+        if not normalized:
+            raise ValueError("Text must not be empty.")
+        if not 0.5 <= speed <= 2.0:
+            raise ValueError("speed must be between 0.5 and 2.0")
+        if not 0.0 <= variation <= 1.0:
+            raise ValueError("variation must be between 0.0 and 1.0")
+
+        chunks = split_text(normalized)
+        pieces: list[np.ndarray] = []
+        for index, chunk in enumerate(chunks):
+            if index:
+                pause = boundary_pause_seconds(chunks[index - 1])
+                pieces.append(
+                    np.zeros(round(SAMPLE_RATE * pause), dtype=np.float32)
+                )
+            pieces.append(
+                self._synthesize_chunk(
+                    chunk,
+                    speed=speed,
+                    variation=variation,
+                    seed=seed + index,
+                )
+            )
+        waveform = np.clip(np.concatenate(pieces), -1.0, 1.0)
+        return SAMPLE_RATE, waveform
+
+    def save(self, text: str, output: str | Path, **kwargs: object) -> Path:
+        destination = Path(output)
+        destination.parent.mkdir(parents=True, exist_ok=True)
+        sample_rate, waveform = self.synthesize(text, **kwargs)
+        sf.write(destination, waveform, sample_rate)
+        return destination
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Run Inflect v2 through ONNX Runtime.")
+    parser.add_argument("--model-dir", type=Path, default=PACKAGE_ROOT)
+    parser.add_argument("--text", required=True)
+    parser.add_argument("--output", type=Path, required=True)
+    parser.add_argument(
+        "--provider",
+        default="cpu",
+        help="cpu, cuda, directml, or an ONNX Runtime provider name",
+    )
+    parser.add_argument("--speed", type=float, default=1.0)
+    parser.add_argument("--variation", type=float, default=0.667)
+    parser.add_argument("--seed", type=int, default=0)
+    args = parser.parse_args()
+
+    engine = InflectONNX(args.model_dir, provider=args.provider)
+    engine.save(
+        args.text,
+        args.output,
+        speed=args.speed,
+        variation=args.variation,
+        seed=args.seed,
+    )
+    print(f"wrote {args.output} at {SAMPLE_RATE} Hz")
+
+
+if __name__ == "__main__":
+    main()

onnx/parity_report.json

@@ -0,0 +1,31 @@
+{
+  "format": "inflect_v2_onnx_parity_v1",
+  "model": "Inflect-Micro-v2",
+  "onnx_checker": "pass",
+  "dynamic_lengths": "pass",
+  "end_to_end_short_text": "pass",
+  "end_to_end_long_text": "pass",
+  "same_seed_determinism": "pass",
+  "different_seed_variation": "pass",
+  "pytorch_parity": {
+    "duration_m_p_exp_max_abs_error": 9.5367431640625e-07,
+    "duration_logs_p_exp_max_abs_error": 3.5762786865234375e-07,
+    "duration_mask_exact": true,
+    "waveform_max_abs_error": 1.75834e-05,
+    "waveform_mean_abs_error": 2.2595e-07,
+    "waveform_correlation": 0.9999999999394602
+  },
+  "independent_reexport_parity": {
+    "duration_outputs_max_abs_error": [0.0, 0.0, 0.0],
+    "waveform_max_abs_error": 1.0617077350616455e-07,
+    "waveform_mean_abs_error": 4.823611288173879e-09,
+    "waveform_correlation": 0.9999999999995599
+  },
+  "validation_environment": {
+    "python": "3.12",
+    "onnx": "1.19.1",
+    "onnxruntime": "1.22.1",
+    "torch_for_reference": "2.4.1+cpu",
+    "provider": "CPUExecutionProvider"
+  }
+}

onnx/requirements.txt

@@ -0,0 +1,7 @@
+numpy>=1.26,<3
+onnxruntime>=1.18,<2
+soundfile>=0.13
+phonemizer>=3.3
+espeakng-loader>=0.2.4
+num2words>=0.5.14
+Unidecode>=1.3.8

release_manifest.json

@@ -7,6 +7,36 @@
   "smoke_audio_seconds": 3.968,
   "standalone_cpu_smoke_audio_seconds": 3.616,
   "files": [
+    {
+      "path": ".gitattributes",
+      "bytes": 1519,
+      "sha256": "11ad7efa24975ee4b0c3c3a38ed18737f0658a5f75a0a96787b576a78a023361"
+    },
+    {
+      "path": "CHANGELOG.md",
+      "bytes": 792,
+      "sha256": "742ddc2b7990ffa3e205f686e9c8dcb8e6e98eb3b910b411e72cfb2a3120b104"
+    },
+    {
+      "path": "CITATION.cff",
+      "bytes": 289,
+      "sha256": "8150876b279aec0438cb25851c12375522c07521261645368565febeaff22408"
+    },
+    {
+      "path": "LICENSE",
+      "bytes": 10455,
+      "sha256": "12c48c640d1d0baf158d45175a4238b9227c71c77f50b0d15791b8efc08cd793"
+    },
+    {
+      "path": "README.md",
+      "bytes": 20101,
+      "sha256": "3250258e2ec7b8e7e312da1579dda93ff68334ed803561f4ac3dcc26a5f2d644"
+    },
+    {
+      "path": "THIRD_PARTY_NOTICES.md",
+      "bytes": 1233,
+      "sha256": "2ead63b7d9cddd6c1d12a96d88de0ade6fd3eabd1278825c0a3a2c1d3647ab4b"
+    },
     {
       "path": "assets/evidence/asr-consensus.svg",
       "bytes": 3225,
@@ -22,6 +52,11 @@
       "bytes": 6123,
       "sha256": "1a2218e0bbe726167a5b0201736a82947edbd7c6803df3f5689e7a711026758e"
     },
+    {
+      "path": "assets/evidence/cpu-throughput.svg",
+      "bytes": 3198,
+      "sha256": "4a74b4f19c87c5da9e8f368dd0117ac2840c437976c677572e3f65d1be785b8b"
+    },
     {
       "path": "assets/evidence/evidence-summary.json",
       "bytes": 52605,
@@ -42,6 +77,11 @@
       "bytes": 7934,
       "sha256": "9086575f7d11395255f4cf4ae67e30bf92d60042c6b4bdc58ee6045595cb5420"
     },
+    {
+      "path": "assets/evidence/quality-vs-cpu-speed.svg",
+      "bytes": 3887,
+      "sha256": "023e0a4a86c71eefea19a4fdab7404b8710ead4045ae3964928f149af1cc04b4"
+    },
     {
       "path": "assets/evidence/quality-vs-footprint.svg",
       "bytes": 5852,
@@ -49,18 +89,33 @@
     },
     {
       "path": "assets/inflect-v2-repository-hero.png",
-      "bytes": 1654856,
-      "sha256": "5de5efe7857fd7d17b2669e014846545014aed47bb4f19def7b06406532ffd2d"
+      "bytes": 1911877,
+      "sha256": "ca59256690b4ce1b11f27e51ee4aa21c30883d58c1e5c87cccae553898afe884"
     },
     {
-      "path": "CHANGELOG.md",
-      "bytes": 437,
-      "sha256": "ebc480bdab578e912a3ece670eb20e005ced74739ab5134f12ec0c05eada3b2f"
+      "path": "assets/nav/collection.svg",
+      "bytes": 652,
+      "sha256": "85387a74b68b0b7b735f4ce3527f32ddd5ca084dfd49dc81fa96ebf816b153e6"
     },
     {
-      "path": "CITATION.cff",
-      "bytes": 289,
-      "sha256": "8150876b279aec0438cb25851c12375522c07521261645368565febeaff22408"
+      "path": "assets/nav/github.svg",
+      "bytes": 517,
+      "sha256": "d55949cc700715bb152b0380799b314631f442414e81b83a6f3553df18dbef6f"
+    },
+    {
+      "path": "assets/nav/micro.svg",
+      "bytes": 529,
+      "sha256": "f56b72a293acdf8e9fed93893f75acf760a20a2fe09b316646a627db2ada741a"
+    },
+    {
+      "path": "assets/nav/nano.svg",
+      "bytes": 726,
+      "sha256": "69275d0d9277144c098b3dce66db5e870ebd113d97ca55cc6912242bda520643"
+    },
+    {
+      "path": "assets/nav/playground.svg",
+      "bytes": 624,
+      "sha256": "fa05ff68190baf290f3d1e9c9b8a7bc89af033fd03066cf7d3e828089b194953"
     },
     {
       "path": "config.json",
@@ -74,8 +129,8 @@
     },
     {
       "path": "docs/DATA_AND_VOICE.md",
-      "bytes": 1671,
-      "sha256": "894e9fa4e5f6fceb76e7b2da2fca4de924230c6043ef5de167ef8c15298c6b6e"
+      "bytes": 1739,
+      "sha256": "8a504473b8377493256d435124851479ebee7f74432671ed0e0dff4e4509d8ba"
     },
     {
       "path": "docs/DEPLOYMENT.md",
@@ -84,13 +139,13 @@
     },
     {
       "path": "docs/EVALUATION.md",
-      "bytes": 12488,
-      "sha256": "9e0d4c1811ca9959529db4e7b6907ca931f094dfd910dbe7f61b550fbc732f17"
+      "bytes": 3246,
+      "sha256": "4041e56250564366ff80ef9278d66bac5139d8833d368264931ba28ffbad7991"
     },
     {
       "path": "docs/EXPORTS.md",
-      "bytes": 1826,
-      "sha256": "ca4eb3949c46ac5fb1145de7270ff94689d18787232ea017b541372f5a2e6ca3"
+      "bytes": 2808,
+      "sha256": "bcb70436c88e92d2926941e76f141498d63e2432e3584e921a9682785c474392"
     },
     {
       "path": "docs/FINETUNING.md",
@@ -259,8 +314,8 @@
     },
     {
       "path": "inference.py",
-      "bytes": 7311,
-      "sha256": "9bccc990aafa8e363a34269367a034fc69d8216f7317650be0e389bdd545eae3"
+      "bytes": 6346,
+      "sha256": "bb8b5302b16c35ee04d6729fd73caa95d1e72b2fc50420b1cdec120327f8b672"
     },
     {
       "path": "inflect_nano_v2_frontend.py",
@@ -272,11 +327,6 @@
       "bytes": 2031,
       "sha256": "70eb4cdb96b2752015f72be984f5e62e0f26d7dc80e708875601702057b95cc2"
     },
-    {
-      "path": "LICENSE",
-      "bytes": 10455,
-      "sha256": "12c48c640d1d0baf158d45175a4238b9227c71c77f50b0d15791b8efc08cd793"
-    },
     {
       "path": "model.pth",
       "bytes": 37529995,
@@ -288,9 +338,49 @@
       "sha256": "f796d72aabd080d1232032c219dc39bbb2f5b517fe8a33a76df2a53c720bdb23"
     },
     {
-      "path": "README.md",
-      "bytes": 18712,
-      "sha256": "0ffb5c8d398ba6851215e6d5395911023bc27f349aa7ff7c75eba19ab3e91daf"
+      "path": "onnx/README.md",
+      "bytes": 3907,
+      "sha256": "ecfaf6a3edca6ffc3152a61ccc6c27240f42daaa0ddf6efc08e1f9b277295734"
+    },
+    {
+      "path": "onnx/SOURCE.json",
+      "bytes": 633,
+      "sha256": "e950bf44034b011a64bba3773b388e386418ef952c3cd17ceefda5bfa94a3aaa"
+    },
+    {
+      "path": "onnx/checksums.sha256",
+      "bytes": 653,
+      "sha256": "d74c3c8d3a148f240d795dc603e18ac2c152a3f7ad3e45dcd66d3e14ee335b6b"
+    },
+    {
+      "path": "onnx/decode.onnx",
+      "bytes": 30427790,
+      "sha256": "7940923add86f76e7fa78d910b0632ca1779f8cc9a2ca2b49236381a9ca77183"
+    },
+    {
+      "path": "onnx/duration.onnx",
+      "bytes": 7322687,
+      "sha256": "b728ca2564b9e5b7d6cf5e446f65e02a6fe2f1880ba281466fec93a667dd2388"
+    },
+    {
+      "path": "onnx/export_onnx.py",
+      "bytes": 7884,
+      "sha256": "77bd27277861c8cae670247e3c9e3b976134a4886506fb069331aa6828752be6"
+    },
+    {
+      "path": "onnx/inference_onnx.py",
+      "bytes": 7333,
+      "sha256": "cf48c4812f68e8315d80568e93650c64a7ad3fdfb9a0051c61a0d1d0746df274"
+    },
+    {
+      "path": "onnx/parity_report.json",
+      "bytes": 1026,
+      "sha256": "10c88564d24fe65ea730dc6ec98aa19ec74d3590efa427595a3c122c4c0f7039"
+    },
+    {
+      "path": "onnx/requirements.txt",
+      "bytes": 126,
+      "sha256": "43abd2c128e4296f1f0171f12946f3fc7dee3f8c8e2bb1858bcae6b43b62b3c9"
     },
     {
       "path": "requirements-tested.txt",
@@ -332,6 +422,11 @@
       "bytes": 206,
       "sha256": "16ca4890f3e0107250b4a522c0127779994a5a2b99fc35d6a27f93496a72153a"
     },
+    {
+      "path": "runtime/text/LICENSE",
+      "bytes": 1053,
+      "sha256": "f89accd9d66f6a011f19997d7a21cf71a35ffe2e2d0911b5460cbd0cab52a581"
+    },
     {
       "path": "runtime/text/__init__.py",
       "bytes": 1572,
@@ -342,11 +437,6 @@
       "bytes": 2880,
       "sha256": "11dc7325547290e529c597e620f3aaf8d0bd8a2a2987d48eb42d2289534e42d1"
     },
-    {
-      "path": "runtime/text/LICENSE",
-      "bytes": 1053,
-      "sha256": "f89accd9d66f6a011f19997d7a21cf71a35ffe2e2d0911b5460cbd0cab52a581"
-    },
     {
       "path": "runtime/text/symbols.py",
       "bytes": 642,
@@ -401,12 +491,13 @@
       "path": "third_party/VITS_LICENSE.txt",
       "bytes": 1090,
       "sha256": "1cdf858b4d555567398b378b4a0166e97f7908c56607101b8bf9bb6754cdff3f"
-    },
-    {
-      "path": "THIRD_PARTY_NOTICES.md",
-      "bytes": 1233,
-      "sha256": "2ead63b7d9cddd6c1d12a96d88de0ade6fd3eabd1278825c0a3a2c1d3647ab4b"
     }
   ],
-  "format": "inflect_v2_release_manifest_v1"
+  "format": "inflect_v2_release_manifest_v1",
+  "onnx_release": {
+    "format": "FP32 ONNX opset 17",
+    "path": "onnx/",
+    "verified": true,
+    "released": "2026-07-25"
+  }
 }