Repository navigation
feat(tts/supertonic3): v2 CoreML bundles fix padding leak at chunk ends - #1002
Conversation
Upstream's ConvNeXt depthwise convs in text_encoder, duration_predictor and vector_estimator pad both sides with ONNX `Pad mode=edge` on the unpadded sequence. The CoreML pipeline right-pads text to T=128 and, on the default ANE path, the latent to the L=128/256/512 bucket, so the v1 conv at the last valid frame read masked zeros instead of a replicated frame. Pure-ONNX FP32 repro of the leak: duration -1%, text_emb max |d| 0.22-0.34, VE 8-step latent RMSE 0.08-0.26 on a ~0.3 rms signal, concentrated in the last frames of each chunk. v2 fills padded frames with the last valid frame before each symmetric conv (x*m + (1-m)*x_last; identical when nothing is padded). Same I/O, sizes and ANE placement; the causal vocoder is unchanged so it keeps its name. Measured (M5 Pro, macOS 27, shared noise vs unpadded ONNX): - mag-STFT SNR, ANE L128: shipped int4 1.0 dB -> v2 fp16 25.4 / int8 19.8 dB - last-250 ms tail energy: v1 median -47 dB (max -5.5, 56/80 files audible) -> v2 -77..-79 dB, matching ONNX (-79); all quant levels - VE 8 steps ~35 ms unchanged (still ANE); WER ~1% on 80 utts, unchanged - Swift CLI e2e, 8-chunk paragraph: chunk tails v1 -109..-21 dB, v2 -109..-81 dB v2 files are published next to v1 in FluidInference/supertonic-3-coreml (HF commit 1b540236), so older releases keep working.
Parakeet EOU Benchmark Results ✅Status: Benchmark passed Performance Metrics
Streaming Metrics
Test runtime: 1m27s • 10/07/2026, 11:32 PM EST RTFx = Real-Time Factor (higher is better) • Processing includes: Model inference, audio preprocessing, state management, and file I/O |
Supertonic3 Smoke Test ✅
Runtime: 0m36s Note: CI VMs lack a physical Neural Engine; the ANE-bucketed VectorEstimator falls back to CPU here. This validates download + variant resolution + synthesis, not ANE residency/perf. |
Speaker Diarization Benchmark ResultsSpeaker Diarization PerformanceEvaluating "who spoke when" detection accuracy
Diarization Pipeline Timing BreakdownTime spent in each stage of speaker diarization
Speaker Diarization Research ComparisonResearch baselines typically achieve 18-30% DER on standard datasets
Note: RTFx shown above is from GitHub Actions runner. On Apple Silicon with ANE:
🎯 Speaker Diarization Test • AMI Corpus ES2004a • 1049.0s meeting audio • 60.3s diarization time • Test runtime: 4m 23s • 10/07/2026, 11:33 PM EST |
PocketTTS Smoke Test ✅
Runtime: 0m27s Note: PocketTTS uses CoreML MLState (macOS 15) KV cache + Mimi streaming state. CI VM lacks physical GPU — audio quality and performance may differ from Apple Silicon. |
Offline VBx Pipeline ResultsSpeaker Diarization Performance (VBx Batch Mode)Optimal clustering with Hungarian algorithm for maximum accuracy
Offline VBx Pipeline Timing BreakdownTime spent in each stage of batch diarization
Speaker Diarization Research ComparisonOffline VBx achieves competitive accuracy with batch processing
Pipeline Details:
🎯 Offline VBx Test • AMI Corpus ES2004a • 1049.0s meeting audio • 147.5s processing • Test runtime: 2m 30s • 10/07/2026, 11:42 PM EST |
Sortformer High-Latency Benchmark ResultsES2004a Performance (30.4s latency config)
Sortformer High-Latency • ES2004a • Runtime: 2m 33s • 2026-10-08T03:47:01.046Z |
VAD Benchmark ResultsPerformance Comparison
Dataset Details
✅: Average F1-Score above 70% |
ASR Benchmark Results ✅Status: All benchmarks passed Parakeet v3 (multilingual)
Parakeet v2 (English-optimized)
Streaming (v3)
Streaming (v2)
Streaming tests use 5 files with 0.5s chunks to simulate real-time audio streaming 25 files per dataset • Test runtime: 9m55s • 10/08/2026, 12:20 AM EST RTFx = Real-Time Factor (higher is better) • Calculated as: Total audio duration ÷ Total processing time Expected RTFx Performance on Physical M1 Hardware:• M1 Mac: ~28x (clean), ~25x (other) Testing methodology follows HuggingFace Open ASR Leaderboard |
1 participant
The Supertonic-3 text encoder, duration predictor and vector estimator use ConvNeXt convs that pad both sides with edge (replicate) padding upstream. We right-pad text to T=128 and, on the default ANE path, the latent to the 128/256/512 bucket. The conv at the last valid frame therefore read padded zeros instead of a copy of the last frame. That left audible energy in the last ~250 ms of most chunks, plus small duration and text-embedding drift.
Fix
_v2bundles fill padded frames with the last valid frame before each two-sided conv. Unpadded inputs give identical results, and inputs, outputs, sizes and ANE placement are unchanged.1b540236), so older releases keep working.ModelNames.Supertonic3now points at theTextEncoder_v2,DurationPredictor_v2andVectorEstimator_v2*bundles. The Vocoder's convs only look backward, so it is unchanged.Results (M5 Pro, macOS 27, compared against unpadded ONNX)
In an end-to-end CLI run on an 8-chunk paragraph, chunk tails went from −109…−21 dB (v1) to −109…−81 dB (v2), with 0% WER for both.
Notes
dyn-int8anddyn-int6builds return garbage on.cpuAndGPUfor v1 and v2 alike. That's a pre-existing OS issue, not addressed here.🤖 Generated with Claude Code