Repository navigation
fix(tts/kokoro-ane): apply misaki's ɾ → T flap step to English phonemes - #999
Conversation
Sortformer High-Latency Benchmark ResultsES2004a Performance (30.4s latency config)
Sortformer High-Latency • ES2004a • Runtime: 3m 35s • 2026-10-08T02:35:20.057Z |
VAD Benchmark ResultsPerformance Comparison
Dataset Details
✅: Average F1-Score above 70% |
Supertonic3 Smoke Test ✅
Runtime: 0m36s Note: CI VMs lack a physical Neural Engine; the ANE-bucketed VectorEstimator falls back to CPU here. This validates download + variant resolution + synthesis, not ANE residency/perf. |
PocketTTS Smoke Test ✅
Runtime: 0m22s Note: PocketTTS uses CoreML MLState (macOS 15) KV cache + Mimi streaming state. CI VM lacks physical GPU — audio quality and performance may differ from Apple Silicon. |
Offline VBx Pipeline ResultsSpeaker Diarization Performance (VBx Batch Mode)Optimal clustering with Hungarian algorithm for maximum accuracy
Offline VBx Pipeline Timing BreakdownTime spent in each stage of batch diarization
Speaker Diarization Research ComparisonOffline VBx achieves competitive accuracy with batch processing
Pipeline Details:
🎯 Offline VBx Test • AMI Corpus ES2004a • 1049.0s meeting audio • 116.4s processing • Test runtime: 2m 5s • 10/07/2026, 11:02 PM EST |
ASR Benchmark Results ✅Status: All benchmarks passed Parakeet v3 (multilingual)
Parakeet v2 (English-optimized)
Streaming (v3)
Streaming (v2)
Streaming tests use 5 files with 0.5s chunks to simulate real-time audio streaming 25 files per dataset • Test runtime: 9m7s • 10/07/2026, 10:53 PM EST RTFx = Real-Time Factor (higher is better) • Calculated as: Total audio duration ÷ Total processing time Expected RTFx Performance on Physical M1 Hardware:• M1 Mac: ~28x (clean), ~25x (other) Testing methodology follows HuggingFace Open ASR Leaderboard |
Parakeet EOU Benchmark Results ✅Status: Benchmark passed Performance Metrics
Streaming Metrics
Test runtime: 1m12s • 10/07/2026, 10:11 PM EST RTFx = Real-Time Factor (higher is better) • Processing includes: Model inference, audio preprocessing, state management, and file I/O |
Speaker Diarization Benchmark ResultsSpeaker Diarization PerformanceEvaluating "who spoke when" detection accuracy
Diarization Pipeline Timing BreakdownTime spent in each stage of speaker diarization
Speaker Diarization Research ComparisonResearch baselines typically achieve 18-30% DER on standard datasets
Note: RTFx shown above is from GitHub Actions runner. On Apple Silicon with ANE:
🎯 Speaker Diarization Test • AMI Corpus ES2004a • 1049.0s meeting audio • 60.2s diarization time • Test runtime: 3m 12s • 10/07/2026, 10:21 PM EST |
The English frontend reads the misaki lexicon raw, which keeps the flap ɾ.
misaki's G2P.__call__ rewrites it to T for Kokoro v1.0 before synthesis.
Apply the same step in the English phonemizer path (also used by
englishPhonemes(for:)).
misaki also rewrites the glottal stop ʔ → t, but this chain reads ʔn better
than tn ("button" → "butt", "mitten" → "mit" with it), so ʔ is kept.
tts-benchmark kokoro-ane, M5 Pro, macOS 27:
- 12 flap/glottal-dense sentences: WER 3.67% → 0.00%, CER 0.56% → 0.00%
("metal" was read as "mattle"); ɾ+ʔ variant 1.67% / 1.04% (button/mitten
broken)
- minimax-english (100): WER 0.68% → 0.68%, CER 0.17% → 0.17%
1df4525 to
eea358f
Compare
The English frontend reads the misaki lexicon raw, so it keeps the flap
ɾ. misaki's ownG2P.__call__rewrites it toTbefore Kokoro v1.0 sees it. Baseline Kokoro read "metal" as "mattle".ɾ → Tin the English phonemizer path (also used byenglishPhonemes(for:))ʔ: misaki also rewrites it tot, but that made Kokoro read "button" as "butt" and "mitten" as "mit"Results (
tts-benchmark --backend kokoro-ane, M5 Pro, macOS 27):ɾ→T+ʔ→tɾ→T)Reviewer notes: the gain is small (one word in a targeted set; MiniMax unchanged). The Mandarin variant's English runs (
englishPhonemes(for:)) also get the rewrite; that path is untested. Unit test added, not run locally (no XCTest here).Found while adding Paradee (#998), which needs both rewrites.
🤖 Generated with Claude Code