Repository navigation
feat(package): TTS and Diarizer opt-out traits (#990) - #1001
Alex-Wengg wants to merge 1 commit into
Conversation
ASR-only consumers compiled the TTS and diarization subsystems and shipped the ~1 MB LuxTTS G2P resource bundle even though nothing could reach them. Package@swift-6.2.swift now declares `TTS` and `Diarizer` traits, default-on alongside `NemoTextProcessing`, so `traits: []` builds ASR + VAD only. - Every file under Sources/FluidAudio/TTS and Sources/FluidAudio/Diarizer is wrapped in `#if TTS` / `#if Diarizer` (the formatter does not indent conditional blocks, so each file gains two lines). - The G2P lexicon moves into its own `LuxTtsG2pResources` target. SwiftPM cannot condition resources inside a target, so the trait conditions the dependency edge instead and the bundle drops out. - `DiarizerError` gets its own unguarded file because Shared/ANEMemoryOptimizer throws it. ModelNames guards its members that use Sortformer, PocketTTS, or Chatterbox types. Inflect frame buckets now live in ModelNames, and InflectConstants reads them from there. - Package.swift (tools < 6.2) and the podspec define both flags unconditionally, so those builds compile everything as before. Behavior change: `traits: []` used to drop only the NeMo engine. Apps that also use diarization now need `traits: ["Diarizer"]`. Leaving it out gives a compile error, never a silent behavior change. Measured on a minimal macOS consumer (release build, strip -x): the default traits give 14.54 MB plus a 1.0 MB bundle, `["NemoTextProcessing"]` gives 3.80 MB, and `[]` gives 3.13 MB with no bundle. Every trait subset builds the library target; the CLI and tests still need TTS and Diarizer, so CI's no-Nemo job now passes `--traits TTS,Diarizer`.
PocketTTS Smoke Test ✅
Runtime: 0m25s Note: PocketTTS uses CoreML MLState (macOS 15) KV cache + Mimi streaming state. CI VM lacks physical GPU — audio quality and performance may differ from Apple Silicon. |
Offline VBx Pipeline ResultsSpeaker Diarization Performance (VBx Batch Mode)Optimal clustering with Hungarian algorithm for maximum accuracy
Offline VBx Pipeline Timing BreakdownTime spent in each stage of batch diarization
Speaker Diarization Research ComparisonOffline VBx achieves competitive accuracy with batch processing
Pipeline Details:
🎯 Offline VBx Test • AMI Corpus ES2004a • 1049.0s meeting audio • 152.6s processing • Test runtime: 2m 38s • 10/07/2026, 10:25 PM EST |
Sortformer High-Latency Benchmark ResultsES2004a Performance (30.4s latency config)
Sortformer High-Latency • ES2004a • Runtime: 4m 6s • 2026-10-08T02:29:26.179Z |
Supertonic3 Smoke Test ✅
Runtime: 0m32s Note: CI VMs lack a physical Neural Engine; the ANE-bucketed VectorEstimator falls back to CPU here. This validates download + variant resolution + synthesis, not ANE residency/perf. |
Speaker Diarization Benchmark ResultsSpeaker Diarization PerformanceEvaluating "who spoke when" detection accuracy
Diarization Pipeline Timing BreakdownTime spent in each stage of speaker diarization
Speaker Diarization Research ComparisonResearch baselines typically achieve 18-30% DER on standard datasets
Note: RTFx shown above is from GitHub Actions runner. On Apple Silicon with ANE:
🎯 Speaker Diarization Test • AMI Corpus ES2004a • 1049.0s meeting audio • 36.1s diarization time • Test runtime: 2m 50s • 10/07/2026, 11:18 PM EST |
ASR Benchmark Results ✅Status: All benchmarks passed Parakeet v3 (multilingual)
Parakeet v2 (English-optimized)
Streaming (v3)
Streaming (v2)
Streaming tests use 5 files with 0.5s chunks to simulate real-time audio streaming 25 files per dataset • Test runtime: 9m46s • 10/07/2026, 11:24 PM EST RTFx = Real-Time Factor (higher is better) • Calculated as: Total audio duration ÷ Total processing time Expected RTFx Performance on Physical M1 Hardware:• M1 Mac: ~28x (clean), ~25x (other) Testing methodology follows HuggingFace Open ASR Leaderboard |
Parakeet EOU Benchmark Results ✅Status: Benchmark passed Performance Metrics
Streaming Metrics
Test runtime: 1m32s • 10/07/2026, 11:25 PM EST RTFx = Real-Time Factor (higher is better) • Processing includes: Model inference, audio preprocessing, state management, and file I/O |
VAD Benchmark ResultsPerformance Comparison
Dataset Details
✅: Average F1-Score above 70% |
1 participant
Closes #990.
ASR-only consumers compiled the TTS and diarization subsystems and shipped the ~1 MB LuxTTS G2P bundle that no code path could reach.
Package@swift-6.2.swiftaddsTTSandDiarizertraits, default-on alongsideNemoTextProcessing.TTS/andDiarizer/is wrapped in#if TTS/#if Diarizer. Each file gains two lines and keeps its indentation.LuxTtsG2pResourcestarget. SwiftPM can't condition resources inside a target, so the trait conditions the dependency edge instead.DiarizerErroris unguarded (ANEMemoryOptimizerthrows it). A fewModelNamesmembers that use TTS or diarizer types are guarded.Package.swift(tools < 6.2) and the podspec define both flags unconditionally, so those builds are unchanged.--traits TTS,Diarizer, because the CLI and tests need both.Behavior change:
traits: []used to drop only the NeMo engine. Apps that also use diarization now needtraits: ["Diarizer"]. Without it they get a compile error, never a silent change. README and PostProcessing.md cover this.Minimal macOS consumer (release build,
strip -x):["NemoTextProcessing"][]The README example says
from: "0.17.7". Adjust it if the release version differs.🤖 Generated with Claude Code