Add classifier-free guidance support for Chatterbox (fixes multilingual) + MinPLogitsWarper

[tibzejoker]

Fixes #1656

TL;DR

The Chatterbox multilingual checkpoint was thought to be unsupported ("requires special setup"). After a full investigation (ONNX graph diffing against the English export, tokenizer comparison, and step-by-step replication of the reference PyTorch sampling loop), it turns out the architecture is already fully supported by the existing ChatterboxModel — the missing piece in the library is classifier-free guidance (CFG), which the reference implementation (resemble-ai/chatterbox T3.inference) always applies with cfg_weight=0.5. Without CFG the multilingual model only produces short unintelligible vocalizations followed by an early EOS; with it, it produces correct cloned speech (validated in-browser on WebGPU in French and German with a 10s reference voice).

Changes

1. ChatterboxModel: CFG support, enabled via guidance_scale (= 1 + cfg_weight, i.e. 1.5 for parity with the python defaults). When guidance_scale > 1, forward internally runs a batch of two sequences — the conditional input and an unconditional copy whose text token embeddings are zeroed (matching text_emb[1].zero_() in the reference implementation; speaker conditioning, exaggeration and speech tokens are shared between the two rows). The two rows are recombined by the existing ClassifierFreeGuidanceLogitsProcessor, so the batch size visible to generate() stays 1. Opt-in: no behavior change when guidance_scale is unset.
2. Processor ordering: move ClassifierFreeGuidanceLogitsProcessor to the front of the list, matching the python transformers ordering — the cond/uncond logits must be combined back into a single batch before any other processor (e.g. repetition penalty) runs.
3. Add MinPLogitsWarper + the min_p generation option (full implementation incl. min_tokens_to_keep, with unit tests). The official Chatterbox sampling parameters use min_p=0.05.

Usage (multilingual)

const waveform = await model.generate({
    ...inputs, ...speaker_data, exaggeration: 0.5,
    max_new_tokens: 1000,
    do_sample: true, temperature: 0.8, top_k: 0, top_p: 1.0,
    min_p: 0.05, repetition_penalty: 1.2,
    guidance_scale: 1.5, // 1 + cfg_weight(0.5) — required for the multilingual checkpoint
});

Note on the multilingual model files (for anyone reproducing)

There is currently no official ONNX repo with complete configs for the multilingual checkpoint. The community mirror's tokenizer.json has a broken post_processor (its TemplateProcessing special tokens are referenced without brackets — "BOS", "EOS", "START_SPEECH", "EXAGGERATION" — which don't exist in the vocab, so all five special tokens encode to [UNK]). The correct template, identical to the working English export, is: [EXAGGERATION](6563) [START](255) …text… [STOP](0) [START_SPEECH](6561) [START_SPEECH](6561). Happy to help fix/publish corrected model files if useful.

Validation

• In-browser (WebGPU, Chrome/macOS): multilingual checkpoint + this PR's build → intelligible cloned speech in French and German from a 10.7s reference (previously: 0.5s of breath noise, RMS ≈ 0.0003 → now ≈ 5–6s of speech, RMS ≈ 0.08). English checkpoint unchanged.
• pnpm build ✓ (incl. typegen), pnpm format:check ✓, logits_process test suite ✓ (incl. 3 new MinPLogitsWarper unit tests).

[nico-martin]

Hi @tibzejoker, thank you so much for looking into this! The CFG approach follows the reference implementation and looks solid overall. Could you add tests for the CFG path itself, make MinP reject NaN, and share the model/tokenizer revision used for the WebGPU checks? I'm holding off on merging because the main CFG behavior is currently untested and NaN silently disables MinP filtering.

[nico-martin]

Please add deterministic coverage for the CFG path: two-row prefill, text-only unconditional masking, retained speaker/speech conditioning, cached-token duplication, two-row cache/mask state, and reduction back to the user-visible batch size. The current tests exercise only MinP.

[nico-martin]

This accepts NaN, after which the threshold is NaN and filtering silently does nothing. Please require Number.isFinite(min_p) and add a rejection test.

[artursopelnik]

+1