Skip to content

fix(wordspotting): zero CTC keyword-spotter audio padding - #1003

Merged
Alex-Wengg merged 1 commit into
FluidInference:mainfrom
mvanhorn:fix/991-ctc-keyword-spotter-padding
Oct 9, 2026
Merged

Alex-Wengg merged 1 commit into
FluidInference:mainfrom
mvanhorn:fix/991-ctc-keyword-spotter-padding

Conversation

@mvanhorn

@mvanhorn mvanhorn commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Why is this change needed?

prepareAudioArray now builds the window through makePaddedAudioArray. That helper still allocates a rank-1 [maxSamples] or rank-2 [1, maxSamples] array in the mel input's float32 or float16 type, calls reset(to: 0) on the buffer, then writes the clamped leading samples. New tests check short clips, an empty clip, a clip that fills the window, a clip longer than the window, both dtypes, and both ranks, and require the copied prefix to match and every padding index to be finite and zero.

CTC keyword spotting on short clips often returns log-probs that ignore the audio. Different sentences can greedy-decode to the same empty string and share one identical matrix, and the same clip can succeed or fail from run to run. prepareAudioArray allocates a fixed-length MLMultiArray for the mel window and copies only the clip into the front. The removed comment treated the tail as zero because MLMultiArray(shape:dataType:) was assumed to zero-fill; that initializer leaves storage uninitialized, and the mel model reads the whole window, so a non-zero or non-finite tail drives the log-probs.

Fixes #991

prepareAudioArray now builds the window through makePaddedAudioArray.
That helper still allocates a rank-1 [maxSamples] or rank-2 [1,
maxSamples] array in the mel input's float32 or float16 type, calls
reset(to: 0) on the buffer, then writes the clamped leading samples. New
tests check short clips, an empty clip, a clip that fills the window, a
clip longer than the window, both dtypes, and both ranks, and require
the copied prefix to match and every padding index to be finite and
zero.

Fixes FluidInference#991
@Alex-Wengg
Alex-Wengg merged commit e05ab92 into FluidInference:main Oct 9, 2026
@mvanhorn

Copy link
Copy Markdown
Contributor Author

@Alex-Wengg thanks for merging the keyword-spotter padding fix, so the CTC window always starts from zeroed audio.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CtcKeywordSpotter: input padding is not zeroed, so CTC log-probs are often independent of the audio

2 participants