Skip to content

Fix UnicodeScripts pre-tokenizer dropping leading spaces - #2382

Open
Kayvan-Zahiri wants to merge 1 commit into
huggingface:mainfrom
Kayvan-Zahiri:fix/unicode-scripts-leading-space
Open

Fix UnicodeScripts pre-tokenizer dropping leading spaces#2382
Kayvan-Zahiri wants to merge 1 commit into
huggingface:mainfrom
Kayvan-Zahiri:fix/unicode-scripts-leading-space

fix(pre_tokenizers): stop UnicodeScripts dropping leading spaces

29bdc37
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs