Malayalam differences vs. Intl.Segmenter and browser text input

[Nicd]

Input ന്ന onto the website. I'm expecting to see one grapheme cluster consisting of 3 codepoints as it's rendered as one character and works as one character in my text input, but instead I see:

Now I'm not a Malayalam speaker, but FWIW Intl.Segmenter in the JavaScript API with the locale ml-IN does segment it into one grapheme. The same happens to, for example, പഞ്ചായത്ത് --- Intl.Segmenter segments to 4 graphemes while the website segments to 6 characters.

[JoshData]

That's interesting! I'd be curious to know how you came across that example.

I'm not an expert in internationalization so I don't know whether the problem is necessarily somewhere in the Unicode realm or if that comes out of glyph shaping rules at a different level (I think these things can also occur at the font rendering level, for example). It could be that the Unicode rules that I've deployed are not the latest from Unicode, or the extended grapheme cluster rules might depend on a locale selection (as you passed to Intl.Segmenter) which I didn't attempt here.

I don't know when I'll be able to look into this more, but thanks for the report.

[Nicd]

I just happened to run into ന്ന on Hacker News in a discussion, and tried it out on the website — which has been really useful btw, so thanks for creating it!

Checking, Intl.Segmenter will return it as one grapheme even when the locale en is used. So it's not due to that. Maybe there's just some discrepancy in how different sources do grapheme splitting.

I don't have any expectations of a fix for this, the site is free after all. :) Just thought you might want to know.