I think @snailplane's idea of weighting wide characters is fairly reasonable. Some say Japanese in Japanese orthography has double density of English character-wise.
(source)
Given that the software can handle Unicode properties, we can utilize East Asian Width feature for actual implementation. In this framework, every character in Unicode is categorized as either of W (wide), Na (narrow), F (fullwidth), H (halfwidth), A (ambiguous) or N (not East Asian).
So, for example we could count each of them:
- W → 2 in length (kanji, kana, Japanese punctuations...)
- Na → 1 in length (mostly Latin letters and their punctuations)
- F → 1 in length, as they are variations of equivalent narrow characters
- H → 2 in length, as they are variations of equivalent wide characters
- N → 1 in length (most other non-CJK scripts)
- A → 1 or 2 according to each case (some symbols)
- could treat all of them wholesale as 1, or 2
- could use their general category property to decide, e.g. Letters and Punctuations are 1, others are 2 etc.
- could have other / more dedicated criteria...
Strictly speaking there are other considerations such as whether to pre-normalize using NFD or NFKD and such things, but this is the basic concept.
if title_text.chars.all? { |c| c.ord < 128 }
)