Improve the ability to search chat history for Asian regional languages, such as Chinese and Japanese. Telegram's chat history search function is based on words, and is suitable for languages such as English and Russian that are separated by spaces. But for languages that do not use spaces for word segmentation, you need to match the entire sentence to get the search results, which makes the search function of some languages almost unusable. I hope Telegram can improve this feature so that it can search languages that do not use spaces to separate words.
i mean, by letter-based n-grams, this: qwertyuiop becomes: qwert, werty, ertyu, rtyui, tyuio, yuiop. this is universal, works for any language. but seems it requires nearly 6 times more resources (ram and cpu power) for english, for example, compared to usual indexing method. because, in that method, in english, it would be something like "qwerty uiop", and it becomes qwerty, uiop - 3 times less items in index, in this example. but in letter-5-grams it would become qwert, werty, erty_, rty_u, or, maybe, spaces should be removed before indexing. (i deleted previous version of this message in order to edit it, i forgot to say i compare with usual space-word-split indexing).
i think, optionally, they can detect pure english texts and index them usual way. also they can detect languages that have ready algorithms to detect their morphemes, and slit them to morphemes.
it is going to use thousands times more memory because there will be different english word pairs. so, it is like 50000 english words become 50000 times more. maybe it is still usable, for example, maybe, if not to load them all in ram.
Regardless server/client. Unable to search is painful 🥲
I
Isaac Wang
To those people who support this: I think it could be a good idea to leave comments under the TG official twitter account. Keep remind them about this issue. It's such a disgusting discrimination.
V
V
Three years after the issue was reported, it’s still not fixed nor taken care of. This is frustrating and obnoxious. Can someone from the support team take a look please?
Log in here to report bugs or suggest features. Please enter your phone number in the international format and we will send a confirmation message to your account via Telegram.
We are not aiming for improvement for English…
Take a look at ASIAN language.
About Asian Language, been ask for almost decade and no implementation.
Unable to search is painful 🥲