Abstract: Methods, systems, and storage media for providing a heuristic multilanguage tokenizer in a web browser environment are disclosed. Exemplary implementations can receive a query; generate a query set comprising at least one query token; generate a result set from a searchable phrase, wherein the result set comprises at least one phrase token; generating at least one supplemental set. Determining that any of the sets comprise at least one of: a first character-type, a second character-type, or an emoji. In response to determining the first character-type: determining a word boundary and extracting a diacritic. In response to determining that the query, result set and the supplemental set comprise the second-type or the emoji, adding a space around each second character-type or emoji in the set; and segmenting the set into a plurality of tokens.

Abstract: A computer-implemented method can include receiving an input via a first keyboard associated with a first language. The method can include determining a first character string based on the input and the first keyboard. The method can include determining a first probability score indicative of a likelihood that the first character string is appropriate in the first language. The method can include determining a second character string based on the input and a second keyboard associated with a second language when the first probability score is less than a first threshold. The method can include determining a second probability score indicative of a likelihood that the second character string is appropriate in the second language. Based on the probability scores, the method can also include generating an output including at least one of the first and character strings.