Patents by Inventor Shumin Zhai
Shumin Zhai has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240029728Abstract: Implementations described herein generally relate to generating a modification selectable element that may be provided for presentation to a user in a smart dictation session with an automated assistant. The modification selectable element may, when selected, cause a transcription, that includes textual data generated based on processing audio data that captures a spoken utterance and that is automatically arranged, to be modified. The transcription may be automatically arranged to include spacing, punctuation, capitalization, indentations, paragraph breaks, and/or other arrangement operations that are not specified by the user in providing the spoken utterance. Accordingly, a subsequent selection of the modification selectable element may cause these automatic arrangement operation(s), and/or the textual data locationally proximate to these automatic arrangement operation(s), to be modified.Type: ApplicationFiled: September 2, 2022Publication date: January 25, 2024Inventors: Nicolo D'Ercole, Shumin Zhai, Swante Scholz, Mehek Sharma, Adrien Olczak, Akshay Kannan, Alvin Abdagic, Julia Proskurnia, Viesturs Zarins
-
Publication number: 20230360444Abstract: An example method includes initiating, by a computing device, a fingerprint sensing operation that is associated with a fingerprint sensor, outputting, by the computing device and over a defined period of time, a sequence of haptic signals with varying intensity, wherein each haptic signal in the sequence has an intensity that is different than a respective intensity of any haptic signal that was previously output in the sequence, obtaining, by the computing device and from the fingerprint sensor, fingerprint data associated with a fingerprint of the finger of the user, and responsive to determining that the finger of the user is still positioned at the fingerprint sensor upon completion of the fingerprint sensing operation, outputting, by the computing device, a discrete haptic signal indicating a successful completion of the fingerprint sensing operation.Type: ApplicationFiled: September 22, 2020Publication date: November 9, 2023Inventors: Scott Jenson, Jianxun Wang, Debanjan Mukherjee, Shumin Zhai
-
Patent number: 11727212Abstract: A computing device outputs for display at a presence-sensitive display, a graphical keyboard comprising a plurality of keys, receives an indication of at least one gesture to select a group of keys of the plurality of keys, and determines at least one characteristic associated with the at least one gesture to select the group of keys of the plurality of keys. The computing device modifies a spatial model based at least in part on the at least one characteristic and determines a candidate word based at least in part on data provided by the spatial model and a language model, wherein the spatial model provides data based at least in part on the indication of the at least one gesture and wherein the language model provides data based at least in part on a lexicon. The computing device outputs for display at the presence-sensitive display, the candidate word.Type: GrantFiled: April 6, 2022Date of Patent: August 15, 2023Assignee: Google LLCInventors: Shumin Zhai, Ciprian Ioan Chelba
-
Patent number: 11698699Abstract: A method includes detecting a touch input by a user at a position on a virtual keyboard. The method further includes applying a spatial model to map the touch input to one or more first candidate keys of the virtual keyboard, wherein the spatial model maps the touch input to the one or more first candidate keys based on a heatmap of prior touch inputs by the user. The method further includes applying a language model to map the touch input to one or more second candidate keys of the virtual keyboard, wherein the language model maps the touch input to the one or more second candidate keys based on comparing the position of the touch input to one or more predicted next keys. The method further includes selecting a particular key of the one or more first and second candidate keys as indicated by the touch input.Type: GrantFiled: May 3, 2022Date of Patent: July 11, 2023Assignee: Google LLCInventors: Xu Liu, Shumin Zhai, Adrian Pittman, Billy Dou
-
Patent number: 11573698Abstract: In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.Type: GrantFiled: September 8, 2021Date of Patent: February 7, 2023Assignee: Google LLCInventors: Shumin Zhai, Thomas Breuel, Ouais Alsharif, Yu Ouyang, Francoise Beaufays, Johan Schalkwyk
-
Publication number: 20220261135Abstract: A method includes detecting a touch input by a user at a position on a virtual keyboard. The method further includes applying a spatial model to map the touch input to one or more first candidate keys of the virtual keyboard, wherein the spatial model maps the touch input to the one or more first candidate keys based on a heatmap of prior touch inputs by the user. The method further includes applying a language model to map the touch input to one or more second candidate keys of the virtual keyboard, wherein the language model maps the touch input to the one or more second candidate keys based on comparing the position of the touch input to one or more predicted next keys. The method further includes selecting a particular key of the one or more first and second candidate keys as indicated by the touch input.Type: ApplicationFiled: May 3, 2022Publication date: August 18, 2022Applicant: Google LLCInventors: Xu LIU, Shumin ZHAI, Adrian PITTMAN, Billy DOU
-
Publication number: 20220229988Abstract: A computing device outputs for display at a presence-sensitive display, a graphical keyboard comprising a plurality of keys, receives an indication of at least one gesture to select a group of keys of the plurality of keys, and determines at least one characteristic associated with the at least one gesture to select the group of keys of the plurality of keys. The computing device modifies a spatial model based at least in part on the at least one characteristic and determines a candidate word based at least in part on data provided by the spatial model and a language model, wherein the spatial model provides data based at least in part on the indication of the at least one gesture and wherein the language model provides data based at least in part on a lexicon. The computing device outputs for display at the presence-sensitive display, the candidate word.Type: ApplicationFiled: April 6, 2022Publication date: July 21, 2022Applicant: Google LLCInventors: Shumin Zhai, Ciprian Ioan Chelba
-
Patent number: 11379663Abstract: A computing device outputs a keyboard for display, receives an indication of a first gesture to select a first sequence of one or more keys, determines a set of candidate strings based in part on the first sequence of keys, and outputs for display at least one of the set of candidate strings. The computing device receives an indication of a second gesture to select a second sequence of one or more keys, and determines that characters associated with the second sequence of keys are included in a first candidate word based at least in part on the set of candidate strings, or are included in a second candidate word not based on the first sequence of keys. The computing device modifies the set of candidate strings based at least in part on the determination and outputs for display at least one of the modified candidate strings.Type: GrantFiled: March 4, 2021Date of Patent: July 5, 2022Assignee: Google LLCInventors: Yu Ouyang, Shumin Zhai, Xiaojun Bi
-
Patent number: 11347352Abstract: A method includes detecting a touch input by a user at a position on a virtual keyboard. The method further includes applying a spatial model to map the touch input to one or more first candidate keys of the virtual keyboard, wherein the spatial model maps the touch input to the one or more first candidate keys based on a heatmap of prior touch inputs by the user. The method further includes applying a language model to map the touch input to one or more second candidate keys of the virtual keyboard, wherein the language model maps the touch input to the one or more second candidate keys based on comparing the position of the touch input to one or more predicted next keys. The method further includes selecting a particular key of the one or more first and second candidate keys as indicated by the touch input.Type: GrantFiled: October 25, 2020Date of Patent: May 31, 2022Assignee: Google LLCInventors: Xu Liu, Shumin Zhai, Adrian Pittman, Billy Dou
-
Patent number: 11334717Abstract: A computing device outputs for display at a presence-sensitive display, a graphical keyboard comprising a plurality of keys, receives an indication of at least one gesture to select a group of keys of the plurality of keys, and determines at least one characteristic associated with the at least one gesture to select the group of keys of the plurality of keys. The computing device modifies a spatial model based at least in part on the at least one characteristic and determines a candidate word based at least in part on data provided by the spatial model and a language model, wherein the spatial model provides data based at least in part on the indication of the at least one gesture and wherein the language model provides data based at least in part on a lexicon. The computing device outputs for display at the presence-sensitive display, the candidate word.Type: GrantFiled: November 14, 2019Date of Patent: May 17, 2022Assignee: Google LLCInventors: Shumin Zhai, Ciprian Ioan Chelba
-
Publication number: 20220129137Abstract: A method includes detecting a touch input by a user at a position on a virtual keyboard. The method further includes applying a spatial model to map the touch input to one or more first candidate keys of the virtual keyboard, wherein the spatial model maps the touch input to the one or more first candidate keys based on a heatmap of prior touch inputs by the user. The method further includes applying a language model to map the touch input to one or more second candidate keys of the virtual keyboard, wherein the language model maps the touch input to the one or more second candidate keys based on comparing the position of the touch input to one or more predicted next keys. The method further includes selecting a particular key of the one or more first and second candidate keys as indicated by the touch input.Type: ApplicationFiled: October 25, 2020Publication date: April 28, 2022Applicant: Google LLCInventors: Xu LIU, Shumin ZHAI, Adrian PITTMAN, Billy DOU
-
Patent number: 11237660Abstract: A method includes a processor of an electronic device receiving first input signals from a first sensor in response to user contact at a first edge of the device and second input signals from a second sensor in response to user contact at a second edge of the electronic device. The first and second sensors are covered by a housing of the device. The processor determines an external context of the device based on analysis of the first input signals and the second input signals. The determined external context indicates at least a position of the device relative to a user or an orientation of the device relative to a user. Responsive to determining the external context, the electronic device executes a particular user input action.Type: GrantFiled: April 27, 2020Date of Patent: February 1, 2022Assignee: Google LLCInventors: Tyler Reed Kugler, Alexey Polyudov, Seungyon Lee, Yun-Ling Lee, Philip Quinn, Kishore Sundara-Rajan, Shumin Zhai, Debanjan Mukherjee, James B. Miller, Isaac William Reynolds
-
Publication number: 20210405868Abstract: In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.Type: ApplicationFiled: September 8, 2021Publication date: December 30, 2021Applicant: Google LLCInventors: Shumin Zhai, Thomas Breuel, Ouais Alsharif, Yu Ouyang, Francoise Beaufays, Johan Schalkwyk
-
Patent number: 11150804Abstract: In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.Type: GrantFiled: April 30, 2020Date of Patent: October 19, 2021Assignee: Google LLCInventors: Shumin Zhai, Thomas Breuel, Ouais Alsharif, Yu Ouyang, Francoise Beaufays, Johan Schalkwyk
-
Publication number: 20210192135Abstract: A computing device outputs a keyboard for display, receives an indication of a first gesture to select a first sequence of one or more keys, determines a set of candidate strings based in part on the first sequence of keys, and outputs for display at least one of the set of candidate strings. The computing device receives an indication of a second gesture to select a second sequence of one or more keys, and determines that characters associated with the second sequence of keys are included in a first candidate word based at least in part on the set of candidate strings, or are included in a second candidate word not based on the first sequence of keys. The computing device modifies the set of candidate strings based at least in part on the determination and outputs for display at least one of the modified candidate strings.Type: ApplicationFiled: March 4, 2021Publication date: June 24, 2021Applicant: Google LLCInventors: Yu Ouyang, Shumin Zhai, Xiaojun Bi
-
Patent number: 10977440Abstract: A computing device outputs a keyboard for display, receives an indication of a first gesture to select a first sequence of one or more keys, determines a set of candidate strings based in part on the first sequence of keys, and outputs for display at least one of the set of candidate strings. The computing device receives an indication of a second gesture to select a second sequence of one or more keys, and determines that characters associated with the second sequence of keys are included in a first candidate word based at least in part on the set of candidate strings, or are included in a second candidate word not based on the first sequence of keys. The computing device modifies the set of candidate strings based at least in part on the determination and outputs for display at least one of the modified candidate strings.Type: GrantFiled: July 12, 2017Date of Patent: April 13, 2021Assignee: Google LLCInventors: Yu Ouyang, Shumin Zhai, Xiaojun Bi
-
Patent number: 10846602Abstract: A computing device is described that receives first input, at an initial time, of a first textual character and a second input, at a subsequent time, of a second textual character. The computing device determines, based on the first and second textual characters, a first character sequence that does not include a space character between the first and second textual characters and a second character sequence that includes the space character between the first and second textual characters. The computing device determines a first score associated with the first character sequence and a second score associated with the second character sequence. The computing device adjusts, based on a duration of time between the initial and subsequent times, the second score to determine a third score, and responsive to determining that the third score exceeds the first score, the computing device outputs the second character sequence.Type: GrantFiled: August 8, 2019Date of Patent: November 24, 2020Assignee: Google LLCInventors: Thomas Deselaers, Daniel Martin Keysers, Abraham Murray, Shumin Zhai
-
Publication number: 20200257447Abstract: In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.Type: ApplicationFiled: April 30, 2020Publication date: August 13, 2020Applicant: Google LLCInventors: Shumin Zhai, Thomas Breuel, Ouais Alsharif, Yu Ouyang, Francoise Beaufays, Johan Schalkwyk
-
Publication number: 20200257391Abstract: A method includes a processor of an electronic device receiving first input signals from a first sensor in response to user contact at a first edge of the device and second input signals from a second sensor in response to user contact at a second edge of the electronic device. The first and second sensors are covered by a housing of the device. The processor determines an external context of the device based on analysis of the first input signals and the second input signals. The determined external context indicates at least a position of the device relative to a user or an orientation of the device relative to a user. Responsive to determining the external context, the electronic device executes a particular user input action.Type: ApplicationFiled: April 27, 2020Publication date: August 13, 2020Inventors: Tyler Reed Kugler, Alexey Polyudov, Seungyon Lee, Yun-Ling Lee, Philip Quinn, Kishore Sundara-Rajan, Shumin Zhai, Debanjan Mukherjee, James B. Miller, Isaac William Reynolds
-
Patent number: 10671281Abstract: In some examples, a computing device includes at least one processor; and at least one module, operable by the at least one processor to: output, for display at an output device, a graphical keyboard; receive an indication of a gesture detected at a location of a presence-sensitive input device, wherein the location of the presence-sensitive input device corresponds to a location of the output device that outputs the graphical keyboard; determine, based on at least one spatial feature of the gesture that is processed by the computing device using a neural network, at least one character string, wherein the at least one spatial feature indicates at least one physical property of the gesture; and output, for display at the output device, based at least in part on the processing of the at least one spatial feature of the gesture using the neural network, the at least one character string.Type: GrantFiled: January 30, 2019Date of Patent: June 2, 2020Assignee: Google LLCInventors: Shumin Zhai, Thomas Breuel, Ouais Alsharif, Yu Ouyang, Francoise Beaufays, Johan Schalkwyk