Patents by Inventor Gengshen Fu
Gengshen Fu has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20250149036Abstract: Systems and methods for preemptive wakeword detection are disclosed. For example, a first part of a wakeword is detected from audio data representing a user utterance. When this occurs, on-device speech processing is initiated prior to when the entire wakeword is detected. When the entire wakeword is detected, results from the on-device speech processing and/or the audio data is sent to a speech processing system to determine a responsive action to be performed by the device. When the entire wakeword is not detected, on-device processing is canceled and the device refrains from sending the audio data to the speech processing system.Type: ApplicationFiled: December 3, 2024Publication date: May 8, 2025Inventors: Eli Joshua Fidler, Aaron Challenner, Zoe Adams, Sree Hari Krishnan Parthasarathi, Gengshen Fu
-
Patent number: 12190875Abstract: Systems and methods for preemptive wakeword detection are disclosed. For example, a first part of a wakeword is detected from audio data representing a user utterance. When this occurs, on-device speech processing is initiated prior to when the entire wakeword is detected. When the entire wakeword is detected, results from the on-device speech processing and/or the audio data is sent to a speech processing system to determine a responsive action to be performed by the device. When the entire wakeword is not detected, on-device processing is canceled and the device refrains from sending the audio data to the speech processing system.Type: GrantFiled: September 30, 2021Date of Patent: January 7, 2025Assignee: Amazon Technologies, Inc.Inventors: Eli Joshua Fidler, Aaron Challenner, Zoe Adams, Sree Hari Krishnan Parthasarathi, Gengshen Fu
-
Publication number: 20240412728Abstract: A device is configured to detect multiple different wakewords. A device may operate a joint encoder that operates on audio data to determine encoded audio data. The device may operate multiple different decoders which process the encoded audio data to determine if a wakeword is detected. Each decoder may correspond to a different wakeword. The decoders may use fewer computing resources than the joint encoder, allowing for the device to more easily perform multiple wakeword processing. Enabling/disabling wakeword(s) may involve the reconfiguring of a wakeword detector to add/remove data for respective decoder(s). Specific decoders may be activated/deactivated depending on device context, thereby efficiently managing device resources.Type: ApplicationFiled: June 12, 2023Publication date: December 12, 2024Inventors: Michael Thomas Peterson, Gengshen Fu, Aaron Challenner, Rong Chen, Cody Jacques, Stefan M Bradstreet
-
Patent number: 11699433Abstract: Techniques for using a dynamic wakeword detection threshold are described. A device detects a wakeword in audio data using a first wakeword detection threshold value. Thereafter, the device receives audio including speech. If the device receives the audio within a predetermined duration of time after detecting the previous wakeword, the device attempts to detect a wakeword in second audio data, corresponding to the audio including the speech, using a second, lower wakeword detection threshold value.Type: GrantFiled: July 23, 2020Date of Patent: July 11, 2023Assignee: Amazon Technologies, Inc.Inventors: Gengshen Fu, Shiv Naga Prasad Vitaladevuni, Paul McIntyre, Shuang Wu
-
Publication number: 20230186902Abstract: A device is configured to detect multiple different wakewords. A device may operate a joint encoder that operates on audio data to determine encoded audio data. The device may operate multiple different decoders which process the encoded audio data to determine if a wakeword is detected. Each decoder may correspond to a different wakeword. The decoders may use fewer computing resources than the joint encoder, allowing for the device to more easily perform multiple wakeword processing. Enabling/disabling wakeword(s) may involve the reconfiguring of a wakeword detector to add/remove data for respective decoder(s).Type: ApplicationFiled: December 10, 2021Publication date: June 15, 2023Inventors: Gengshen Fu, Huitian Lei, Sai Kiran Venkata Subramanya Rupanagudi, Yuriy Mishchenko, Cody Jacques
-
Patent number: 11308939Abstract: A system and method performs wakeword detection and automatic speech recognition using the same acoustic model. A mapping engine maps phones/senones output by the acoustic model to phones/senones corresponding to the wakeword. A hidden Markov model (HMM) may determine that the wakeword is present in audio data; the HMM may have multiple paths for multiple wakewords or may have multiple models. Once the wakeword is detected, ASR is performed using the acoustic model.Type: GrantFiled: September 25, 2018Date of Patent: April 19, 2022Assignee: Amazon Technologies, Inc.Inventors: Yixin Gao, Ming Sun, Varun Nagaraja, Gengshen Fu, Chao Wang, Shiv Naga Prasad Vitaladevuni
-
Patent number: 11205420Abstract: A system and method performs wakeword detection using a neural network model that includes a recurrent neural network (RNN) for processing variable-length wakewords. To prevent the model from being influenced by non-wakeword speech, multiple instances of the model are created to process audio data, and each instance is configured to use weights determined by training data. The model may instead or in addition be used to process the audio data only when a likelihood that the audio data corresponds to the wakeword is greater than a threshold. The model may process the audio data as represented by groups of acoustic feature vectors; computations for feature vectors common to different groups may be re-used.Type: GrantFiled: June 10, 2019Date of Patent: December 21, 2021Assignee: Amazon Technologies, Inc.Inventors: Gengshen Fu, Thibaud Senechal, Shiv Naga Prasad Vitaladevuni, Michael J. Rodehorst, Varun K. Nagaraja
-
Patent number: 10872599Abstract: A device monitors audio data for a predetermined and/or user-defined wakeword. The device detects an error in detecting the wakeword in the audio data, such as a false-positive detection of the wakeword or a false-negative detection of the wakeword. Upon detecting the error, the device updates a model trained to detect the wakeword to create an updated trained model; the updated trained model reduces or eliminates further errors in detecting the wakeword. Data corresponding to the updated trained model may be collected by a server from a plurality of devices and used to create an updated trained model aggregating the data; this updated trained model may be sent to some or all of the devices.Type: GrantFiled: June 28, 2018Date of Patent: December 22, 2020Assignee: Amazon Technologies, Inc.Inventors: Shuang Wu, Thibaud Senechal, Gengshen Fu, Shiv Naga Prasad Vitaladevuni
-
Publication number: 20200388273Abstract: Techniques for using a dynamic wakeword detection threshold are described. A device detects a wakeword in audio data using a first wakeword detection threshold value. Thereafter, the device receives audio including speech. If the device receives the audio within a predetermined duration of time after detecting the previous wakeword, the device attempts to detect a wakeword in second audio data, corresponding to the audio including the speech, using a second, lower wakeword detection threshold value.Type: ApplicationFiled: July 23, 2020Publication date: December 10, 2020Inventors: Gengshen Fu, Shiv Naga Prasad Vitaladevuni, Paul McIntyre, Shuang Wu
-
Patent number: 10777189Abstract: Techniques for using a dynamic wakeword detection threshold are described. A device detects a wakeword in audio data using a first wakeword detection threshold value. Thereafter, the device receives audio including speech. If the device receives the audio within a predetermined duration of time after detecting the previous wakeword, the device attempts to detect a wakeword in second audio data, corresponding to the audio including the speech, using a second, lower wakeword detection threshold value.Type: GrantFiled: December 5, 2017Date of Patent: September 15, 2020Assignee: AMAZON TECHNOLOGIES, INC.Inventors: Gengshen Fu, Shiv Naga Prasad Vitaladevuni, Paul McIntyre, Shuang Wu
-
Patent number: 10510340Abstract: Techniques for using a dynamic wakeword detection threshold are described. A server(s) may receive audio data corresponding to an utterance from a device in response to the device detecting a wakeword using a wakeword detection threshold. The server(s) may then determine the device should use a lower wakeword detection threshold for a duration of time. In addition to sending the device output data responsive to the utterance, the server(s) may send the device an instruction to use the lower wakeword detection threshold for the duration of time. Alternatively, the server(s) may train a machine learning model to determine when the device should use a lower wakeword detection threshold. The server(s) may send the trained machine learned model to the device for use at runtime.Type: GrantFiled: December 5, 2017Date of Patent: December 17, 2019Assignee: Amazon Technologies, Inc.Inventors: Gengshen Fu, Shiv Naga Prasad Vitaladevuni, Paul McIntyre, Shuang Wu