Patents by Inventor Paul Heck
Paul Heck has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11961522Abstract: The disclosure relates to an electronic apparatus for recognizing user voice and a method of recognizing, by the electronic apparatus, the user voice. According to an embodiment, the method of recognizing the user voice includes obtaining an audio signal segmented into a plurality of frame units, determining an energy component for each filter bank by applying a filter bank distributed according to a preset scale to a frequency spectrum of the audio signal segmented into the frame units, smoothing the determined energy component for each filter bank, extracting a feature vector of the audio signal based on the smoothed energy component for each filter bank, and recognizing the user voice in the audio signal by inputting the extracted feature vector to a voice recognition model.Type: GrantFiled: November 22, 2019Date of Patent: April 16, 2024Assignee: Samsung Electronics Co., Ltd.Inventors: Chanwoo Kim, Dhananjaya N. Gowda, Sungsoo Kim, Minkyu Shin, Larry Paul Heck, Abhinav Garg, Kwangyoun Kim, Mehul Kumar
-
Publication number: 20230419960Abstract: Determining a dialog state of an electronic dialog that includes an automated assistant and at least one user, and performing action(s) based on the determined dialog state. The dialog state can be represented as one or more slots and, for each of the slots, one or more candidate values for the slot and a corresponding score (e.g., a probability) for each of the candidate values. Candidate values for a slot can be determined based on language processing of user utterance(s) and/or system utterance(s) during the dialog. In generating scores for candidate value(s) of a given slot at a given turn of an electronic dialog, various features are determined based on processing of the user utterance and the system utterance using a memory network. The various generated features can be processed using a scoring model to generate scores for candidate value(s) of the given slot at the given turn.Type: ApplicationFiled: September 13, 2023Publication date: December 28, 2023Inventors: Abhinav Rastogi, Larry Paul Heck, Dilek Hakkani-Tur
-
Patent number: 11790899Abstract: Determining a dialog state of an electronic dialog that includes an automated assistant and at least one user, and performing action(s) based on the determined dialog state. The dialog state can be represented as one or more slots and, for each of the slots, one or more candidate values for the slot and a corresponding score (e.g., a probability) for each of the candidate values. Candidate values for a slot can be determined based on language processing of user utterance(s) and/or system utterance(s) during the dialog. In generating scores for candidate value(s) of a given slot at a given turn of an electronic dialog, various features are determined based on processing of the user utterance and the system utterance using a memory network. The various generated features can be processed using a scoring model to generate scores for candidate value(s) of the given slot at the given turn.Type: GrantFiled: November 19, 2020Date of Patent: October 17, 2023Assignee: GOOGLE LLCInventors: Abhinav Rastogi, Larry Paul Heck, Dilek Hakkani-Tur
-
Publication number: 20230206911Abstract: Determining slot value(s) based on received natural language input and based on descriptor(s) for the slot(s). In some implementations, natural language input is received as part of human-to-automated assistant dialog. A natural language input embedding is generated based on token(s) of the natural language input. Further, descriptor embedding(s) are generated (or received), where each of the descriptor embeddings is generated based on descriptor(s) for a corresponding slot that is assigned to a domain indicated by the dialog. The natural language input embedding and the descriptor embedding(s) are applied to layer(s) of a neural network model to determine, for each of the slot(s), which token(s) of the natural language input correspond to the slot. A command is generated that includes slot value(s) for slot(s), where the slot value(s) for one or more of the slot(s) are determined based on the token(s) determined to correspond to the slot(s).Type: ApplicationFiled: March 1, 2023Publication date: June 29, 2023Inventors: Ankur Bapna, Larry Paul Heck
-
Patent number: 11610579Abstract: Determining slot value(s) based on received natural language input and based on descriptor(s) for the slot(s). In some implementations, natural language input is received as part of human-to-automated assistant dialog. A natural language input embedding is generated based on token(s) of the natural language input. Further, descriptor embedding(s) are generated (or received), where each of the descriptor embeddings is generated based on descriptor(s) for a corresponding slot that is assigned to a domain indicated by the dialog. The natural language input embedding and the descriptor embedding(s) are applied to layer(s) of a neural network model to determine, for each of the slot(s), which token(s) of the natural language input correspond to the slot. A command is generated that includes slot value(s) for slot(s), where the slot value(s) for one or more of slot(s) are determined based on the token(s) determined to correspond to the slot(s).Type: GrantFiled: June 18, 2017Date of Patent: March 21, 2023Assignee: GOOGLE LLCInventors: Ankur Bapna, Larry Paul Heck
-
Publication number: 20220005481Abstract: The disclosure relates to an electronic apparatus for recognizing user voice and a method of recognizing, by the electronic apparatus, the user voice. According to an embodiment, the method of recognizing the user voice includes obtaining an audio signal segmented into a plurality of frame units, determining an energy component for each filter bank by applying a filter bank distributed according to a preset scale to a frequency spectrum of the audio signal segmented into the frame units, smoothing the determined energy component for each filter bank, extracting a feature vector of the audio signal based on the smoothed energy component for each filter bank, and recognizing the user voice in the audio signal by inputting the extracted feature vector to a voice recognition model.Type: ApplicationFiled: November 22, 2019Publication date: January 6, 2022Inventors: Chanwoo KIM, Dhananjaya N. GOWDA, Sungsoo KIM, Minkyu SHIN, Larry Paul HECK, Abhinav GARG, Kwangyoun KIM, Mehul KUMAR
-
Publication number: 20210074279Abstract: Determining a dialog state of an electronic dialog that includes an automated assistant and at least one user, and performing action(s) based on the determined dialog state. The dialog state can be represented as one or more slots and, for each of the slots, one or more candidate values for the slot and a corresponding score (e.g., a probability) for each of the candidate values. Candidate values for a slot can be determined based on language processing of user utterance(s) and/or system utterance(s) during the dialog. In generating scores for candidate value(s) of a given slot at a given turn of an electronic dialog, various features are determined based on processing of the user utterance and the system utterance using a memory network. The various generated features can be processed using a scoring model to generate scores for candidate value(s) of the given slot at the given turn.Type: ApplicationFiled: November 19, 2020Publication date: March 11, 2021Inventors: Abhinav Rastogi, Larry Paul Heck, Dilek Hakkani-Tur
-
Patent number: 10878009Abstract: Natural language query translation may be provided. A statistical model may be trained to detect domains according to a plurality of query click log data. Upon receiving a natural language query, the statistical model may be used to translate the natural language query into an action. The action may then be performed and at least one result associated with performing the action may be provided.Type: GrantFiled: July 24, 2018Date of Patent: December 29, 2020Assignee: Microsoft Technology Licensing, LLCInventors: Dilek Zeynep Hakkani-Tur, Gokhan Tur, Rukmini Iyer, Larry Paul Heck
-
Patent number: 10867599Abstract: Determining a dialog state of an electronic dialog that includes an automated assistant and at least one user, and performing action(s) based on the determined dialog state. The dialog state can be represented as one or more slots and, for each of the slots, one or more candidate values for the slot and a corresponding score (e.g., a probability) for each of the candidate values. Candidate values for a slot can be determined based on language processing of user utterance(s) and/or system utterance(s) during the dialog. In generating scores for candidate value(s) of a given slot at a given turn of an electronic dialog, various features are determined based on processing of the user utterance and the system utterance using a memory network. The various generated features can be processed using a scoring model to generate scores for candidate value(s) of the given slot at the given turn.Type: GrantFiled: October 12, 2017Date of Patent: December 15, 2020Assignee: GOOGLE LLCInventors: Abhinav Rastogi, Larry Paul Heck, Dilek Hakkani-Tur
-
Publication number: 20200320988Abstract: Determining a dialog state of an electronic dialog that includes an automated assistant and at least one user, and performing action(s) based on the determined dialog state. The dialog state can be represented as one or more slots and, for each of the slots, one or more candidate values for the slot and a corresponding score (e.g., a probability) for each of the candidate values. Candidate values for a slot can be determined based on language processing of user utterance(s) and/or system utterance(s) during the dialog. In generating scores for candidate value(s) of a given slot at a given turn of an electronic dialog, various features are determined based on processing of the user utterance and the system utterance using a memory network. The various generated features can be processed using a scoring model to generate scores for candidate value(s) of the given slot at the given turn.Type: ApplicationFiled: October 12, 2017Publication date: October 8, 2020Inventors: Abhinav Rastogi, Larry Paul Heck, Dilek Hakkani-Tur
-
Publication number: 20200202846Abstract: Determining slot value(s) based on received natural language input and based on descriptor(s) for the slot(s). In some implementations, natural language input is received as part of human-to-automated assistant dialog. A natural language input embedding is generated based on token(s) of the natural language input. Further, descriptor embedding(s) are generated (or received), where each of the descriptor embeddings is generated based on descriptor(s) for a corresponding slot that is assigned to a domain indicated by the dialog. The natural language input embedding and the descriptor embedding(s) are applied to layer(s) of a neural network model to determine, for each of the slot(s), which token(s) of the natural language input correspond to the slot. A command is generated that includes slot value(s) for slot(s), where the slot value(s) for one or more of slot(s) are determined based on the token(s) determined to correspond to the slot(s).Type: ApplicationFiled: June 18, 2017Publication date: June 25, 2020Inventors: Ankur BAPNA, Larry Paul HECK
-
Patent number: 10642934Abstract: An augmented conversational understanding architecture may be provided. Upon receiving a natural language phrase from a user, the phrase may be translated into a search phrase and a search action may be performed on the search phrase.Type: GrantFiled: March 31, 2011Date of Patent: May 5, 2020Assignee: MICROSOFT TECHNOLOGY LICENSING, LLCInventors: Larry Paul Heck, Madhusudan Chinthakunta, David Mitby, Lisa Stifelman
-
Patent number: 10585957Abstract: Identification of user intents may be provided. A plurality of network applications may be identified, and an ontology associated with each of the plurality of applications may be defined. If a phrase received from a user is associated with at least one of the defined ontologies, an action associated with the network application may be executed.Type: GrantFiled: November 20, 2017Date of Patent: March 10, 2020Assignee: Microsoft Technology Licensing, LLCInventors: Larry Paul Heck, Madhusudan Chinthakunta, David Mitby, Lisa Stifelman
-
Publication number: 20200036762Abstract: Individuals may utilize devices to engage in conversations about topics respectively associated with a location (e.g., restaurants where the individuals may meet for dinner). Often, the individual momentarily withdraws from the conversation in order to issue commands to the device to retrieve and present such information, and may miss parts of the conversation while interacting with the device. Additionally, the individual often explores such topics individually on a device and conveys such information to the other individuals through messages, which is inefficient and error-prone. Presented herein are techniques enabling devices to facilitate conversations by monitoring the conversation for references, by one individual to another (rather than as a command to the device), to a topic associated with a location. In the absence of a command from an individual, the device may automatically present a map alongside a conversation interface showing the location(s) of the topic(s) referenced in the conversation.Type: ApplicationFiled: August 5, 2019Publication date: January 30, 2020Inventors: Lisa Stifelman, Madhusudan Chinthakunta, Julian James Odell, Larry Paul Heck, Daniel Dole
-
Patent number: 10529326Abstract: Techniques are described herein that are capable of suggesting intent frame(s) for user request(s). For instance, the intent frame(s) may be suggested to elicit a request from a user. An intent frame is a natural language phrase (e.g., a sentence) that includes at least one carrier phrase and at least one slot. A slot in an intent frame is a placeholder that is identified as being replaceable by one or more words that identify an entity and/or an action to indicate an intent of the user. A carrier phrase in an intent frame includes one or more words that suggest a type of entity and/or action that is to be identified by the one or more words that may replace the corresponding slot. In accordance with these techniques, the intent frame(s) are suggested in response to determining that natural language functionality of a processing system is activated.Type: GrantFiled: January 13, 2017Date of Patent: January 7, 2020Assignee: Microsoft Technology Licensing, LLCInventors: Shane J. Landry, Anne K. Sullivan, Lisa J. Stifelman, Adam D. Elman, Larry Paul Heck, Sarangarajan Parthasarathy
-
Patent number: 10424302Abstract: Techniques are described related to turn-based reinforcement learning for dialog management. In various implementations, dialog states and corresponding responsive actions generated during a multi-turn human-to-computer dialog session may be obtained. A plurality of turn-level training instances may be generated, each including: a given dialog state of the plurality of dialog states at an outset of a given turn of the human-to-computer dialog session; and a given responsive action that was selected based on the given dialog state. One or more of the turn-level training instances may further include a turn-level feedback value that reflects on the given responsive action selected during the given turn. A reward value may be generated based on an outcome of the human-to-computer dialog session. The dialog management policy model may be trained based on turn-level feedback values of the turn-level training instance(s) and the reward value.Type: GrantFiled: October 12, 2017Date of Patent: September 24, 2019Assignee: GOOGLE LLCInventors: Pararth Shah, Larry Paul Heck, Dilek Hakkani-Tur
-
Patent number: 10375129Abstract: Individuals may utilize devices to engage in conversations about topics respectively associated with a location (e.g., restaurants where the individuals may meet for dinner). Often, the individual momentarily withdraws from the conversation in order to issue commands to the device to retrieve and present such information, and may miss parts of the conversation while interacting with the device. Additionally, the individual often explores such topics individually on a device and conveys such information to the other individuals through messages, which is inefficient and error-prone. Presented herein are techniques enabling devices to facilitate conversations by monitoring the conversation for references, by one individual to another (rather than as a command to the device), to a topic associated with a location. In the absence of a command from an individual, the device may automatically present a map alongside a conversation interface showing the location(s) of the topic(s) referenced in the conversation.Type: GrantFiled: June 17, 2014Date of Patent: August 6, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Lisa Stifelman, Madhusudan Chinthakunta, Julian James Odell, Larry Paul Heck, Daniel Dole
-
Patent number: 10296587Abstract: An augmented conversational understanding agent may be provided. Upon receiving, by an agent, at least one natural language phrase from a user, a context associated with the at least one natural language phrase may be identified. The natural language phrase may be associated, for example, with a conversation between the user and a second user. An agent action associated with the identified context may be performed according to the at least one natural language phrase and 201 a result associated with performing the action may be displayed.Type: GrantFiled: June 12, 2017Date of Patent: May 21, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Larry Paul Heck, Madhusudan Chinthakunta, David Mitby, Lisa Stifelman
-
Publication number: 20190115027Abstract: Techniques are described related to turn-based reinforcement learning for dialog management. In various implementations, dialog states and corresponding responsive actions generated during a multi-turn human-to-computer dialog session may be obtained. A plurality of turn-level training instances may be generated, each including: a given dialog state of the plurality of dialog states at an outset of a given turn of the human-to-computer dialog session; and a given responsive action that was selected based on the given dialog state. One or more of the turn-level training instances may further include a turn-level feedback value that reflects on the given responsive action selected during the given turn. A reward value may be generated based on an outcome of the human-to-computer dialog session. The dialog management policy model may be trained based on turn-level feedback values of the turn-level training instance(s) and the reward value.Type: ApplicationFiled: October 12, 2017Publication date: April 18, 2019Inventors: Pararth Shah, Larry Paul Heck, Dilek Hakkani-Tur
-
Patent number: 10191999Abstract: Aspects of the present invention provide a technique to validate the transfer of intents or entities between existing natural language model domains (hereafter “domain” or “NLU”) using click logs, a knowledge graph, or both. At least two different types of transfers are possible. Intents from a first domain may be transferred to a second domain. Alternatively or additionally, entities from the second domain may be transferred to an existing intent in the first domain. Either way, additional intent/entity pairs can be generated and validated. Before the new intent/entity pair is added to a domain, aspects of the present invention validate that the intent or entity is transferable between domains. Validation techniques that are consistent with aspects of the invention can use a knowledge graph, search query click logs, or both to validate a transfer of intents or entities from one domain to another.Type: GrantFiled: April 30, 2014Date of Patent: January 29, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Xiaohu Liu, Ali Mamdouh Elkahky, Ruhi Sarikaya, Gokhan Tur, Dilek Hakkani-Tur, Larry Paul Heck