Patents by Inventor Larry Heck
Larry Heck has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240080528Abstract: Methods and apparatus for generating personalized menus and a media content interface are provided. In one example, different device specific media asset lists are generated at different user devices for the same user based on the media consumption history at each device. In another example, media asset listings are displayed for multicast and on-demand media assets determined to be of interest to the user. An on-demand media asset is listed at a time when no other multicast media assets are listed. In another example, media asset listings for successive episodes of a media series are displayed in sequential order in response to an input to view successive episodes of a media series. In another example, a text search for media content is performed in which the search priority of media provider results is decreased relative to the search priority of media asset results as additional characters are added to a search string.Type: ApplicationFiled: October 20, 2023Publication date: March 7, 2024Inventors: Amit Bagga, Larry Cynkin, Jan Neumann, Hans Sayyadi, Ehsan Younessian, Mevan Samarasinghe, Brian Curtis, Muthu Manickam, Oliver Jojic, Jeanine Heck
-
Patent number: 10901500Abstract: Improving accuracy in understanding and/or resolving references to visual elements in a visual context associated with a computerized conversational system is described. Techniques described herein leverage gaze input with gestures and/or speech input to improve spoken language understanding in computerized conversational systems. Leveraging gaze input and speech input improves spoken language understanding in conversational systems by improving the accuracy by which the system can resolve references—or interpret a user's intent—with respect to visual elements in a visual context. In at least one example, the techniques herein describe tracking gaze to generate gaze input, recognizing speech input, and extracting gaze features and lexical features from the user input. Based at least in part on the gaze features and lexical features, user utterances directed to visual elements in a visual context can be resolved.Type: GrantFiled: April 30, 2019Date of Patent: January 26, 2021Assignee: Microsoft Technology Licensing, LLCInventors: Anna Prokofieva, Fethiye Asli Celikyilmaz, Dilek Z Hakkani-Tur, Larry Heck, Malcolm Slaney
-
Publication number: 20200175384Abstract: Methods, devices, and computer-readable media for incremental learning in image classification and/or object detection. A method for incremental learning includes identifying, for a model for object detection or classification, a first set of object classes the model is trained to detect or classify and adapting the model for use with a second set of object classes different from the first set of object classes to generate an adapted model. The method further includes retaining detection or classification performance on the first set of object classes in the adapted model by performing a knowledge distillation process for the model; and using the adapted model to detect or classify one or more objects from the first set of object classes and one or more objects from the second set of object classes.Type: ApplicationFiled: January 23, 2019Publication date: June 4, 2020Inventors: Junting Zhang, Jie Zhang, Shalini Ghosh, Dawei Li, Serafettin Tasci, Larry Heck
-
Patent number: 10572602Abstract: Tools are provided to allow developers to enable applications for Conversational Understanding (CU) using assets from a CU service. The tools may be used to select functionality from existing domains, extend the coverage of one or more domains, as well as to create new domains in the CU service. A developer may provide example Natural Language (NL) sentences that are analyzed by the tools to assist the developer in labeling data that is used to update the models in the CU service. For example, the tools may assist a developer in identifying domains, determining intent actions, determining intent objects and determining slots from example NL sentences. After the developer tags all or a portion of the example NL sentences, the models in the CU service are automatically updated and validated. For example, validation tools may be used to determine an accuracy of the model against test data.Type: GrantFiled: May 22, 2017Date of Patent: February 25, 2020Assignee: Microsoft Technology Licensing, LLCInventors: Ruhi Sarikaya, Daniel Boies, Larry Heck, Tasos Anastasakos
-
Patent number: 10529321Abstract: Prosodic features are used for discriminating computer-directed speech from human-directed speech. Statistics and models describing energy/intensity patterns over time, speech/pause distributions, pitch patterns, vocal effort features, and speech segment duration patterns may be used for prosodic modeling. The prosodic features for at least a portion of an utterance are monitored over a period of time to determine a shape associated with the utterance. A score may be determined to assist in classifying the current utterance as human directed or computer directed without relying on knowledge of preceding utterances or utterances following the current utterance. Outside data may be used for training lexical addressee detection systems for the H-H-C scenario. H-C training data can be obtained from a single-user H-C collection and that H-H speech can be modeled using general conversational speech. H-C and H-H language models may also be adapted using interpolation with small amounts of matched H-H-C data.Type: GrantFiled: August 7, 2017Date of Patent: January 7, 2020Assignee: Microsoft Technology Licensing, LLCInventors: Elizabeth Shriberg, Andreas Stolcke, Dilek Hakkani-Tur, Larry Heck, Heeyoung Lee
-
Publication number: 20190391640Abstract: Improving accuracy in understanding and/or resolving references to visual elements in a visual context associated with a computerized conversational system is described. Techniques described herein leverage gaze input with gestures and/or speech input to improve spoken language understanding in computerized conversational systems. Leveraging gaze input and speech input improves spoken language understanding in conversational systems by improving the accuracy by which the system can resolve references—or interpret a user's intent—with respect to visual elements in a visual context. In at least one example, the techniques herein describe tracking gaze to generate gaze input, recognizing speech input, and extracting gaze features and lexical features from the user input. Based at least in part on the gaze features and lexical features, user utterances directed to visual elements in a visual context can be resolved.Type: ApplicationFiled: April 30, 2019Publication date: December 26, 2019Inventors: Anna Prokofieva, Fethiye Asli Celikyilmaz, Dilek Z Hakkani-Tur, Larry Heck, Malcom Slaney
-
Patent number: 10317992Abstract: Improving accuracy in understanding and/or resolving references to visual elements in a visual context associated with a computerized conversational system is described. Techniques described herein leverage gaze input with gestures and/or speech input to improve spoken language understanding in computerized conversational systems. Leveraging gaze input and speech input improves spoken language understanding in conversational systems by improving the accuracy by which the system can resolve references—or interpret a user's intent—with respect to visual elements in a visual context. In at least one example, the techniques herein describe tracking gaze to generate gaze input, recognizing speech input, and extracting gaze features and lexical features from the user input. Based at least in part on the gaze features and lexical features, user utterances directed to visual elements in a visual context can be resolved.Type: GrantFiled: September 25, 2014Date of Patent: June 11, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Anna Prokofieva, Fethiye Asli Celikyilmaz, Dilek Z Hakkani-Tur, Larry Heck, Malcolm Slaney
-
Patent number: 10304448Abstract: Environmental conditions, along with other information, are used to adjust a response of a conversational dialog system. The environmental conditions may be used at different times within the conversational dialog system. For example, the environmental conditions can be used to adjust the dialog manager's output (e.g., the machine action). The dialog state information that is used by the dialog manager includes environmental conditions for the current turn in the dialog as well as environmental conditions for one or more past turns in the dialog. The environmental conditions can also be used after receiving the machine action to adjust the response that is provided to the user. For example, the environmental conditions may affect the machine action that is determined as well as how the action is provided to the user. The dialog manager and the response generation components in the conversational dialog system each use the available environmental conditions.Type: GrantFiled: February 21, 2017Date of Patent: May 28, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Daniel Boies, Larry Heck, Tasos Anastasakos, Ruhi Sarikaya
-
Patent number: 10235358Abstract: Structured web pages are accessed and parsed to obtain implicit annotation for natural language understanding tasks. Search queries that hit these structured web pages are automatically mined for information that is used to semantically annotate the queries. The automatically annotated queries may be used for automatically building statistical unsupervised slot filling models without using a semantic annotation guideline. For example, tags that are located on a structured web page that are associated with the search query may be used to annotate the query. The mined search queries may be filtered to create a set of queries that is in a form of a natural language query and/or remove queries that are difficult to parse. A natural language model may be trained using the resulting mined queries. Some queries may be set aside for testing and the model may be adapted using in-domain sentences that are not annotated.Type: GrantFiled: February 21, 2013Date of Patent: March 19, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Gokhan Tur, Dilek Hakkani-Tur, Larry Heck, Minwoo Jeong, Ye-Yi Wang
-
Publication number: 20190051293Abstract: Prosodic features are used for discriminating computer-directed speech from human-directed speech. Statistics and models describing energy/intensity patterns over time, speech/pause distributions, pitch patterns, vocal effort features, and speech segment duration patterns may be used for prosodic modeling. The prosodic features for at least a portion of an utterance are monitored over a period of time to determine a shape associated with the utterance. A score may be determined to assist in classifying the current utterance as human directed or computer directed without relying on knowledge of preceding utterances or utterances following the current utterance. Outside data may be used for training lexical addressee detection systems for the H-H-C scenario. H-C training data can be obtained from a single-user H-C collection and that H-H speech can be modeled using general conversational speech. H-C and H-H language models may also be adapted using interpolation with small amounts of matched H-H-C data.Type: ApplicationFiled: August 7, 2017Publication date: February 14, 2019Applicant: Microsoft Technology Licensing, LLCInventors: Elizabeth Shriberg, Andreas Stolcke, Dilek Hakkani-Tur, Larry Heck, Heeyoung Lee
-
Patent number: 9830039Abstract: A wizard control panel may be used by a human wizard to adjust the operation of a Natural Language (NL) conversational system during a real-time dialog flow. Input to the wizard control panel is detected and used to interrupt/change an automatic operation of one or more of the NL conversational system components used during the flow. For example, the wizard control panel may be used to adjust results determined by an Automated Speech Recognition (ASR) component, a Natural Language Understanding (NLU) component, a Dialog Manager (DM) component, and a Natural Language Generation (NLG) before the results are used to perform an automatic operation within the flow. A timeout may also be set such that when the timeout expires, the conversational system performs an automated operation by using the results shown in the wizard control panel (edited/not edited).Type: GrantFiled: March 4, 2013Date of Patent: November 28, 2017Assignee: Microsoft Technology Licensing, LLCInventors: Lisa Stifelman, Dilek Hakkani-Tur, Larry Heck, Madhusudan Chinthakunta
-
Patent number: 9761247Abstract: Prosodic features are used for discriminating computer-directed speech from human-directed speech. Statistics and models describing energy/intensity patterns over time, speech/pause distributions, pitch patterns, vocal effort features, and speech segment duration patterns may be used for prosodic modeling. The prosodic features for at least a portion of an utterance are monitored over a period of time to determine a shape associated with the utterance. A score may be determined to assist in classifying the current utterance as human directed or computer directed without relying on knowledge of preceding utterances or utterances following the current utterance. Outside data may be used for training lexical addressee detection systems for the H-H-C scenario. H-C training data can be obtained from a single-user H-C collection and that H-H speech can be modeled using general conversational speech. H-C and H-H language models may also be adapted using interpolation with small amounts of matched H-H-C data.Type: GrantFiled: January 31, 2013Date of Patent: September 12, 2017Assignee: Microsoft Technology Licensing, LLCInventors: Elizabeth Shriberg, Andreas Stolcke, Dilek Hakkani-Tur, Larry Heck, Heeyoung Lee
-
Publication number: 20170255612Abstract: Tools are provided to allow developers to enable applications for Conversational Understanding (CU) using assets from a CU service. The tools may be used to select functionality from existing domains, extend the coverage of one or more domains, as well as to create new domains in the CU service. A developer may provide example Natural Language (NL) sentences that are analyzed by the tools to assist the developer in labeling data that is used to update the models in the CU service. For example, the tools may assist a developer in identifying domains, determining intent actions, determining intent objects and determining slots from example NL sentences. After the developer tags all or a portion of the example NL sentences, the models in the CU service are automatically updated and validated. For example, validation tools may be used to determine an accuracy of the model against test data.Type: ApplicationFiled: May 22, 2017Publication date: September 7, 2017Applicant: Microsoft Technology Licensing, LLCInventors: Ruhi Sarikaya, Daniel Boies, Larry Heck, Tasos Anastasakos
-
Publication number: 20170199909Abstract: A device may facilitate a query dialog involving queries that successively modify a query state. However, fulfilling such queries in the context of possible query domains, query intents, and contextual meanings of query terms may be difficult. Presented herein are techniques for modifying a query state in view of a query by utilizing a set of query state modifications, each representing a modification of the query state possibly intended by the user while formulating the query (e.g., adding, substituting, or removing query terms; changing the query domain or query intent; and navigating within a hierarchy of saved query states). Upon receiving a query, an embodiment may calculate the probability of the query connoting each query state modification (e.g., using a Bayesian classifier), and parsing the query according to a query state modification having a high probability (e.g., mapping respective query terms to query slots within the current query intent).Type: ApplicationFiled: March 24, 2017Publication date: July 13, 2017Applicant: Microsoft Technology Licensing, LLCInventors: Dilek Hakkani-Tur, Gokhan Tur, Larry Heck, Ashley Fidler, Fehtiye Asli Celikyilmaz
-
Patent number: 9697200Abstract: Tools are provided to allow developers to enable applications for Conversational Understanding (CU) using assets from a CU service. The tools may be used to select functionality from existing domains, extend the coverage of one or more domains, as well as to create new domains in the CU service. A developer may provide example Natural Language (NL) sentences that are analyzed by the tools to assist the developer in labeling data that is used to update the models in the CU service. For example, the tools may assist a developer in identifying domains, determining intent actions, determining intent objects and determining slots from example NL sentences. After the developer tags all or a portion of the example NL sentences, the models in the CU service are automatically updated and validated. For example, validation tools may be used to determine an accuracy of the model against test data.Type: GrantFiled: March 21, 2016Date of Patent: July 4, 2017Assignee: Microsoft Technology Licensing, LLCInventors: Ruhi Sarikaya, Daniel Boies, Larry Heck, Tasos Anastasakos
-
Publication number: 20170162201Abstract: Environmental conditions, along with other information, are used to adjust a response of a conversational dialog system. The environmental conditions may be used at different times within the conversational dialog system. For example, the environmental conditions can be used to adjust the dialog manager's output (e.g., the machine action). The dialog state information that is used by the dialog manager includes environmental conditions for the current turn in the dialog as well as environmental conditions for one or more past turns in the dialog. The environmental conditions can also be used after receiving the machine action to adjust the response that is provided to the user. For example, the environmental conditions may affect the machine action that is determined as well as how the action is provided to the user. The dialog manager and the response generation components in the conversational dialog system each use the available environmental conditions.Type: ApplicationFiled: February 21, 2017Publication date: June 8, 2017Applicant: Microsoft Technology Licensing, LLCInventors: Daniel Boies, Larry Heck, Tasos Anastasakos, Ruhi Sarikaya
-
Patent number: 9607046Abstract: A device may facilitate a query dialog involving queries that successively modify a query state. However, fulfilling such queries in the context of possible query domains, query intents, and contextual meanings of query terms may be difficult. Presented herein are techniques for modifying a query state in view of a query by utilizing a set of query state modifications, each representing a modification of the query state possibly intended by the user while formulating the query (e.g., adding, substituting, or removing query terms; changing the query domain or query intent; and navigating within a hierarchy of saved query states). Upon receiving a query, an embodiment may calculate the probability of the query connoting each query state modification (e.g., using a Bayesian classifier), and parsing the query according to a query state modification having a high probability (e.g., mapping respective query terms to query slots within the current query intent).Type: GrantFiled: December 14, 2012Date of Patent: March 28, 2017Assignee: Microsoft Technology Licensing, LLCInventors: Dilek Hakkani-Tur, Gokhan Tur, Larry Heck, Ashley Fidler, Fehtiye Asli Celikyilmaz
-
Patent number: 9589565Abstract: Environmental conditions, along with other information, are used to adjust a response of a conversational dialog system. The environmental conditions may be used at different times within the conversational dialog system. For example, the environmental conditions can be used to adjust the dialog manager's output (e.g., the machine action). The dialog state information that is used by the dialog manager includes environmental conditions for the current turn in the dialog as well as environmental conditions for one or more past turns in the dialog. The environmental conditions can also be used after receiving the machine action to adjust the response that is provided to the user. For example, the environmental conditions may affect the machine action that is determined as well as how the action is provided to the user. The dialog manager and the response generation components in the conversational dialog system each use the available environmental conditions.Type: GrantFiled: June 21, 2013Date of Patent: March 7, 2017Assignee: Microsoft Technology Licensing, LLCInventors: Daniel Boies, Larry Heck, Tasos Anastasakos, Ruhi Sarikaya
-
Publication number: 20160203125Abstract: Tools are provided to allow developers to enable applications for Conversational Understanding (CU) using assets from a CU service. The tools may be used to select functionality from existing domains, extend the coverage of one or more domains, as well as to create new domains in the CU service. A developer may provide example Natural Language (NL) sentences that are analyzed by the tools to assist the developer in labeling data that is used to update the models in the CU service. For example, the tools may assist a developer in identifying domains, determining intent actions, determining intent objects and determining slots from example NL sentences. After the developer tags all or a portion of the example NL sentences, the models in the CU service are automatically updated and validated. For example, validation tools may be used to determine an accuracy of the model against test data.Type: ApplicationFiled: March 21, 2016Publication date: July 14, 2016Applicant: Microsoft Technology Licensing, LLCInventors: Ruhi Sarikaya, Daniel Boies, Larry Heck, Tasos Anastasakos
-
Patent number: 9311298Abstract: Tools are provided to allow developers to enable applications for Conversational Understanding (CU) using assets from a CU service. The tools may be used to select functionality from existing domains, extend the coverage of one or more domains, as well as to create new domains in the CU service. A developer may provide example Natural Language (NL) sentences that are analyzed by the tools to assist the developer in labeling data that is used to update the models in the CU service. For example, the tools may assist a developer in identifying domains, determining intent actions, determining intent objects and determining slots from example NL sentences. After the developer tags all or a portion of the example NL sentences, the models in the CU service are automatically updated and validated. For example, validation tools may be used to determine an accuracy of the model against test data.Type: GrantFiled: June 21, 2013Date of Patent: April 12, 2016Assignee: Microsoft Technology Licensing, LLCInventors: Ruhi Sarikaya, Daniel Boies, Larry Heck, Tasos Anastasakos