Patents by Inventor Paul Anthony Crook
Paul Anthony Crook has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20210118442Abstract: In one embodiment, a method includes accessing visual data from a client system associated with a user, wherein the visual data comprises images portraying one or more objects, receiving, from the client system, a user request, wherein the user request comprises a coreference to a target object, resolving the coreference to the target object from among the one or more objects, resolving the target object to a specific entity, and sending, to the client system, instructions for providing a response to the user request, wherein the response comprises attribute information about the specific entity.Type: ApplicationFiled: August 28, 2020Publication date: April 22, 2021Inventors: Shivani Poddar, Seungwhan Moon, Paul Anthony Crook, Rajen Subba
-
Publication number: 20210117681Abstract: In one embodiment, a method includes receiving, from a client system associated with a user, a user request comprising a reference to a target object, accessing visual data from the client system, wherein the visual data comprises images portraying the target object and one or more additional objects, and wherein attribute information of the target object is recorded in a multimodal dialog state, resolving the reference to the target object based on the attribute information recorded in the multimodal dialog state, determining relational information between the target object and one or more of the additional objects portrayed in the visual data, and sending, to the client system, instructions for presenting a response to the user request, wherein the response comprises the attribute information and the determined relational information.Type: ApplicationFiled: August 28, 2020Publication date: April 22, 2021Inventors: Shivani Poddar, Seungwhan Moon, Paul Anthony Crook, Rajen Subba
-
Publication number: 20200401422Abstract: In one embodiment, a method includes receiving a user request from a first user from a client system associated with a first user, wherein the user request comprise a gesture-input from the first user and a speech-input from the first user, determining an intent corresponding to the user request based on the gesture-input by a personalized gesture-classification model associated with the first user, executing one or more tasks based on the determined intent and the speech-input, and sending instructions for presenting execution results of the one or more tasks to the client system responsive the user request.Type: ApplicationFiled: September 2, 2020Publication date: December 24, 2020Inventors: Xiaohu Liu, Paul Anthony Crook, Francislav P. Penov, Rajen Subba
-
Patent number: 10802848Abstract: In one embodiment, a method includes accessing a plurality of input tuples associated with a first user from a data store, wherein each input tuple comprises a gesture-input and a corresponding speech-input, determining a plurality of intents corresponding to the plurality of speech-inputs, respectively, by a natural-language understanding (NLU) module, generating a plurality of feature representations for the plurality of gesture-inputs based on one or more machine-learning models, determining a plurality of gesture identifiers for the plurality of gesture-inputs, respectively, based on their respective feature representations, associating the plurality of intents with the plurality of gesture identifiers, respectively, and training a personalized gesture-classification model for the first user based on the plurality of feature representations of their respective gesture-inputs and the associations between the plurality of intents and their respective gesture identifiers.Type: GrantFiled: April 18, 2019Date of Patent: October 13, 2020Assignee: Facebook Technologies, LLCInventors: Xiaohu Liu, Paul Anthony Crook, Francislav P. Penov, Rajen Subba
-
Publication number: 20200225839Abstract: Aspects herein provide third party application authors with a user interface authoring platform that automates and simplifies a task definition process while also providing the ability to leverage pre-existing language understanding models and canonicalization and resolution modules that are provided by the operating system on which the CU system resides or as provided by other third parties. In particular, the present disclosure provides a method and system for authoring a task using a user interface authoring platform.Type: ApplicationFiled: March 26, 2020Publication date: July 16, 2020Applicant: Microsoft Technology Licensing, LLCInventors: Marius Alexandru Marin, Paul Anthony Crook, Nikhil Holenarsipur Ramesh, Vipul Agarwal, Omar Zia Khan, Alexandre Rochette, Jean-Philippe Robichaud, Ruhi Sarikaya
-
Patent number: 10635281Abstract: Aspects herein provide third party application authors with a user interface authoring platform that automates and simplifies a task definition process while also providing the ability to leverage pre-existing language understanding models and canonicalization and resolution modules that are provided by the operating system on which the CU system resides or as provided by other third parties. In particular, the present disclosure provides a method and system for authoring a task using a user interface authoring platform.Type: GrantFiled: February 12, 2016Date of Patent: April 28, 2020Assignee: Microsoft Technology Licensing, LLCInventors: Marius Alexandru Marin, Paul Anthony Crook, Nikhil Holenarsipur Ramesh, Vipul Agarwal, Omar Zia Khan, Alexandre Rochette, Jean-Philippe Robichaud, Ruhi Sarikaya
-
Patent number: 10474439Abstract: Systems and methods for building conversational understanding systems are provided. More specifically, the systems and methods utilize prebuilt conversational items that can form a CU system upon selection by a builder without requiring any further input from the builder. Accordingly, the systems and methods for building a conversational understanding system reduce the expertise, time, and resources necessary to build a conversational understanding system for an application when compared to systems and methods that utilize conversational items that require further input from the builder.Type: GrantFiled: June 16, 2016Date of Patent: November 12, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Omar Zia Khan, Paul Anthony Crook, Marius Alexandru Marin, Ruhi Sarikaya
-
Publication number: 20190324553Abstract: In one embodiment, a method includes accessing a plurality of input tuples associated with a first user from a data store, wherein each input tuple comprises a gesture-input and a corresponding speech-input, determining a plurality of intents corresponding to the plurality of speech-inputs, respectively, by a natural-language understanding (NLU) module, generating a plurality of feature representations for the plurality of gesture-inputs based on one or more machine-learning models, determining a plurality of gesture identifiers for the plurality of gesture-inputs, respectively, based on their respective feature representations, associating the plurality of intents with the plurality of gesture identifiers, respectively, and training a personalized gesture-classification model for the first user based on the plurality of feature representations of their respective gesture-inputs and the associations between the plurality of intents and their respective gesture identifiers.Type: ApplicationFiled: April 18, 2019Publication date: October 24, 2019Inventors: Xiaohu Liu, Paul Anthony Crook, Francislav P. Penov, Rajen Subba
-
Patent number: 10339916Abstract: Non-limiting examples of the present disclosure describe generation and application of a universal hypothesis ranking model to rank/re-re-rank dialog hypotheses. An input is received through a user interface of an application for dialog processing. A plurality of dialog hypotheses are generated based on input understanding processing of the received input. The plurality of dialog hypotheses are ranked using a universal hypothesis ranking model that is applicable to a plurality of languages and locales. The ranking of the plurality of dialog hypotheses comprises using the universal hypothesis ranking model to analyze language independent features of the plurality of dialog hypotheses for policy determination. Other examples are also described including examples directed to generation of the universal hypothesis ranking model.Type: GrantFiled: August 31, 2015Date of Patent: July 2, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Ruhi Sarikaya, Paul Anthony Crook
-
Patent number: 10338959Abstract: Non-limiting examples of the present disclosure describe decoupling task state tracking that is managed by a shared task completion platform from execution of tasks by a task resource owner. Task registration data is received at a shared task state platform for a task that is executable by a task owner resource. Task registration data comprises parameters to be collected for execution of the task and ancillary information, such as the name of the task and whether to confirm the values of the parameters after collection. During interaction with a user, the shared task completion platform receives an input and determines the task is associated with the received input. During the interaction, parameters of the received task registration data are utilized to collect data for execution of the task. The collected data is transmitted to the task owner resource for execution of the task.Type: GrantFiled: July 13, 2015Date of Patent: July 2, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Omar Zia Khan, Paul Anthony Crook, Marius Alexandru Marin, Ruhi Sarikaya, Jean-Philippe Robichaud
-
Patent number: 10268679Abstract: A processing unit can operate an end-to-end recurrent neural network (RNN) with limited contextual dialog memory that can be jointly trained by supervised signals-user slot tagging, intent prediction and/or system action prediction. The end-to-end RNN, or joint model has shown advantages over separate models for natural language understanding (NLU) and dialog management and can capture expressive feature representations beyond conventional aggregation of slot tags and intents, to mitigate effects of noisy output from NLU. The joint model can apply a supervised signal from system actions to refine the NLU model. By back-propagating errors associated with system action prediction to the NLU model, the joint model can use machine learning to predict user intent by a binary classification obtained by both forward and backward output, and perform slot tagging, and make system action predictions based on user input, e.g., utterances across a number of domains.Type: GrantFiled: December 2, 2016Date of Patent: April 23, 2019Assignee: MICROSOFT TECHNOLOGY LICENSING, LLCInventors: Xiujun Li, Paul Anthony Crook, Li Deng, Jianfeng Gao, Yun-Nung Chen, Xuesong Yang
-
Publication number: 20190034780Abstract: Described herein is a conversation engine that can be used in a system such as a personal digital assistant or search engine that combines a dynamic knowledge graph built during execution of a request and one or more static knowledge graphs holding long term knowledge. The conversation engine comprises a state tracker that holds the dynamic knowledge graph representing the current state of the conversation, a policy engine that selects entities in the dynamic knowledge graph and executes actions provided by those entities to move the state of the conversation toward completion, and a knowledge graph search engine to search the static knowledge graph(s). The conversation is completed by building the dynamic knowledge graph over multiple rounds and chaining together operations that build toward completion of the conversation. Completion of the conversation results in completion of a request by a user.Type: ApplicationFiled: July 31, 2017Publication date: January 31, 2019Inventors: Marius Alexandru Marin, Paul Anthony Crook, Vipul Agarwal, Imed Zitouni
-
Publication number: 20180197104Abstract: Described herein is a personal digital agent system that interacts with a user in order to process various requests from the user. The personal digital agent system is associated with a dynamic knowledge graph that is tailored specifically for the user and is automatically updated when the personal digital agent interacts with the user.Type: ApplicationFiled: January 6, 2017Publication date: July 12, 2018Applicant: Microsoft Technology Licensing, LLCInventors: Marius Alexandru Marin, Paul Anthony Crook
-
Publication number: 20180157638Abstract: A processing unit can operate an end-to-end recurrent neural network (RNN) with limited contextual dialogue memory that can be jointly trained by supervised signals—user slot tagging, intent prediction and/or system action prediction. The end-to-end RNN, or joint model has shown advantages over separate models for natural language understanding (NLU) and dialogue management and can capture expressive feature representations beyond conventional aggregation of slot tags and intents, to mitigate effects of noisy output from NLU. The joint model can apply a supervised signal from system actions to refine the NLU model. By back-propagating errors associated with system action prediction to the NLU model, the joint model can use machine learning to predict user intent, and perform slot tagging, and make system action predictions based on user input, e.g., utterances across a number of domains.Type: ApplicationFiled: December 2, 2016Publication date: June 7, 2018Inventors: Xiujun Li, Paul Anthony Crook, Li Deng, Jianfeng Gao, Yun-Nung Chen, Xuesong Yang
-
Publication number: 20170364336Abstract: Systems and methods for building conversational understanding systems are provided. More specifically, the systems and methods utilize prebuilt conversational items that can form a CU system upon selection by a builder without requiring any further input from the builder. Accordingly, the systems and methods for building a conversational understanding system reduce the expertise, time, and resources necessary to build a conversational understanding system for an application when compared to systems and methods that utilize conversational items that require further input from the builder.Type: ApplicationFiled: June 16, 2016Publication date: December 21, 2017Applicant: Microsoft Technology Licensing, LLCInventors: Omar Zia Khan, Paul Anthony Crook, Marius Alexandru Marin, Ruhi Sarikaya
-
Publication number: 20170235465Abstract: Aspects herein provide third party application authors with a user interface authoring platform that automates and simplifies a task definition process while also providing the ability to leverage pre-existing language understanding models and canonicalization and resolution modules that are provided by the operating system on which the CU system resides or as provided by other third parties. In particular, the present disclosure provides a method and system for authoring a task using a user interface authoring platform.Type: ApplicationFiled: February 12, 2016Publication date: August 17, 2017Applicant: Microsoft Technology Licensing, LLCInventors: Marius Alexandru Marin, Paul Anthony Crook, Nikhil Holenarsipur Ramesh, Vipul Agarwal, Omar Zia Khan, Alexandre Rochette, Jean-Philippe Robichaud, Ruhi Sarikaya
-
Publication number: 20170061956Abstract: Non-limiting examples of the present disclosure describe generation and application of a universal hypothesis ranking model to rank/re-re-rank dialog hypotheses. An input is received through a user interface of an application for dialog processing. A plurality of dialog hypotheses are generated based on input understanding processing of the received input. The plurality of dialog hypotheses are ranked using a universal hypothesis ranking model that is applicable to a plurality of languages and locales. The ranking of the plurality of dialog hypotheses comprises using the universal hypothesis ranking model to analyze language independent features of the plurality of dialog hypotheses for policy determination. Other examples are also described including examples directed to generation of the universal hypothesis ranking model.Type: ApplicationFiled: August 31, 2015Publication date: March 2, 2017Applicant: MICROSOFT TECHNOLOGY LICENSING, LLCInventors: Ruhi Sarikaya, Paul Anthony Crook
-
Publication number: 20170017519Abstract: Non-limiting examples of the present disclosure describe decoupling task state tracking that is managed by a shared task completion platform from execution of tasks by a task resource owner. Task registration data is received at a shared task state platform for a task that is executable by a task owner resource. Task registration data comprises parameters to be collected for execution of the task and ancillary information, such as the name of the task and whether to confirm the values of the parameters after collection. During interaction with a user, the shared task completion platform receives an input and determines the task is associated with the received input. During the interaction, parameters of the received task registration data are utilized to collect data for execution of the task. The collected data is transmitted to the task owner resource for execution of the task.Type: ApplicationFiled: July 13, 2015Publication date: January 19, 2017Applicant: MICROSOFT TECHNOLOGY LICENSING, LLCInventors: Omar Zia Khan, Paul Anthony Crook, Alex Marin, Ruhi Sarikaya, Jean-Philippe Robichaud