Patents by Inventor Asli Celikyilmaz

Asli Celikyilmaz has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20220100972
    Abstract: Examples of the present disclosure describe systems and methods of configuring generic language understanding models. In aspects, one or more previously configured schemas for various applications may be identified and collected. A generic schema may be generated using the collected schemas. The collected schemas may be programmatically mapped to the generic schema. The generic schema may be used to train on ore more models. An interface may be provided to allow browsing the models. The interface may include a configuration mechanism that provides for selecting on or more of the models. The selected models may be bundled programmatically, such that the information and instructions needed to implement the models are configured programmatically. The bundled models may then be provided to a requestor.
    Type: Application
    Filed: December 14, 2021
    Publication date: March 31, 2022
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Ruhi Sarikaya, Asli Celikyilmaz, Young-Bum Kim, Zhaleh Feizollahi, Nikhil Ramesh, Hisami Suzuki, Alexandre Rochette
  • Patent number: 10901500
    Abstract: Improving accuracy in understanding and/or resolving references to visual elements in a visual context associated with a computerized conversational system is described. Techniques described herein leverage gaze input with gestures and/or speech input to improve spoken language understanding in computerized conversational systems. Leveraging gaze input and speech input improves spoken language understanding in conversational systems by improving the accuracy by which the system can resolve references—or interpret a user's intent—with respect to visual elements in a visual context. In at least one example, the techniques herein describe tracking gaze to generate gaze input, recognizing speech input, and extracting gaze features and lexical features from the user input. Based at least in part on the gaze features and lexical features, user utterances directed to visual elements in a visual context can be resolved.
    Type: Grant
    Filed: April 30, 2019
    Date of Patent: January 26, 2021
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Anna Prokofieva, Fethiye Asli Celikyilmaz, Dilek Z Hakkani-Tur, Larry Heck, Malcolm Slaney
  • Patent number: 10839165
    Abstract: Systems and methods for determining knowledge-guided information for a recurrent neural networks (RNN) to guide the RNN in semantic tagging of an input phrase are presented. A knowledge encoding module of a Knowledge-Guided Structural Attention Process (K-SAP) receives an input phrase and, in conjunction with additional sub-components or cooperative components generates a knowledge-guided vector that is provided with the input phrase to the RNN for linguistic semantic tagging. Generating the knowledge-guided vector comprises at least parsing the input phrase and generating a corresponding hierarchical linguistic structure comprising one or more discrete sub-structures. The sub-structures may be encoded into vectors along with attention weighting identifying those sub-structures that have greater importance in determining the semantic meaning of the input phrase.
    Type: Grant
    Filed: June 18, 2019
    Date of Patent: November 17, 2020
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Yun-Nung Vivian Chen, Dilek Z. Hakkani-Tur, Gokhan Tur, Asli Celikyilmaz, Jianfeng Gao, Li Deng
  • Publication number: 20190391640
    Abstract: Improving accuracy in understanding and/or resolving references to visual elements in a visual context associated with a computerized conversational system is described. Techniques described herein leverage gaze input with gestures and/or speech input to improve spoken language understanding in computerized conversational systems. Leveraging gaze input and speech input improves spoken language understanding in conversational systems by improving the accuracy by which the system can resolve references—or interpret a user's intent—with respect to visual elements in a visual context. In at least one example, the techniques herein describe tracking gaze to generate gaze input, recognizing speech input, and extracting gaze features and lexical features from the user input. Based at least in part on the gaze features and lexical features, user utterances directed to visual elements in a visual context can be resolved.
    Type: Application
    Filed: April 30, 2019
    Publication date: December 26, 2019
    Inventors: Anna Prokofieva, Fethiye Asli Celikyilmaz, Dilek Z Hakkani-Tur, Larry Heck, Malcom Slaney
  • Publication number: 20190303440
    Abstract: Systems and methods for determining knowledge-guided information for a recurrent neural networks (RNN) to guide the RNN in semantic tagging of an input phrase are presented. A knowledge encoding module of a Knowledge-Guided Structural Attention Process (K-SAP) receives an input phrase and, in conjunction with additional sub-components or cooperative components generates a knowledge-guided vector that is provided with the input phrase to the RNN for linguistic semantic tagging. Generating the knowledge-guided vector comprises at least parsing the input phrase and generating a corresponding hierarchical linguistic structure comprising one or more discrete sub-structures. The sub-structures may be encoded into vectors along with attention weighting identifying those sub-structures that have greater importance in determining the semantic meaning of the input phrase.
    Type: Application
    Filed: June 18, 2019
    Publication date: October 3, 2019
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Yun-Nung Vivian Chen, Dilek Z. Hakkani-Tur, Gokhan Tur, Asli Celikyilmaz, Jianfeng Gao, Li Deng
  • Publication number: 20190287012
    Abstract: An encoder-decoder neural network for sequence-to-sequence mapping tasks, such as, e.g., abstractive summarization, may employ multiple communicating encoder agents to encode multiple respective input sequences that collectively constitute the overall input. The outputs of the encoder agents may be fed into the decoder, which may use an associated attention mechanism to select which encoder agent to pay attention to at each decoding time step. Additional features and embodiments are disclosed.
    Type: Application
    Filed: March 16, 2018
    Publication date: September 19, 2019
    Inventors: Fethiye Asli Celikyilmaz, Xiaodong He
  • Patent number: 10366163
    Abstract: Systems and methods for determining knowledge-guided information for a recurrent neural networks (RNN) to guide the RNN in semantic tagging of an input phrase are presented. A knowledge encoding module of a Knowledge-Guided Structural Attention Process (K-SAP) receives an input phrase and, in conjunction with additional sub-components or cooperative components generates a knowledge-guided vector that is provided with the input phrase to the RNN for linguistic semantic tagging. Generating the knowledge-guided vector comprises at least parsing the input phrase and generating a corresponding hierarchical linguistic structure comprising one or more discrete sub-structures. The sub-structures may be encoded into vectors along with attention weighting identifying those sub-structures that have greater importance in determining the semantic meaning of the input phrase.
    Type: Grant
    Filed: September 7, 2016
    Date of Patent: July 30, 2019
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Yun-Nung Chen, Dilek Z. Hakkani-Tur, Gokhan Tur, Asli Celikyilmaz, Jianfeng Gao, Li Deng
  • Patent number: 10317992
    Abstract: Improving accuracy in understanding and/or resolving references to visual elements in a visual context associated with a computerized conversational system is described. Techniques described herein leverage gaze input with gestures and/or speech input to improve spoken language understanding in computerized conversational systems. Leveraging gaze input and speech input improves spoken language understanding in conversational systems by improving the accuracy by which the system can resolve references—or interpret a user's intent—with respect to visual elements in a visual context. In at least one example, the techniques herein describe tracking gaze to generate gaze input, recognizing speech input, and extracting gaze features and lexical features from the user input. Based at least in part on the gaze features and lexical features, user utterances directed to visual elements in a visual context can be resolved.
    Type: Grant
    Filed: September 25, 2014
    Date of Patent: June 11, 2019
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Anna Prokofieva, Fethiye Asli Celikyilmaz, Dilek Z Hakkani-Tur, Larry Heck, Malcolm Slaney
  • Publication number: 20190005385
    Abstract: Systems and methods are disclosed for inquiry-based deep learning. In one implementation, a first content segment is selected from a body of content. The content segment includes a first content element. The first content segment is compared to a second content segment to identify a content element present in the first content segment that is not present in the second content segment. Based on an identification of the content element present in the first content segment that is not present in the second content segment, the content element is stored in a session memory. A first question is generated based on the first content segment. The session memory is processed to compute an answer to the first question. An action is initiated based on the answer. Using deep learning, content segments can be encoded into memory. Incremental questioning can serve to focus various deep learning operations on certain content segments.
    Type: Application
    Filed: June 30, 2017
    Publication date: January 3, 2019
    Inventors: Fethiye Asli Celikyilmaz, Li Deng, Lihong Li, Chong Wang
  • Patent number: 10007660
    Abstract: Methods and systems are provided for contextual language understanding. A natural language expression may be received at a single-turn model and a multi-turn model for determining an intent of a user. For example, the single-turn model may determine a first prediction of at least one of a domain classification, intent classification, and slot type of the natural language expression. The multi-turn model may determine a second prediction of at least one of a domain classification, intent classification, and slot type of the natural language expression. The first prediction and the second prediction may be combined to produce a final prediction relative to the intent of the natural language expression. An action may be performed based on the final prediction of the natural language expression.
    Type: Grant
    Filed: May 26, 2017
    Date of Patent: June 26, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Ruhi Sarikaya, Puyang Xu, Alexandre Rochette, Asli Celikyilmaz
  • Patent number: 9916301
    Abstract: Click logs are automatically mined to assist in discovering candidate variations for named entities. The named entities may be obtained from one or more sources and include an initial list of named entities. A search may be performed within one or more search engines to determine common phrases that are used to identify the named entity in addition to the named entity initially included in the named entity list. Click logs associated with results of past searches are automatically mined to discover what phrases determined from the searches are candidate variations for the named entity. The candidate variations are scored to assist in determining the variations to include within an understanding model. The variations may also be used when delivering responses and displayed output in the SLU system. For example, instead of using the listed named entity, a popular and/or shortened name may be used by the system.
    Type: Grant
    Filed: December 21, 2012
    Date of Patent: March 13, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Dustin Hillard, Fethiye Asli Celikyilmaz, Dilek Hakkani-Tur, Rukmini Iyer, Gokhan Tur
  • Publication number: 20180067923
    Abstract: Systems and methods for determining knowledge-guided information for a recurrent neural networks (RNN) to guide the RNN in semantic tagging of an input phrase are presented. A knowledge encoding module of a Knowledge-Guided Structural Attention Process (K-SAP) receives an input phrase and, in conjunction with additional sub-components or cooperative components generates a knowledge-guided vector that is provided with the input phrase to the RNN for linguistic semantic tagging. Generating the knowledge-guided vector comprises at least parsing the input phrase and generating a corresponding hierarchical linguistic structure comprising one or more discrete sub-structures. The sub-structures may be encoded into vectors along with attention weighting identifying those sub-structures that have greater importance in determining the semantic meaning of the input phrase.
    Type: Application
    Filed: September 7, 2016
    Publication date: March 8, 2018
    Inventors: Yun-Nung Chen, Dilek Z. Hakkani-Tur, Gokhan Tur, Asli Celikyilmaz, Jianfeng Gao, Li Deng
  • Patent number: 9886958
    Abstract: A universal model-based approach for item disambiguation and selection is provided. An utterance may be received by a computing device in response to a list of items for selection. In aspects, the list of items may be displayed on a display screen. The universal disambiguation model may then be applied to the utterance. The universal disambiguation model may be utilized to determine whether the utterance is directed to at least one of the list of items and identify an item from the list corresponding to the utterance, based on identified language and/or domain independent referential features. The computing device may then perform an action which may include selecting the identified item associated with utterance.
    Type: Grant
    Filed: December 11, 2015
    Date of Patent: February 6, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Fethiye Asli Celikyilmaz, Zhaleh Feizollahi, Dilek Hakkani-Tur, Ruhi Sarikaya
  • Patent number: 9875237
    Abstract: An understanding model is trained to account for human perception of the perceived relative importance of different tagged items (e.g. slot/intent/domain). Instead of treating each tagged item as equally important, human perception is used to adjust the training of the understanding model by associating a perceived weight with each of the different predicted items. The relative perceptual importance of the different items may be modeled using different methods (e.g. as a simple weight vector, a model trained using features (lexical, knowledge, slot type, . . . ), and the like). The perceptual weight vector and/or or model are incorporated into the understanding model training process where items that are perceptually more important are weighted more heavily as compared to the items that are determined by human perception as less important.
    Type: Grant
    Filed: March 14, 2013
    Date of Patent: January 23, 2018
    Assignee: MICROSFOT TECHNOLOGY LICENSING, LLC
    Inventors: Ruhi Sarikaya, Anoop Deoras, Fethiye Asli Celikyilmaz, Zhaleh Feizollahi
  • Patent number: 9870356
    Abstract: Functionality is described herein for determining the intents of linguistic items (such as queries), to produce intent output information. For some linguistic items, the functionality deterministically assigns intents to the linguistic items based on known intent labels, which, in turn, may be obtained or derived from a knowledge graph or other type of knowledge resource. For other linguistic items, the functionality infers the intents of the linguistic items based on selection log data (such as click log data provided by a search system). In some instances, the intent output information may reveal new intents that are not represented by the known intent labels. In one implementation, the functionality can use the intent output information to train a language understanding model.
    Type: Grant
    Filed: February 13, 2014
    Date of Patent: January 16, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Dilek Hakkani-Tür, Fethiye Asli Celikyilmaz, Larry P. Heck, Gokhan Tur, Yangfeng Ji
  • Publication number: 20170372199
    Abstract: A processing unit can train a model as a joint multi-domain recurrent neural network (JRNN), such as a bi-directional recurrent neural network (bRNN) and/or a recurrent neural network with long-short term memory (RNN-LSTM) for spoken language understanding (SLU). The processing unit can use the trained model to, e.g., jointly model slot filling, intent determination, and domain classification. The joint multi-domain model described herein can estimate a complete semantic frame per query, and the joint multi-domain model enables multi-task deep learning leveraging the data from multiple domains. The joint multi-domain recurrent neural network (JRNN) can leverage semantic intents (such as, finding or identifying, e.g., a domain specific goal) and slots (such as, dates, times, locations, subjects, etc.) across multiple domains.
    Type: Application
    Filed: August 4, 2016
    Publication date: December 28, 2017
    Inventors: Dilek Z Hakkani-Tur, Asli Celikyilmaz, Yun-Nung Chen, Li Deng, Jianfeng Gao, Gokhan Tur, Ye-Yi Wang
  • Publication number: 20170262432
    Abstract: Methods and systems are provided for contextual language understanding. A natural language expression may be received at a single-turn model and a multi-turn model for determining an intent of a user. For example, the single-turn model may determine a first prediction of at least one of a domain classification, intent classification, and slot type of the natural language expression. The multi-turn model may determine a second prediction of at least one of a domain classification, intent classification, and slot type of the natural language expression. The first prediction and the second prediction may be combined to produce a final prediction relative to the intent of the natural language expression. An action may be performed based on the final prediction of the natural language expression.
    Type: Application
    Filed: May 26, 2017
    Publication date: September 14, 2017
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Ruhi Sarikaya, Puyang Xu, Alexandre Rochette, Asli Celikyilmaz
  • Publication number: 20170212886
    Abstract: Examples of the present disclosure describe systems and methods of configuring generic language understanding models. In aspects, one or more previously configured schemas for various applications may be identified and collected. A generic schema may be generated using the collected schemas. The collected schemas may be programmatically mapped to the generic schema. The generic schema may be used to train on ore more models. An interface may be provided to allow browsing the models. The interface may include a configuration mechanism that provides for selecting on or more of the models. The selected models may be bundled programmatically, such that the information and instructions needed to implement the models are configured programmatically. The bundled models may then be provided to a requestor.
    Type: Application
    Filed: January 22, 2016
    Publication date: July 27, 2017
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Ruhi Sarikaya, Asli Celikyilmaz, Young-Bum Kim, Zhaleh Feizollahi, Nikhil Ramesh, Hisami Suzuki, Alexandre Rochette
  • Publication number: 20170199909
    Abstract: A device may facilitate a query dialog involving queries that successively modify a query state. However, fulfilling such queries in the context of possible query domains, query intents, and contextual meanings of query terms may be difficult. Presented herein are techniques for modifying a query state in view of a query by utilizing a set of query state modifications, each representing a modification of the query state possibly intended by the user while formulating the query (e.g., adding, substituting, or removing query terms; changing the query domain or query intent; and navigating within a hierarchy of saved query states). Upon receiving a query, an embodiment may calculate the probability of the query connoting each query state modification (e.g., using a Bayesian classifier), and parsing the query according to a query state modification having a high probability (e.g., mapping respective query terms to query slots within the current query intent).
    Type: Application
    Filed: March 24, 2017
    Publication date: July 13, 2017
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Dilek Hakkani-Tur, Gokhan Tur, Larry Heck, Ashley Fidler, Fehtiye Asli Celikyilmaz
  • Patent number: 9690776
    Abstract: Methods and systems are provided for contextual language understanding. A natural language expression may be received at a single-turn model and a multi-turn model for determining an intent of a user. For example, the single-turn model may determine a first prediction of at least one of a domain classification, intent classification, and slot type of the natural language expression. The multi-turn model may determine a second prediction of at least one of a domain classification, intent classification, and slot type of the natural language expression. The first prediction and the second prediction may be combined to produce a final prediction relative to the intent of the natural language expression. An action may be performed based on the final prediction of the natural language expression.
    Type: Grant
    Filed: December 1, 2014
    Date of Patent: June 27, 2017
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Ruhi Sarikaya, Puyang Xu, Alexandre Rochette, Asli Celikyilmaz