Patents by Inventor Michael A. Laurenzano

Michael A. Laurenzano has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20200257856
    Abstract: Systems and methods for synthesizing training data for multi-intent utterance segmentation include identifying a first corpus of utterances comprising a plurality of distinct single-intent in-domain utterances; identifying a second corpus of utterances comprising a plurality of distinct single-intent out-of-domain utterances; identifying a third corpus comprising a plurality of distinct conjunction terms; forming a multi-intent training corpus comprising synthetic multi-intent utterances, wherein forming each distinct multi-intent utterance includes: selecting a first distinct in-domain utterance from the first corpus of utterances; probabilistically selecting one of a first out-of-domain utterance from the second corpus and a second in-domain utterance from the first corpus; probabilistically selecting or not selecting a distinct conjunction term from the third corpus; and forming a synthetic multi-intent utterance including appending the first in-domain utterance with one of the first out-of-domain utteranc
    Type: Application
    Filed: February 6, 2020
    Publication date: August 13, 2020
    Inventors: Joseph Peper, Parker Hill, Kevin Leach, Sean Stapleton, Jonathan K. Kummerfeld, Johann Hauswald, Michael A. Laurenzano, Lingjia Tang, Jason Mars
  • Publication number: 20200257857
    Abstract: Systems and methods for synthesizing training data for multi-intent utterance segmentation include identifying a first corpus of utterances comprising a plurality of distinct single-intent in-domain utterances; identifying a second corpus of utterances comprising a plurality of distinct single-intent out-of-domain utterances; identifying a third corpus comprising a plurality of distinct conjunction terms; forming a multi-intent training corpus comprising synthetic multi-intent utterances, wherein forming each distinct multi-intent utterance includes: selecting a first distinct in-domain utterance from the first corpus of utterances; probabilistically selecting one of a first out-of-domain utterance from the second corpus and a second in-domain utterance from the first corpus; probabilistically selecting or not selecting a distinct conjunction term from the third corpus; and forming a synthetic multi-intent utterance including appending the first in-domain utterance with one of the first out-of-domain utteranc
    Type: Application
    Filed: April 21, 2020
    Publication date: August 13, 2020
    Inventors: Joseph Peper, Parker Hill, Kevin Leach, Sean Stapleton, Jonathan K. Kummerfeld, Johann Hauswald, Michael Laurenzano, Lingjia Tang, Jason Mars
  • Publication number: 20200258007
    Abstract: A system and method for improving a machine learning-based dialogue system includes: sourcing a corpus of raw machine learning training data from sources of training data based on a plurality of seed training samples, wherein the corpus of raw machine learning training data comprises a plurality of distinct instances of training data; generating a vector representation for each distinct instance of training data; identifying statistical characteristics of the corpus of raw machine learning training data based on a mapping of the vector representation for each distinct instance of training data; identifying anomalous instances of the plurality of distinct instances of training data of the corpus of raw machine learning training data based on the identified statistical characteristics of the corpus; and curating the corpus of raw machine learning training data based on each of the instances of training data identified as anomalous instances.
    Type: Application
    Filed: April 30, 2020
    Publication date: August 13, 2020
    Inventors: Stefan Larson, Anish Mahendran, Andrew Lee, Jonathan K. Kummerfeld, Parker Hill, Michael A. Laurenzano, Johann Hauswald, Lingjia Tang, Jason Mars
  • Patent number: 10740371
    Abstract: A system and method for intelligently configuring a machine learning-based dialogue system includes a conversational deficiency assessment of a target dialog system, wherein implementing the conversational deficiency assessment includes: (i) identifying distinct corpora of mishandled utterances based on an assessment of the distinct corpora of dialogue data; (ii) identifying candidate corpus of mishandled utterances from the distinct corpora of mishandled utterances as suitable candidates for building new dialogue competencies for the target dialogue system if candidate metrics of the candidate corpus of mishandled utterances satisfy a candidate threshold; building the new dialogue competencies for the target dialogue system for each of the candidate corpus of mishandled utterances having candidate metrics that satisfy the candidate threshold; and configuring a dialogue system control structure for the target dialogue system based on the new dialogue competencies, wherein the dialogue system control structure
    Type: Grant
    Filed: October 30, 2019
    Date of Patent: August 11, 2020
    Assignee: Clinc, Inc.
    Inventors: Jason Mars, Lingjia Tang, Michael A. Laurenzano, Johann Hauswald, Parker Hill, Yiping Kang, Yunqi Zhang
  • Publication number: 20200250382
    Abstract: A system and method for intelligently configuring a machine learning-based dialogue system includes a conversational deficiency assessment of a target dialog system, wherein implementing the conversational deficiency assessment includes: (i) identifying distinct corpora of mishandled utterances based on an assessment of the distinct corpora of dialogue data; (ii) identifying candidate corpus of mishandled utterances from the distinct corpora of mishandled utterances as suitable candidates for building new dialogue competencies for the target dialogue system if candidate metrics of the candidate corpus of mishandled utterances satisfy a candidate threshold; building the new dialogue competencies for the target dialogue system for each of the candidate corpus of mishandled utterances having candidate metrics that satisfy the candidate threshold; and configuring a dialogue system control structure for the target dialogue system based on the new dialogue competencies, wherein the dialogue system control structure
    Type: Application
    Filed: March 10, 2020
    Publication date: August 6, 2020
    Inventors: Jason Mars, Lingjia Tang, Michael A. Laurenzano, Johann Hauswald, Parker Hill, Yiping Kang, Yunqi Zhang
  • Publication number: 20200193331
    Abstract: A system and method for improving a machine learning-based dialogue system includes: sourcing a corpus of raw machine learning training data from sources of training data based on a plurality of seed training samples, wherein the corpus of raw machine learning training data comprises a plurality of distinct instances of training data; generating a vector representation for each distinct instance of training data; identifying statistical characteristics of the corpus of raw machine learning training data based on a mapping of the vector representation for each distinct instance of training data; identifying anomalous instances of the plurality of distinct instances of training data of the corpus of raw machine learning training data based on the identified statistical characteristics of the corpus; and curating the corpus of raw machine learning training data based on each of the instances of training data identified as anomalous instances.
    Type: Application
    Filed: November 20, 2019
    Publication date: June 18, 2020
    Inventors: Stefan Larson, Anish Mahendran, Andrew Lee, Jonathan K. Kummerfeld, Parker Hill, Michael A. Laurenzano, Johann Hauswald, Lingjia Tang, Jason Mars
  • Publication number: 20200193265
    Abstract: A system and method of configuring a graphical control structure for controlling a machine learning-based automated dialogue system includes configuring a root dialogue classification node that performs a dialogue intent classification task for utterance data input; configuring a plurality of distinct dialogue state classification nodes that are arranged downstream of the root dialogue classification node; configuring a graphical edge connection between the root dialogue classification node and the plurality of distinct state dialogue classification nodes that graphically connects each of the plurality of distinct state dialogue classification nodes to the root dialogue classification node, wherein (i) the root dialogue classification node, (ii) the plurality of distinct classification nodes, (iii) and the transition edge connections define a graphical dialogue system control structure that governs an active dialogue between a user and the machine learning-based automated dialogue system.
    Type: Application
    Filed: November 13, 2019
    Publication date: June 18, 2020
    Inventors: Parker Hill, Jason Mars, Lingjia Tang, Michael A. Laurenzano, Johann Hauswald, Yiping Kang, Yunqi Zhang
  • Patent number: 10679100
    Abstract: Systems and methods of intelligent formation and acquisition of machine learning training data for implementing an artificially intelligent dialogue system includes constructing a corpora of machine learning test corpus that comprise a plurality of historical queries and commands sampled from production logs of a deployed dialogue system; configuring training data sourcing parameters to source a corpora of raw machine learning training data from remote sources of machine learning training data; calculating efficacy metrics of the corpora of raw machine learning training data, wherein calculating the efficacy metrics includes calculating one or more of a coverage metric value and a diversity metric value of the corpora of raw machine learning training data; using the corpora of raw machine learning training data to train the at least one machine learning classifier if the calculated coverage metric value of the corpora of machine learning training data satisfies a minimum coverage metric threshold.
    Type: Grant
    Filed: April 10, 2019
    Date of Patent: June 9, 2020
    Assignee: Clinc, Inc.
    Inventors: Yiping Kang, Yunqi Zhang, Jonathan K. Kummerfeld, Parker Hill, Johann Hauswald, Michael A. Laurenzano, Lingjia Tang, Jason Mars
  • Patent number: 10679150
    Abstract: A system and method for improving a machine learning-based dialogue system includes: sourcing a corpus of raw machine learning training data from sources of training data based on a plurality of seed training samples, wherein the corpus of raw machine learning training data comprises a plurality of distinct instances of training data; generating a vector representation for each distinct instance of training data; identifying statistical characteristics of the corpus of raw machine learning training data based on a mapping of the vector representation for each distinct instance of training data; identifying anomalous instances of the plurality of distinct instances of training data of the corpus of raw machine learning training data based on the identified statistical characteristics of the corpus; and curating the corpus of raw machine learning training data based on each of the instances of training data identified as anomalous instances.
    Type: Grant
    Filed: November 20, 2019
    Date of Patent: June 9, 2020
    Assignee: Clinc, Inc.
    Inventors: Stefan Larson, Anish Mahendran, Andrew Lee, Jonathan K. Kummerfeld, Parker Hill, Michael A. Laurenzano, Johann Hauswald, Lingjia Tang, Jason Mars
  • Publication number: 20200151566
    Abstract: Systems and methods for implementing an artificially intelligent virtual assistant includes collecting a user query; using a competency classification machine learning model to generate a competency label for the user query; using a slot identification machine learning model to segment the text of the query and label each of the slots of the query; generating a slot value for each of the slots of the query; generating a handler for each of the slot values; and using the slot values to: identify an external data source relevant to the user query, fetch user data from the external data source, and apply one or more operations to the query to generate response data; and using the response data, to generate a response to the user query.
    Type: Application
    Filed: January 15, 2020
    Publication date: May 14, 2020
    Inventors: Jason Mars, Lingjia Tang, Michael Laurenzano, Johann Hauswald, Parker Hill
  • Patent number: 10572801
    Abstract: Systems and methods for implementing an artificially intelligent virtual assistant includes collecting a user query; using a competency classification machine learning model to generate a competency label for the user query; using a slot identification machine learning model to segment the text of the query and label each of the slots of the query; generating a slot value for each of the slots of the query; generating a handler for each of the slot values; and using the slot values to: identify an external data source relevant to the user query, fetch user data from the external data source, and apply one or more operations to the query to generate response data; and using the response data, to generate a response to the user query.
    Type: Grant
    Filed: November 22, 2017
    Date of Patent: February 25, 2020
    Assignee: Clinc, Inc.
    Inventors: Jason Mars, Lingjia Tang, Michael Laurenzano, Johann Hauswald, Parker Hill
  • Publication number: 20190294925
    Abstract: Systems and methods of intelligent formation and acquisition of machine learning training data for implementing an artificially intelligent dialogue system includes constructing a corpora of machine learning test corpus that comprise a plurality of historical queries and commands sampled from production logs of a deployed dialogue system; configuring training data sourcing parameters to source a corpora of raw machine learning training data from remote sources of machine learning training data; calculating efficacy metrics of the corpora of raw machine learning training data, wherein calculating the efficacy metrics includes calculating one or more of a coverage metric value and a diversity metric value of the corpora of raw machine learning training data; using the corpora of raw machine learning training data to train the at least one machine learning classifier if the calculated coverage metric value of the corpora of machine learning training data satisfies a minimum coverage metric threshold.
    Type: Application
    Filed: April 10, 2019
    Publication date: September 26, 2019
    Inventors: Yiping Kang, Yunqi Zhang, Jonathan K. Kummerfeld, Parker Hill, Johann Hauswald, Michael A. Laurenzano, Lingjia Tang, Jason Mars
  • Publication number: 20190272479
    Abstract: Systems and methods for intelligently training a machine learning model includes: configuring a machine learning (ML) training data request for a pre-existing machine learning classification model; transmitting the machine learning training data request to each of a plurality of external training data sources, wherein each of the plurality of external training data sources is different; collecting and storing the machine learning training data from each of the plurality of external training data sources; processing the collected machine learning training data using a predefined training data processing algorithm; and in response to processing the collected machine learning training data, deploying a subset of the collected machine learning training data into a live machine learning model.
    Type: Application
    Filed: April 4, 2019
    Publication date: September 5, 2019
    Inventors: Jason Mars, Lingjia Tang, Michael Laurenzano, Johann Hauswald
  • Patent number: 10303978
    Abstract: Systems and methods of intelligent formation and acquisition of machine learning training data for implementing an artificially intelligent dialogue system includes constructing a corpora of machine learning test corpus that comprise a plurality of historical queries and commands sampled from production logs of a deployed dialogue system; configuring training data sourcing parameters to source a corpora of raw machine learning training data from remote sources of machine learning training data; calculating efficacy metrics of the corpora of raw machine learning training data, wherein calculating the efficacy metrics includes calculating one or more of a coverage metric value and a diversity metric value of the corpora of raw machine learning training data; using the corpora of raw machine learning training data to train the at least one machine learning classifier if the calculated coverage metric value of the corpora of machine learning training data satisfies a minimum coverage metric threshold.
    Type: Grant
    Filed: September 27, 2018
    Date of Patent: May 28, 2019
    Assignee: Clinc, Inc.
    Inventors: Yiping Kang, Yunqi Zhang, Jonathan K. Kummerfeld, Parker Hill, Johann Hauswald, Michael A. Laurenzano, Lingjia Tang, Jason Mars
  • Publication number: 20190156198
    Abstract: Systems and methods for implementing an artificially intelligent virtual assistant includes collecting a user query; using a competency classification machine learning model to generate a competency label for the user query; using a slot identification machine learning model to segment the text of the query and label each of the slots of the query; generating a slot value for each of the slots of the query; generating a handler for each of the slot values; and using the slot values to: identify an external data source relevant to the user query, fetch user data from the external data source, and apply one or more operations to the query to generate response data; and using the response data, to generate a response to the user query.
    Type: Application
    Filed: November 22, 2017
    Publication date: May 23, 2019
    Inventors: Jason Mars, Lingjia Tang, Michael Laurenzano, Johann Hauswald, Parker Hill
  • Patent number: 10296848
    Abstract: Systems and methods for intelligently training a machine learning model includes: configuring a machine learning (ML) training data request for a pre-existing machine learning classification model; transmitting the machine learning training data request to each of a plurality of external training data sources, wherein each of the plurality of external training data sources is different; collecting and storing the machine learning training data from each of the plurality of external training data sources; processing the collected machine learning training data using a predefined training data processing algorithm; and in response to processing the collected machine learning training data, deploying a subset of the collected machine learning training data into a live machine learning model.
    Type: Grant
    Filed: March 5, 2018
    Date of Patent: May 21, 2019
    Assignee: Clinc, Inc.
    Inventors: Jason Mars, Lingjia Tang, Michael Laurenzano, Johann Hauswald
  • Publication number: 20190130244
    Abstract: Systems and methods for implementing an artificially intelligent virtual assistant includes collecting a user query; using a competency classification machine learning model to generate a competency label for the user query; using a slot identification machine learning model to segment the text of the query and label each of the slots of the query; generating a slot value for each of the slots of the query; generating a handler for each of the slot values; and using the slot values to: identify an external data source relevant to the user query, fetch user data from the external data source, and apply one or more operations to the query to generate response data; and using the response data, to generate a response to the user query.
    Type: Application
    Filed: October 30, 2017
    Publication date: May 2, 2019
    Inventors: Jason Mars, Lingjia Tang, Michael Laurenzano, Johann Hauswald
  • Patent number: 10223141
    Abstract: A system is provided for monitoring, regenerating and replacing the code of running applications with semantically equivalent, specialized code versions that reflect the demands of the execution environment. The system includes a co-designed compiler and runtime system that virtualizes a selected set of edges in a host program, where these edges provide hooks through which the runtime system may redirect execution into an intermediate representation utilized to optimize introspective and extrospective processes.
    Type: Grant
    Filed: February 9, 2017
    Date of Patent: March 5, 2019
    Assignee: THE REGENTS OF THE UNIVERSITY OF MICHIGAN
    Inventors: Jason Mars, Michael Laurenzano, Lingjia Tang
  • Patent number: 9921859
    Abstract: A system is provided for monitoring, regenerating and replacing the code of running applications with semantically equivalent, specialized code versions that reflect the demands of the execution environment. The system includes a co-designed compiler and runtime system that virtualizes a selected set of edges in a host program, where these edges provide hooks through which the runtime system may redirect execution into an intermediate representation utilized to optimize introspective and extrospective processes.
    Type: Grant
    Filed: December 11, 2015
    Date of Patent: March 20, 2018
    Assignee: THE REGENTS OF THE UNIVERSITY OF MICHIGAN
    Inventors: Jason Mars, Michael Laurenzano, Lingjia Tang
  • Publication number: 20170249172
    Abstract: A system is provided for monitoring, regenerating and replacing the code of running applications with semantically equivalent, specialized code versions that reflect the demands of the execution environment. The system includes a co-designed compiler and runtime system that virtualizes a selected set of edges in a host program, where these edges provide hooks through which the runtime system may redirect execution into an intermediate representation utilized to optimize introspective and extrospective processes.
    Type: Application
    Filed: February 9, 2017
    Publication date: August 31, 2017
    Inventors: Jason Mars, Michael Laurenzano, Lingjia Tang