Abstract: A method includes obtaining a speech signal. The method also includes predicting a first likelihood of a wake word or phrase being spoken in the speech signal using a first machine learning model trained to receive the speech signal as input. The method further includes, responsive to the first likelihood exceeding a first threshold, performing automatic speech recognition on the speech signal to determine a textual representation of the speech signal. The method also includes predicting a second likelihood of the wake word or phrase being spoken in the speech signal using a second machine learning model trained to receive at least one of the textual representation, audio features associated with the speech signal, and context features associated with the electronic device. In addition, the method includes, responsive to the second likelihood exceeding a second threshold, generating instructions to perform an action requested in the speech signal.

Abstract: A method includes receiving, by at least one processing device of an electronic device, an utterance provided by a user. The method also includes delexicalizing at least a portion of the utterance using a named entity database stored on the electronic device to create an encoded utterance. The method further includes transmitting the encoded utterance to a server on which a language model is stored. The method also includes receiving an intent and one or more slots associated with the utterance, where at least one slot of the one or more slots is a representative tag. The method further includes identifying a named entity corresponding to the at least one slot based on the named entity database. In addition, the method includes performing an action based on the intent and the one or more slots.

Abstract: In an embodiment of the present disclosure, a method of simultaneously identifying intent and slots in a voice assistant command includes tokenizing, into a plurality of tokens, a current utterance of a user of a device comprising the voice assistant command, prepending the plurality of tokens with a previous utterance and a separation token, obtaining, using a transformer-based machine learning model, one or more predictions for the voice assistant command from the prepended plurality of tokens, aligning, according to one or more constraints, the at least one of the flag prediction, the goal prediction, and the sub-goal prediction, providing, to the device, the identified intent and the identified slots based on the intent prediction and the aligned at least one of the flag prediction, the goal prediction, and the sub-goal prediction, causing the device to perform the voice assistant command according to the identified intent and the identified slots.

Abstract: An electronic system provides visual information to help a development person improve a machine learning model. The visual information includes logic flows, scatter plots, confusion matrices and instances of utterances which are related to poor performance. Guided by the logic flows, the development person is able trace scatter plots confusion matrices in order to identify weaknesses of the machine learning model being developed. A user interface help the development person then specify or indicate a request for additional training data, improved labels or an improved labelling guide. The logic flows, scatter plots and confusion matrices are used iteratively by the development person to repeatedly discover how the machine learning model can be improved and then getting the data to improve the machine learning model. A relatively unskilled person is able to develop a highly effective machine learning model using the logic flows, scatter plots and confusion matrices.

Abstract: A method includes generating, using at least one processor of an electronic device, a plurality of expert labels for a sample using a plurality of machine learned classifiers. The method also includes determining, using the at least one processor, an expert consensus label among the plurality of expert labels. The method further includes comparing, using the at least one processor, the expert consensus label to a ground truth label associated with the sample in response to determining that a consensus is found among the plurality of machine learned classifiers. The method also includes identifying, using the at least one processor, the ground truth label as a clean label in response to determining that the expert consensus label and the ground truth label match. In addition, the method includes identifying, using the at least one processor, the ground truth label for reassessment in response to determining that the expert consensus label and the ground truth label do not match.

Abstract: A method for predicting a task completion of a voice assistant from online user logs may include obtaining a voice assistant log regarding a user voice input of a user of an electronic device requesting a voice assistant of the electronic device to perform a task; extracting a set of features from the voice assistant log; and identifying a task completion estimation metric that is indicative of a performance of the voice assistant in performing the task, based on the set of features and a trained artificial intelligence (AI) model.

Abstract: Systems, methods, and computer-readable storage media are provided for conflating entities using a persistent entity index. Information (including attributes) pertaining to a plurality of entities is received. The received information is either matched with one or more existing entities in the persistent entity index or, if no match is found, selected for addition to the persistent entity index. The persistent entity index includes entity-attribute pairs associated therewith. Attributes associated with matching entities for which information is received are aggregated and/or reconciled with the entity-attribute pairs associated with existing entities included in the persistent entity index. The persistent entity index may be incrementally updated at predetermined time intervals to insure the accuracy and freshness of the information associated therewith.

Abstract: A system and method for processing a document to generate a set of related documents. A system is provided that includes a textual analytics system that analyzes unstructured data contained in a source document and extracts a set of structured information about the source document; and a compare system that identifies a set of related documents by comparing the set of structured information with metadata indexed from a set of publications.

Abstract: The subject matter disclosed herein relates to generating an abstract of an electronic document based at least in part on a utility metric associated with a group phrases as well as on a ranked order of sentences.

Abstract: Systems, methods, and computer-readable storage media are provided for conflating entities using a persistent entity index. Information (including attributes) pertaining to a plurality of entities is received. The received information is either matched with one or more existing entities in the persistent entity index or, if no match is found, selected for addition to the persistent entity index. The persistent entity index includes entity-attribute pairs associated therewith. Attributes associated with matching entities for which information is received are aggregated and/or reconciled with the entity-attribute pairs associated with existing entities included in the persistent entity index. The persistent entity index may be incrementally updated at predetermined time intervals to insure the accuracy and freshness of the information associated therewith.

Abstract: In an effort to enhance computer user engagement with a search results page, systems and methods are presented which are configured to identify an entity as being the subject matter of a user's search query. If the entity is a known entity, i.e., entity information is stored in an entity store for the identified entity, a subset of entity attributes are identified and a representative entity attribute question is obtained for each of the attributes in the subset of entity attributes. The representative entity attribute questions are identified according to the probability that they are formed linguistically correct. The representative entity attribute questions are included in a search results page that is generated in response to the user's search query.

Abstract: This patent application pertains to answer model comparison. One implementation can determine a first frequency at which an individual answer category appears in an individual slot on a query results page when utilizing a first model. The method can ascertain a second frequency at which the individual answer category appears in the individual slot on the query results page when utilizing a second model. The method can calibrate the second model so that the second frequency approaches the first frequency.

Abstract: A vectorization process is employed in which chemical identifier strings are converted into respective vectors. These vectors may then be searched to identify molecules that are identical or similar to each other. The dimensions of the vector space can be defined by sequences of symbols that make up the chemical identifier strings. The International Chemical Identifier (InChI) string defined by the International Union of Pure and Applied Chemistry (IUPAC) is particularly well suited for these methods.

Abstract: Methods and computer-storage media having computer-executable instructions embodied thereon that facilitate generating a machine-learned model for ranking search results using click-based data are provided. Data is referenced from user queries, which may include search results generated by general search engines and vertical search engines. A training set is generated from the search results and click-based judgments are associated with the search results in the training set. Based on click-based judgments, identifiable features are determined from the search results in a training set. Based on determining identifiable features in a training set, a rule set is generated for ranking subsequent search results.

Abstract: This patent application pertains to answer model comparison. One implementation can determine a first frequency at which an individual answer category appears in an individual slot on a query results page when utilizing a first model. The method can ascertain a second frequency at which the individual answer category appears in the individual slot on the query results page when utilizing a second model. The method can calibrate the second model so that the second frequency approaches the first frequency.

Abstract: A vectorization process is employed in which chemical identifier strings are converted into respective vectors. These vectors may then be searched to identify molecules that are identical or similar to each other. The dimensions of the vector space can be defined by sequences of symbols that make up the chemical identifier strings. The International Chemical Identifier (InChI) string defined by the International Union of Pure and Applied Chemistry (IUPAC) is particularly well suited for these methods.

Abstract: A vectorization process is employed in which chemical identifier strings are converted into respective vectors. These vectors may then be searched to identify molecules that are identical or similar to each other. The dimensions of the vector space can be defined by sequences of symbols that make up the chemical identifier strings. The International Chemical Identifier (InChI) string defined by the International Union of Pure and Applied Chemistry (IUPAC) is particularly well suited for these methods.

Abstract: Methods and computer-storage media having computer-executable instructions embodied thereon that facilitate generating a machine-learned model for ranking search results using click-based data are provided. Data is referenced from user queries, which may include search results generated by general search engines and vertical search engines. A training set is generated from the search results and click-based judgments are associated with the search results in the training set. Based on click-based judgments, identifiable features are determined from the search results in a training set. Based on determining identifiable features in a training set, a rule set is generated for ranking subsequent search results.

Abstract: Disclosed are apparatus and methods for ranking lines of text. In one embodiment, an intent of a query is ascertained. A relevance of each one of a plurality of lines of text of a document is determined based upon the intent of the query, content of the query, and content of each of the plurality of lines of text. The plurality of lines of text may then be ranked according to the determined relevance of each of the plurality of lines of text.

Abstract: Determine a plurality of first dwell durations for a plurality of first web pages, each first dwell duration indicating a time period a user has spent with a first web page. Access a plurality of first quality ratings for the first web pages, each first quality rating indicating a quality of a first web page as a part of a search result generated for a first search query. Access a predefined quality rating threshold. Correlate the first dwell durations and the first quality ratings. And, determine a dwell duration threshold, such that a second user spending a second dwell duration greater than or equal to the dwell duration threshold with a second web page indicates that the second user is satisfied with the second web page identified in a second search result generated by a search engine in response to a second search query requested by the second user.