Abstract: A method includes receiving, by at least one processing device of an electronic device, an utterance provided by a user. The method also includes delexicalizing at least a portion of the utterance using a named entity database stored on the electronic device to create an encoded utterance. The method further includes transmitting the encoded utterance to a server on which a language model is stored. The method also includes receiving an intent and one or more slots associated with the utterance, where at least one slot of the one or more slots is a representative tag. The method further includes identifying a named entity corresponding to the at least one slot based on the named entity database. In addition, the method includes performing an action based on the intent and the one or more slots.

Abstract: In an embodiment of the present disclosure, a method of simultaneously identifying intent and slots in a voice assistant command includes tokenizing, into a plurality of tokens, a current utterance of a user of a device comprising the voice assistant command, prepending the plurality of tokens with a previous utterance and a separation token, obtaining, using a transformer-based machine learning model, one or more predictions for the voice assistant command from the prepended plurality of tokens, aligning, according to one or more constraints, the at least one of the flag prediction, the goal prediction, and the sub-goal prediction, providing, to the device, the identified intent and the identified slots based on the intent prediction and the aligned at least one of the flag prediction, the goal prediction, and the sub-goal prediction, causing the device to perform the voice assistant command according to the identified intent and the identified slots.

Abstract: An electronic system provides visual information to help a development person improve a machine learning model. The visual information includes logic flows, scatter plots, confusion matrices and instances of utterances which are related to poor performance. Guided by the logic flows, the development person is able trace scatter plots confusion matrices in order to identify weaknesses of the machine learning model being developed. A user interface help the development person then specify or indicate a request for additional training data, improved labels or an improved labelling guide. The logic flows, scatter plots and confusion matrices are used iteratively by the development person to repeatedly discover how the machine learning model can be improved and then getting the data to improve the machine learning model. A relatively unskilled person is able to develop a highly effective machine learning model using the logic flows, scatter plots and confusion matrices.

Abstract: A method includes generating, using at least one processor of an electronic device, a plurality of expert labels for a sample using a plurality of machine learned classifiers. The method also includes determining, using the at least one processor, an expert consensus label among the plurality of expert labels. The method further includes comparing, using the at least one processor, the expert consensus label to a ground truth label associated with the sample in response to determining that a consensus is found among the plurality of machine learned classifiers. The method also includes identifying, using the at least one processor, the ground truth label as a clean label in response to determining that the expert consensus label and the ground truth label match. In addition, the method includes identifying, using the at least one processor, the ground truth label for reassessment in response to determining that the expert consensus label and the ground truth label do not match.

Abstract: A method for predicting a task completion of a voice assistant from online user logs may include obtaining a voice assistant log regarding a user voice input of a user of an electronic device requesting a voice assistant of the electronic device to perform a task; extracting a set of features from the voice assistant log; and identifying a task completion estimation metric that is indicative of a performance of the voice assistant in performing the task, based on the set of features and a trained artificial intelligence (AI) model.