Abstract: Systems and processes for operating an intelligent automated assistant are provided. In accordance with one example, a method includes, at an electronic device with one or more processors and memory, obtaining a first set of data corresponding to one or more interactions between a user and the digital assistant on the electronic device; obtaining a second set of data corresponding to one or more interactions between the user and an application on the electronic device; and storing the first set of data and the second set of data. The method further includes receiving a set of executable instructions; processing, based on the received set of executable instructions, the first set of data and the second set of data to obtain a third set of data; and transmitting the third set of data to a remote electronic device.

Abstract: Speech recognition is performed on a received utterance to determine a plurality of candidate text representations of the utterance, including a primary text representation and one or more alternative text representations. Natural language processing is performed on the primary text representation to determine a plurality of candidate actionable intents, including a primary actionable intent and one or more alternative actionable intents. A result is determined based on the primary actionable intent. The result is provided to the user. A recognition correction trigger is detected. In response to detecting the recognition correction trigger, a set of alternative intent affordances and a set of alternative text affordances are concurrently displayed.

Abstract: Speech recognition is performed on a received utterance to determine a plurality of candidate text representations of the utterance, including a primary text representation and one or more alternative text representations. Natural language processing is performed on the primary text representation to determine a plurality of candidate actionable intents, including a primary actionable intent and one or more alternative actionable intents. A result is determined based on the primary actionable intent. The result is provided to the user. A recognition correction trigger is detected. In response to detecting the recognition correction trigger, a set of alternative intent affordances and a set of alternative text affordances are concurrently displayed.

Abstract: Speech recognition is performed on a received utterance to determine a plurality of candidate text representations of the utterance, including a primary text representation and one or more alternative text representations. Natural language processing is performed on the primary text representation to determine a plurality of candidate actionable intents, including a primary actionable intent and one or more alternative actionable intents. A result is determined based on the primary actionable intent. The result is provided to the user. A recognition correction trigger is detected. In response to detecting the recognition correction trigger, a set of alternative intent affordances and a set of alternative text affordances are concurrently displayed.

Abstract: Systems and processes for operating an intelligent automated assistant are provided. In accordance with one example, a method includes, at an electronic device with one or more processors and memory, obtaining a first set of data corresponding to one or more interactions between a user and the digital assistant on the electronic device; obtaining a second set of data corresponding to one or more interactions between the user and an application on the electronic device; and storing the first set of data and the second set of data. The method further includes receiving a set of executable instructions; processing, based on the received set of executable instructions, the first set of data and the second set of data to obtain a third set of data; and transmitting the third set of data to a remote electronic device.

Abstract: Systems and processes for improved machine-learned systems are provided. In accordance with one example, a method includes receiving a first speech recognition result and a first accuracy score corresponding to the first speech recognition result; receiving, from another electronic device, a second speech recognition result and a second accuracy score corresponding to the second recognition result; determining whether the second accuracy score is greater than the first accuracy score; in accordance with a determination that the second accuracy score is greater than the first accuracy score, providing a speech recognition system of the electronic device based on the second speech recognition result; and in accordance with a determination that the second accuracy score is not greater than the first accuracy score, forgoing providing a speech recognition system of the electronic device based on the second speech recognition result.

Abstract: Speech recognition is performed on a received utterance to determine a plurality of candidate text representations of the utterance, including a primary text representation and one or more alternative text representations. Natural language processing is performed on the primary text representation to determine a plurality of candidate actionable intents, including a primary actionable intent and one or more alternative actionable intents. A result is determined based on the primary actionable intent. The result is provided to the user. A recognition correction trigger is detected. In response to detecting the recognition correction trigger, a set of alternative intent affordances and a set of alternative text affordances are concurrently displayed.

Abstract: Systems and processes for evaluating embedded personalized systems are provided. In one example process, instructions that define an experiment associated with a personalized speech recognition system can be received. The instructions can define one or more experimental parameters. In accordance with the received instructions, a second personalized speech recognition system can be generated based on the personalized speech recognition system and the one or more experimental parameters. Additionally, the plurality of user speech samples can be processed using the second personalized speech recognition system to generate a plurality of speech recognition results and a plurality of accuracy scores corresponding to the plurality of speech recognition results. Second instructions can be received based on the plurality of accuracy scores. In accordance with the second instructions, the second speech recognition system can be activated.

Abstract: Systems and processes for evaluating embedded personalized systems are provided. In one example process, instructions that define an experiment associated with a personalized speech recognition system can be received. The instructions can define one or more experimental parameters. In accordance with the received instructions, a second personalized speech recognition system can be generated based on the personalized speech recognition system and the one or more experimental parameters. Additionally, the plurality of user speech samples can be processed using the second personalized speech recognition system to generate a plurality of speech recognition results and a plurality of accuracy scores corresponding to the plurality of speech recognition results. Second instructions can be received based on the plurality of accuracy scores. In accordance with the second instructions, the second speech recognition system can be activated.