Abstract: An example process includes: receiving a natural language input having a start time and including first and second portions respectively received from the start time to a first time and from the start time to a second time after the first time; determining an end time of the natural language input; executing, at least partially between the first time and the end time, a first task flow based on the first portion, including: obtaining a first executable object representing a first candidate action; executing, at least partially between the second time and the end time, a second task flow based on the second portion, including: obtaining a second executable object representing a second candidate action; in response to determining the end time, selecting a candidate action from a plurality of candidate actions each represented by a respective executable object; and executing the respective executable object representing the selected candidate action.

Abstract: Systems and processes for operating an intelligent automated assistant are provided. An example process includes, at an electronic device having one or more processors and memory: performing a first task specified in a first user speech input; receiving a second user speech input; and in accordance with a determination that the second user speech input includes a modification to the first task, performing a second task, wherein performance of the second task modifies at least a portion of the performance of the first task.

Abstract: Systems and processes for operating an intelligent automated assistant are provided. An example process includes, at an electronic device having one or more processors and memory: performing a first task specified in a first user speech input; receiving a second user speech input; and in accordance with a determination that the second user speech input includes a modification to the first task, performing a second task, wherein performance of the second task modifies at least a portion of the performance of the first task.

Abstract: Systems and processes for operating an intelligent automated assistant are provided. An example process includes, at an electronic device having one or more processors and memory, receiving a user input including a request for potential tasks executable by a digital assistant and a content of the electronic device; and in response to receiving the request: determining, based on the context included in the request, a textual representation of an utterance for performing a task; and providing the textual representation of the utterance for performing the task in an affordance displayed over a user interface on the screen of the electronic device.

Abstract: Systems and processes for operating an intelligent automated assistant are provided. An intelligent automated assistant receives a user input and generates a set of one or more token sequences based on the user input. The set of one or more token sequences are interpreted to generate a plurality of candidate interpretations each including a corresponding action and metadata associated with the action. A top candidate interpretation is selected from the plurality of candidate interpretations, and the corresponding action is performed based on the associated metadata.

Abstract: Systems and processes for operating an intelligent automated assistant are provided. An example method includes, at an electronic device having one or more processors and memory, receiving an utterance including a user request, determining a natural language representation of the user request, determining a first software process associated with the natural language representation, determining whether the natural language representation can be executed by a task flow of the first software process, and in accordance with a determination that the natural language representation cannot be executed by the task flow of the first software process: determining a set of transformation instructions, determining a revised natural language representation using the set of transformation instructions, and providing the revised natural language representation to a second software process.

Abstract: Systems and processes for using display representations are provided. For example, at a first time, a first plurality of objects is displayed, and a first plurality of relationships associated with the first plurality of objects is identified. A display representation is obtained, including a plurality of first object representations as displayed at the first time and a plurality of first relationship representation as identified as the first time. At a second time, an input is received from the user. In response to receiving the input from the user, the display representation is retrieved, the retrieved display representation including a plurality of second object representations corresponding to a second plurality of displayed object as displayed at the second time and a plurality of second relationship representations as identified at the second time. An output responsive to the input is provided based on the retrieved display representation.

Abstract: An example process includes: receiving a natural language input having a start time and including first and second portions respectively received from the start time to a first time and from the start time to a second time after the first time; determining an end time of the natural language input; executing, at least partially between the first time and the end time, a first task flow based on the first portion, including: obtaining a first executable object representing a first candidate action; executing, at least partially between the second time and the end time, a second task flow based on the second portion, including: obtaining a second executable object representing a second candidate action; in response to determining the end time, selecting a candidate action from a plurality of candidate actions each represented by a respective executable object; and executing the respective executable object representing the selected candidate action.

Abstract: Speech recognition device uses visual information to narrow down the range of likely adaptation parameters even before a speaker makes an utterance. Images of the speaker and/or the environment are collected using an image capturing device, and then processed to extract biometric features and environmental features. The extracted features and environmental features are then used to estimate adaptation parameters. A voice sample may also be collected to refine the adaptation parameters for more accurate speech recognition.

Abstract: Speech recognition device uses visual information to narrow down the range of likely adaptation parameters even before a speaker makes an utterance. Images of the speaker and/or the environment are collected using an image capturing device, and then processed to extract biometric features and environmental features. The extracted features and environmental features are then used to estimate adaptation parameters. A voice sample may also be collected to refine the adaptation parameters for more accurate speech recognition.