Abstract: Systems and processes for operating a digital assistant are provided. An example process for performing a task includes, at an electronic device having one or more processors and memory, receiving a spoken input including a request, receiving an image input including a plurality of objects, selecting a reference resolution module of a plurality of reference resolution modules based on the request and the image input, determining, with the selected reference resolution module, whether the request references a first object of the plurality of objects based on at least the spoken input, and in accordance with a determination that the request references the first object of the plurality of objects, determining a response to the request including information about the first object.

Abstract: Systems and processes for operating an intelligent automated assistant are provided. An example process includes receiving an utterance including a user request and determining whether at least a portion of the user request is ambiguous. If at least the portion of the user request is ambiguous then a set of context data based on the ambiguous portion of the user request is determined, metadata is extracted from the context data and a response to the user request is determined based on the extracted metadata.

Abstract: Systems and processes for operating an intelligent automated assistant to provide a set of predicted responses are provided. An example method includes, at an electronic device having one or more processors, receiving one or more messages and analyzing the unstructured natural language information of the one or more messages. The method also includes determining, based on the analysis of the unstructured natural language information, whether one or more predicted responses are to be provided. The method further includes, in accordance with a determination that one or more predicted responses are to be provided, determining, from a plurality of sets of candidate predicted responses, one set of predicted responses to be provided to the user based on context information. The method further includes providing the determined set of one or more predicted responses to the user.

Abstract: Systems and processes for operating an intelligent automated assistant to provide a set of predicted responses are provided. An example method includes, at an electronic device having one or more processors, receiving one or more messages and analyzing the unstructured natural language information of the one or more messages. The method also includes determining, based on the analysis of the unstructured natural language information, whether one or more predicted responses are to be provided. The method further includes, in accordance with a determination that one or more predicted responses are to be provided, determining, from a plurality of sets of candidate predicted responses, one set of predicted responses to be provided to the user based on context information. The method further includes providing the determined set of one or more predicted responses to the user.