Abstract: A device implementing a system for machine-learning based gesture recognition includes at least one processor configured to, receive, sensor data for a first window of time and additional sensor data for a second window of time overlapping the first window of time. The sensor data and the additional sensor data are provided as inputs to a machine learning model, the machine learning model having been trained to output a predicted gesture, predicted gesture start time, and predicted gesture end time based on the sensor data. A predicted gesture is determined based on an output from the machine learning model, and to perform, in response to determining the predicted gesture, a predetermined action on the device.

Abstract: The present disclosure generally relates to navigating user interfaces using hand gestures.

Abstract: While a first view of a three-dimensional environment is visible, a computer system detects a first input meeting selection criteria. If, when the first input was detected, a user was directing attention to a first portion of the first view that has a spatial relationship to a viewport through which the three-dimensional environment is visible, the computer system displays a user interface object including affordances for accessing functions of the computer system; otherwise, the computer system forgoes displaying the user interface object. While a different view of the three-dimensional environment is visible, the computer system detects a second input meeting the selection criteria. If, when the second input was detected, the user was directing attention to a second portion of the different view that has the same spatial relationship to the viewport, the computer system displays the user interface object; otherwise, the computer system forgoes displaying the user interface object.

Abstract: Some embodiments described in this disclosure are directed to a first electronic device that operates in a remote interaction mode with a second electronic device, where user interactions with images displayed on the first electronic device cause the second electronic device to update display of the images and/or corresponding user interfaces on the second electronic device.

Abstract: The present disclosure generally relates to techniques and interfaces for generating synthesized speech outputs. For example, a user interface for a text-to-speech service can include ranked and/or categorized phrases, which can be selected to enter as text. A synthesized speech output is then generated to deliver any entered text, for example, using a personalized voice model.

Abstract: The present disclosure generally relates to detecting text. The present disclosure describes at least methods for managing a text detection mode, identifying targeted text, and managing modes of a computer system.

Abstract: The present disclosure generally relates to communicating between computer systems, and more specifically to techniques for communicating user interface content.

Abstract: A device implementing a system for machine-learning based gesture recognition includes at least one processor configured to, receive, from a first sensor of the device, first sensor output of a first type, and receive, from a second sensor of the device, second sensor output of a second type that differs from the first type. The at least one processor is further configured to provide the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted gesture based on sensor output of the first type and sensor output of the second type. The at least one processor is further configured to determine the predicted gesture based on an output from the machine learning model, and to perform, in response to determining the predicted gesture, a predetermined action on the device.

Abstract: The present disclosure generally relates to exploring a geographic region that is displayed in computer user interfaces. In some embodiments, a method includes at an electronic device with a display and one or more input devices, displaying a map of a geographic region on the display and detecting a first user input to select a starting location on the map. After detecting the first user input, the method includes detecting a second user input to select a first direction of navigation from the starting location. In response to detecting the second user input, the method includes determining a path on the map that traverses in the first direction of navigation and connects the starting location to an ending location, and providing audio that includes traversal information about traversing along the path in the geographic region in the first direction of navigation and from the starting location to the ending location.

Abstract: An electronic device with a display and a touch-sensitive surface displays, on the display, a first visual indicator. The electronic device receives a first single touch input on the touch-sensitive surface at a location that corresponds to the first visual indicator; and, in response to detecting the first single touch input on the touch-sensitive surface at a location that corresponds to the first visual indicator, replaces display of the first visual indicator with display of a first menu. The first menu includes a virtual touches selection icon. In response to detecting selection of the virtual touches selection icon, the electronic device displays a menu of virtual multitouch contacts.

Abstract: An electronic device with a camera obtains, with the camera, one or more images of a scene. The electronic device detects a respective feature within the scene. In accordance with a determination that a first mode is active on the device, the electronic device provides a first audible description of the scene. The first audible description provides information indicating a size and/or position of the respective feature relative to a first set of divisions applied to the one or more images of the scene. In accordance with a determination that the first mode is not active on the device, the electronic device provides a second audible description of the plurality of objects. The second audible description is distinct from the first audible description and does not include the information indicating the size and/or position of the respective feature relative to the first set of divisions.

Abstract: An electronic device with a display and a touch-sensitive surface displays, on the display, a first visual indicator. The electronic device receives a first single touch input on the touch-sensitive surface at a location that corresponds to the first visual indicator; and, in response to detecting the first single touch input on the touch-sensitive surface at a location that corresponds to the first visual indicator, replaces display of the first visual indicator with display of a first menu. The first menu includes a virtual touches selection icon. In response to detecting selection of the virtual touches selection icon, the electronic device displays a menu of virtual multitouch contacts.

Abstract: An electronic device with a display and a touch-sensitive surface displays, on the display, a first visual indicator that corresponds to a virtual touch. The device receives a first input from an adaptive input device. In response to receiving the first input from the adaptive input device, the device displays a first menu on the display. The first menu includes a virtual touches selection icon. In response to detecting selection of the virtual touches selection icon, a menu of virtual multitouch contacts is displayed.

Abstract: While a view of a three-dimensional environment is visible via a display generation component of a computer system, the computer system receives, from a user, one or more first user inputs corresponding to selection of a respective direction in the three-dimensional environment relative to a reference point associated with the user, and displays a ray extending from the reference point in the respective direction. While displaying the ray, the system displays a selection cursor moving along the ray independently of user input. When the selection cursor is at a respective position along the ray, the system receives one or more second user inputs corresponding to a request to stop the movement of the selection cursor along the ray and, in response, sets a target location for a next user interaction to a location in the three-dimensional environment that corresponds to the respective position of the selection cursor along the ray.

Abstract: Some embodiments described in this disclosure are directed to a first electronic device that operates in a remote interaction mode with a second electronic device, where user interactions with images displayed on the first electronic device cause the second electronic device to update display of the images and/or corresponding user interfaces on the second electronic device.

Abstract: The present disclosure generally relates to activating and managing dual user interface modes for an operating system. An electronic device configured to run an operating system in a first mode including a first user interface receives, while operating in the first user interface mode, a predefined sequence of inputs, and in response to receiving the predefined sequence of inputs, transitions the operating system from the first user interface mode to a second user interface mode. The second user interface mode has different input capabilities with respect to the first user interface mode.

Abstract: In some embodiments, an computer system detects objects, such as physical objects in the physical environment of the electronic device. In some embodiments, the computer system presents indications of characteristics of the physical objects. In some embodiments, the physical objects are entry points to physical locations.

Abstract: The present disclosure generally relates to displaying user interfaces with device controls.

Abstract: In some embodiments, a computer system presents navigation directions from a first location to a second location. In some embodiments, if the current location of the computer system is different from the first location, the computer system presents audio and/or tactile feedback indicating distance and/or direction to the first location.

Abstract: A device implementing a system for machine-learning based gesture recognition includes at least one processor configured to, receive, from a first sensor of the device, first sensor output of a first type, and receive, from a second sensor of the device, second sensor output of a second type that differs from the first type. The at least one processor is further configured to provide the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted gesture based on sensor output of the first type and sensor output of the second type. The at least one processor is further configured to determine the predicted gesture based on an output from the machine learning model, and to perform, in response to determining the predicted gesture, a predetermined action on the device.