Abstract: Systems and processes for an integrated sensor framework are provided. For example, a first electronic device receives at least one input including sensor data from a second device. A representation of a physical environment associated with the first electronic device is obtained based on sensor data from the first electronic device and the sensor data from the second device. Movement information corresponding to movement of an object within the physical environment is identified. Event information is determined corresponding to activity within the physical environment, wherein the event information is determined based on the identified movement information and the representation of the physical environment. Accordingly, an output is provided to the user based on the event information.

Abstract: The subject technology receives, from a first sensor of a device, first sensor output of a first type. The subject technology receives, from a second sensor of the device, second sensor output of a second type, the first and second sensors being non-touch sensors. The subject technology provides the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted touch-based gesture based on sensor output of the first type and sensor output of the second type. The subject technology provides a predicted touch-based gesture based on output from the machine learning model. Further, the subject technology adjusts an audio output level of the device based on the predicted gesture, and where the device is an audio output device.

Abstract: The subject technology receives, from a first sensor of a device, first sensor output of a first type. The subject technology receives, from a second sensor of the device, second sensor output of a second type, the first and second sensors being non-touch sensors. The subject technology provides the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted touch-based gesture based on sensor output of the first type and sensor output of the second type. The subject technology provides a predicted touch-based gesture based on output from the machine learning model. Further, the subject technology adjusts an audio output level of the device based on the predicted gesture, and where the device is an audio output device.

Abstract: Systems and processes for integrating a digital assistant with telephony are provided. For example, an incoming call may be received, from a caller, at an electronic device. A communication session may be established between the caller and the digital assistant of the electronic device. In accordance with a determination that the identification of the caller is unknown, determination is made whether the caller corresponds to an automated calling system. In accordance with a determination that the identification of the caller is known, a response is provided by the digital assistant to the caller. An output including information corresponding to the communication is provided at the electronic device.

Abstract: Systems and methods are disclosed for triggering an update to a machine-learning model upon detecting that a distribution of particular (e.g., recently collected) input data set is sufficiently different from a distribution training input data set used to train the model. The distributions may be determined to be sufficiently different when a classifier can identify to which distribution individual data elements belong (e.g., to at least a predetermined degree). An update to the machine-learning model can include morphing weights used by the model and/or retraining the model.

Abstract: Systems and processes for integrating a digital assistant with telephony are provided. For example, an incoming call may be received, from a caller, at an electronic device. A communication session may be established between the caller and the digital assistant of the electronic device. In accordance with a determination that the identification of the caller is unknown, determination is made whether the caller corresponds to an automated calling system. In accordance with a determination that the identification of the caller is known, a response is provided by the digital assistant to the caller. An output including information corresponding to the communication is provided at the electronic device.

Abstract: Systems and methods are disclosed for triggering an update to a machine-learning model upon detecting that a distribution of particular (e.g., recently collected) input data set is sufficiently different from a distribution training input data set used to train the model. The distributions may be determined to be sufficiently different when a classifier can identify to which distribution individual data elements belong (e.g., to at least a predetermined degree). An update to the machine-learning model can include morphing weights used by the model and/or retraining the model.

Abstract: Systems and processes for integrating a digital assistant with telephony are provided. For example, an incoming call may be received, from a caller, at an electronic device. A communication session may be established between the caller and the digital assistant of the electronic device. In accordance with a determination that the identification of the caller is unknown, determination is made whether the caller corresponds to an automated calling system. In accordance with a determination that the identification of the caller is known, a response is provided by the digital assistant to the caller. An output including information corresponding to the communication is provided at the electronic device.

Abstract: The present disclosure generally relates to assigning tasks to various user inputs, and detecting and responding to user inputs. In some embodiments, the present disclosure relates to assigning tasks to various user inputs received on a back surface of a device, and detecting and responding to user inputs on the back surface of the device.

Abstract: The subject technology receives a neural network model in a model format, the model format including information for a set of layers of the neural network model, each layer of the set of layers including a set of respective operations. The subject technology generates neural network (NN) code from the neural network model, the NN code being in a programming language distinct from the model format, and the NN code comprising a respective memory allocation for each respective layer of the set of layers of the neural network model, where the generating comprises determining the respective memory allocation for each respective layer based at least in part on a resource constraint of a target device. The subject technology compiles the NN code into a binary format. The subject technology generates a package for deploying the compiled NN code on the target device.

Abstract: Systems and methods are disclosed for triggering an update to a machine-learning model upon detecting that a distribution of particular (e.g., recently collected) input data set is sufficiently different from a distribution training input data set used to train the model. The distributions may be determined to be sufficiently different when a classifier can identify to which distribution individual data elements belong (e.g., to at least a predetermined degree). An update to the machine-learning model can include morphing weights used by the model and/or retraining the model.

Abstract: Described herein is a split virtual keyboard that is displayed on a tablet (slate) computing device. The split virtual keyboard includes a first portion and a second portion, the first portion being separated from the second portion. The first portion includes a plurality of character keys that are representative at least one respective character. The tablet computing device is configured to support text generation by way of a continuous sequence of strokes over the plurality of character keys in the first portion of the split virtual keyboard.

Abstract: The subject technology receives, from a first sensor of a device, first sensor output of a first type. The subject technology receives, from a second sensor of the device, second sensor output of a second type, the first and second sensors being non-touch sensors. The subject technology provides the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted touch-based gesture based on sensor output of the first type and sensor output of the second type. The subject technology provides a predicted touch-based gesture based on output from the machine learning model. Further, the subject technology adjusts an audio output level of the device based on the predicted gesture, and where the device is an audio output device.

Abstract: The subject technology receives a neural network model in a model format, the model format including information for a set of layers of the neural network model, each layer of the set of layers including a set of respective operations. The subject technology generates neural network (NN) code from the neural network model, the NN code being in a programming language distinct from the model format, and the NN code comprising a respective memory allocation for each respective layer of the set of layers of the neural network model, where the generating comprises determining the respective memory allocation for each respective layer based at least in part on a resource constraint of a target device. The subject technology compiles the NN code into a binary format. The subject technology generates a package for deploying the compiled NN code on the target device.

Abstract: Techniques for providing an arc layout for a soft input panel are described. In some examples, an electronic device can transition from providing a substantially straight and/or horizontal layout for the soft input panel to an arc layout for the soft input panel. The electronic device can display a graphical element at a location on the soft input panel. The electronic device can receive input corresponding to, for example, a swipe of the graphical element in a curved manner across the soft input panel. Based on the input, the electronic device can transition from displaying the soft input panel using the substantially straight layout to displaying the soft input panel using the arc layout.

Abstract: Apparatus and methods are disclosed for providing feedback and guidance to touch screen device users to improve the text entry user experience and performance. According to one embodiment, a method comprises receiving a text entry, receiving input on a touch screen in the form of a first single touch input located over a word of previously entered text, and presenting the user with one or more suggestion candidates indicated possible replacement words related to the selected word. The user can then select one of the suggestion candidates using a second single touch input to replace the selected word with a word associated with the selected suggestion candidate.

Abstract: Techniques are provided for customizing a design of a soft input panel. An electronic device can provide a keyboard customization interface to enable a user to specify a color, font, background image, and the like for the soft input panel. The user may provide input through the keyboard customization interface to customize the design (e.g., theme) of the soft input panel. The user may provide an image to be used as a background for the soft input panel. The electronic device can analyze the image to determine a most prominent color of the image. A portion of the soft input panel can be set to a color that is based on the most prominent color of the image.

Abstract: Described herein is a split virtual keyboard that is displayed on a tablet (slate) computing device. The split virtual keyboard includes a first portion and a second portion, the first portion being separated from the second portion. The first portion includes a plurality of character keys that are representative at least one respective character. The tablet computing device is configured to support text generation by way of a continuous sequence of strokes over the plurality of character keys in the first portion of the split virtual keyboard.

Abstract: Described herein is a split virtual keyboard that is displayed on a tablet (slate) computing device. The split virtual keyboard includes a first portion and a second portion, the first portion being separated from the second portion. The first portion includes a plurality of character keys that are representative at least one respective character. The tablet computing device is configured to support text generation by way of a continuous sequence of strokes over the plurality of character keys in the first portion of the split virtual keyboard.

Abstract: Systems, methods, and computer media for intent-based, dynamic generation and display of task icons within virtual keyboards are provided herein. A system can include a processor, an intent classifier, and a user interface generator. The intent classifier can be configured to determine user intent candidates based on contextual information. A user interface generator can be configured to generate the virtual keyboard for display and, upon receiving an indication of a user intent determined based on the user intent candidates, generate a task icon within the virtual keyboard. The task icon represents functionality associated with the determined user intent. Interaction with the task icon in the virtual keyboard can launch functionality associated with the determined intent. The user interface generator can also be configured to present a search interface upon receiving an indication of a user interaction with the virtual keyboard.