Abstract: A method including identifying a plurality of entities associated with a location within a physical environment, generating a representation for a subset of the plurality of entities based on a feature of the entities included in the subset, and causing a user interface, including the representation, to be rendered on a display of a wearable device.

Abstract: Systems, methods, and media for causing an action to be performed on a user device are provided. In some implementations, the systems comprise: a first user device comprising at least one hardware processor that is configured to: detect a second user device in proximity to the first user device; receive a user input indicative of an action to be performed; determine a plurality of candidate devices that are capable of performing the action, wherein the plurality of candidate devices includes the second user device; determine a plurality of device types corresponding to the plurality of candidate devices; determine a plurality of priorities associated with the plurality of candidate devices based at least in part on the plurality of device types; select a target device from the plurality of candidate devices based at least in part on the plurality of priorities; and cause the action to be performed by the target device.

Abstract: Described techniques may be utilized to receive a transcription stream including transcribed text that has been transcribed from speech, and to receive a summary request for a summary to be provided on a display of a device. Extracted text may be identified from the transcribed text and in response to the summary request. The extracted text may be processed using a summarization machine learning (ML) model to obtain a summary of the extracted text, and the summary may be displayed on the display of the device. When an image is captured, an augmented summary may be generated that includes the image together with a visual indication of one or more of an emotion, an entity, or an intent associated with the image, the summary, or the extracted text.

Abstract: Described techniques may be utilized to process transcribed text of a transcription stream using a summarization ML model to obtain a summary stream. In this way, a live conversation or other live speech may be provided to a user in real-time, using a display of a head-mounted device (HMD) or other suitable device. A user interface may be provided with a toggle or other stream selector for selecting either or both of the transcription stream or the summary stream at a given point in time. Actionable items may be identified within either or both of the transcription stream or the summary stream, and a scroll bar or other action selector may be provided in the user interface to execute corresponding actions.

Abstract: A method performed by a computing system comprises generating text from audio data and determining an end portion of the text to include in a summarization of the text based on a length of a portion of the audio data from which the text was generated and which ends with a proposed end portion and a time value associated with the proposed end portion, the proposed end portion including a word from the text.

Abstract: Systems, methods, and media for causing an action to be performed on a user device are provided. In some implementations, the systems comprise: a first user device comprising at least one hardware processor that is configured to: detect a second user device in proximity to the first user device; receive a user input indicative of an action to be performed; determine a plurality of candidate devices that are capable of performing the action, wherein the plurality of candidate devices includes the second user device; determine a plurality of device types corresponding to the plurality of candidate devices; determine a plurality of priorities associated with the plurality of candidate devices based at least in part on the plurality of device types; select a target device from the plurality of candidate devices based at least in part on the plurality of priorities; and cause the action to be performed by the target device.

Abstract: Systems, methods, and media for causing an action to be performed on a user device are provided. In some implementations, the systems comprise: a first user device comprising at least one hardware processor that is configured to: detect a second user device in proximity to the first user device; receive a user input indicative of an action to be performed; determine a plurality of candidate devices that are capable of performing the action, wherein the plurality of candidate devices includes the second user device; determine a plurality of device types corresponding to the plurality of candidate devices; determine a plurality of priorities associated with the plurality of candidate devices based at least in part on the plurality of device types; select a target device from the plurality of candidate devices based at least in part on the plurality of priorities; and cause the action to be performed by the target device.

Abstract: A system for producing a live performance combining separate performances of remote players comprises a plurality of connected computing devices transmitting a plurality of communication streams to one another in series. The system can employ the plurality of series conceded computing devices, each for receiving and playing back an immediate predecessor communication stream from an immediate predecessor computing device, and each for capturing and transmitting a current performance along with the played back immediate predecessor communication stream. Typically, the individual performances of remote players can comprise live instrument music and/or vocals which can also include live video.

Abstract: Systems, methods, and media for causing an action to be performed on a user device are provided. In some implementations, the systems comprise: a first user device comprising at least one hardware processor that is configured to: detect a second user device in proximity to the first user device; receive a user input indicative of an action to be performed; determine a plurality of candidate devices that are capable of performing the action, wherein the plurality of candidate devices includes the second user device; determine a plurality of device types corresponding to the plurality of candidate devices; determine a plurality of priorities associated with the plurality of candidate devices based at least in part on the plurality of device types; select a target device from the plurality of candidate devices based at least in part on the plurality of priorities; and cause the action to be performed by the target device.

Abstract: Systems, methods, and media for causing an action to be performed on a user device are provided. In some implementations, the systems comprise: a first user device comprising at least one hardware processor that is configured to: detect a second user device in proximity to the first user device; receive a user input indicative of an action to be performed; determine a plurality of candidate devices that are capable of performing the action, wherein the plurality of candidate devices includes the second user device; determine a plurality of device types corresponding to the plurality of candidate devices; determine a plurality of priorities associated with the plurality of candidate devices based at least in part on the plurality of device types; select a target device from the plurality of candidate devices based at least in part on the plurality of priorities; and cause the action to be performed by the target device.

Abstract: Systems, methods and apparatus for using a magnetic controller to control a device. In one aspect, a system includes a magnetic controller external to a device, the magnetic controller including: a magnetic device for altering a surrounding magnetic field of a device; one or more input actuators, each operatively coupled to the magnetic device and that when actuated cause the magnetic device to alter the surrounding magnetic field according to a predefined change associated with the input actuator; and a model executable by the device and that models as device inputs the differences in the surrounding magnetic field of the device caused by the actuation of the one or more input actuators.

Abstract: A system for producing a live performance combining separate performances of remote players comprises a plurality of connected computing devices transmitting a plurality of communication streams to one another in series. The system can employ the plurality of series conceded computing devices, each for receiving and playing back an immediate predecessor communication stream from an immediate predecessor computing device, and each for capturing and transmitting a current performance along with the played back immediate predecessor communication stream. Typically, the individual performances of remote players can comprise live instrument music and/or vocals which can also include live video.

Abstract: Systems, methods, and media for causing an action to be performed on a user device are provided. In some implementations, the systems comprise: a first user device comprising at least one hardware processor that is configured to: detect a second user device in proximity to the first user device; receive a user input indicative of an action to be performed; determine a plurality of candidate devices that are capable of performing the action, wherein the plurality of candidate devices includes the second user device; determine a plurality of device types corresponding to the plurality of candidate devices; determine a plurality of priorities associated with the plurality of candidate devices based at least in part on the plurality of device types; select a target device from the plurality of candidate devices based at least in part on the plurality of priorities; and cause the action to be performed by the target device.

Abstract: Systems, methods, and media for causing an action to be performed on a user device are provided. In some implementations, the systems comprise: a first user device comprising at least one hardware processor that is configured to: detect a second user device in proximity to the first user device; receive a user input indicative of an action to be performed; determine a plurality of candidate devices that are capable of performing the action, wherein the plurality of candidate devices includes the second user device; determine a plurality of device types corresponding to the plurality of candidate devices; determine a plurality of priorities associated with the plurality of candidate devices based at least in part on the plurality of device types; select a target device from the plurality of candidate devices based at least in part on the plurality of priorities; and cause the action to be performed by the target device.

Abstract: Among other things, this document describes a computer-implemented method. The method can include receiving, at a first device, an indication of user input to cause the first device to establish a wireless data connection with another device. A determination can be made at the first device that one or more sensors on the first device are oriented toward a second device. In response to at least one of (i) receiving the indication of user input to cause the first device to establish a wireless data connection with another device and (ii) determining that the one or more sensors on the first device are oriented toward the second device, a first wireless data connection can be established between the first device and the second device. A first stream of audio data can be received and played at the first device.

Abstract: Among other things, this document describes a computer-implemented method. The method can include receiving, at a first device, an indication of user input to cause the first device to establish a wireless data connection with another device. A determination can be made at the first device that one or more sensors on the first device are oriented toward a second device. In response to at least one of (i) receiving the indication of user input to cause the first device to establish a wireless data connection with another device and (ii) determining that the one or more sensors on the first device are oriented toward the second device, a first wireless data connection can be established between the first device and the second device. A first stream of audio data can be received and played at the first device.

Abstract: Systems, methods and apparatus for using a magnetic controller to control device. In one aspect, a system includes a magnetic controller external to a device, the magnetic controller including: a magnetic device for altering a surrounding magnetic field of a device; one or more input actuators, each operatively coupled to the magnetic device and that when actuated cause the magnetic device alter the surrounding magnetic field according to a predefined change associated with the input actuator; and a model executable by the device and that models as device inputs the differences in the surrounding magnetic field of the device caused by the actuation of the one or more input actuators.

Abstract: A computer-implemented method and associated computing device for translating speech can include receiving, at a microphone of a computing device, an audio signal representing speech of a user in a first language or in a second language at a first time. A positional relationship between the user and the computing device at the first time can be determined and utilized to determine whether the speech is in the first language or the second language. The method can further include obtaining, at the computing device, a machine translation of the speech represented by the audio signal based on the determined language, wherein the machine translation is: (i) in the second language when the determined language is the first language, or (ii) in the first language when the determined language is the second language. An audio representation of the machine translation can be output from a speaker of the computing device.

Abstract: Systems, methods and apparatus for using a magnetic controller to control device. In one aspect, a system includes a magnetic controller external to a device, the magnetic controller including: a magnetic device for altering a surrounding magnetic field of a device; one or more input actuators, each operatively coupled to the magnetic device and that when actuated cause the magnetic device alter the surrounding magnetic field according to a predefined change associated with the input actuator; and a model executable by the device and that models as device inputs the differences in the surrounding magnetic field of the device caused by the actuation of the one or more input actuators.

Abstract: Among other things, this document describes a computer-implemented method. The method can include receiving, at a first device, an indication of user input to cause the first device to establish a wireless data connection with another device. A determination can be made at the first device that one or more sensors on the first device are oriented toward a second device. In response to at least one of (i) receiving the indication of user input to cause the first device to establish a wireless data connection with another device and (ii) determining that the one or more sensors on the first device are oriented toward the second device, a first wireless data connection can be established between the first device and the second device. A first stream of audio data can be received and played at the first device.