Abstract: A head-mounted device (HMD) can be configured to determine a request for recognizing at least one content item included within content framed within a display of the HMD. The HMD can be configured to initiate a head-tracking process that maintains a coordinate system with respect to the content, and a pointer-tracking process that tracks a pointer that is visible together with the content within the display. The HMD can be configured to capture a first image of the content and a second image of the content, the second image including the pointer. The HMD can be configured to map a location of the pointer within the second image to a corresponding image location within the first image, using the coordinate system, and provide the at least one content item from the corresponding image location.

Abstract: A method of vertical planar surface detection includes identifying, at an electronic device, a plurality of feature points of an environment proximate the electronic device based on imagery captured at the electronic device and based on received non-visual pose information obtained from non-visual sensors. The electronic device queries the plurality of feature points to identify one or more planar surfaces in the imagery captured at the electronic device. Further, the electronic device computes a plurality of oriented point vectors associated with the one or more planar surfaces. A vertical planar surface is detected based at least in part on one or more of the plurality of oriented point vectors being aligned in a direction perpendicular to gravity within a predetermined threshold.

Abstract: A wearable computing device includes a frame, a camera mounted on the frame so as to capture images of an environment outside of the wearable computing device, a display device mounted on the frame so as to display the images captured by the camera, and at least one eye gaze tracking device mounted on the frame so as to track a gaze directed at the images displayed by the display device. In response to the detection of a fixation of the gaze on the display of images, the system may identify a pixel area corresponding to a fixation point of the fixation gaze on the display of images. The system may identify an object in the ambient environment corresponding to the identified pixel area, and set the identified object as a selected object for user interaction.

Abstract: A wearable computing device includes a frame, a camera mounted on the frame so as to capture images of an environment outside of the wearable computing device, a display device mounted on the frame so as to display the images captured by the camera, and at least one eye gaze tracking device mounted on the frame so as to track a gaze directed at the images displayed by the display device. In response to the detection of a fixation of the gaze on the display of images, the system may identify a pixel area corresponding to a fixation point of the fixation gaze on the display of images. The system may identify an object in the ambient environment corresponding to the identified pixel area, and set the identified object as a selected object for user interaction.

Abstract: A head-mounted device (HMD) can be configured to determine a request for recognizing at least one content item included within content framed within a display of the HMD. The HMD can be configured to initiate a head-tracking process that maintains a coordinate system with respect to the content, and a pointer-tracking process that tracks a pointer that is visible together with the content within the display. The HMD can be configured to capture a first image of the content and a second image of the content, the second image including the pointer. The HMD can be configured to map a location of the pointer within the second image to a corresponding image location within the first image, using the coordinate system, and provide the at least one content item from the corresponding image location.

Abstract: A wearable computing device includes a frame, a camera mounted on the frame so as to capture images of an environment outside of the wearable computing device, a display device mounted on the frame so as to display the images captured by the camera, and at least one eye gaze tracking device mounted on the frame so as to track a gaze directed at the images displayed by the display device. In response to the detection of a fixation of the gaze on the display of images, the system may identify a pixel area corresponding to a fixation point of the fixation gaze on the display of images. The system may identify an object in the ambient environment corresponding to the identified pixel area, and set the identified object as a selected object for user interaction.

Abstract: In a general aspect, a head-mounted device (HMD) can be configured to receive a selection of a mode of operation of a content reader of the HMD from a plurality of modes of operation, and initiate, in response to the selection, a content capture process to capture content within a field of view (FOV) of a camera of the HMD. The HMD can be further configured to identify, in the captured content, a plurality of content items, receive a navigation command to select a content item from the plurality of content items, and provide, in response to the navigation command, readback of the selected content item.

Abstract: A method of vertical planar surface detection includes identifying, at an electronic device (100), a plurality of feature points (124, 126, 128) of an environment (112) proximate the electronic device based on imagery captured at the electronic device and based on received non-visual pose information (142) obtained from non-visual sensors (610). The electronic device queries (406) the plurality of feature points to identify one or more planar surfaces in the imagery captured at the electronic device. Further, the electronic device computes (408) a plurality of oriented point vectors associated with the one or more planar surfaces. A vertical planar surface (204, 206) is detected based at least in part on one or more of the plurality of oriented point vectors being aligned in a direction perpendicular to gravity within a predetermined threshold.

Abstract: According to an aspect, a method for creating a three-dimensional map for augmented reality (AR) localization includes obtaining a digital representation of a scene of an AR environment, where the digital representation has been captured by a computing device. The method includes identifying, using a machine learning (ML) model, a region of the digital representation having visual data identified as likely to change, and removing a portion of the digital representation that corresponds to the region of the digital representation to obtain a reduced digital representation, where the reduced digital representation is used to generate a three-dimensional (3D) map for the AR environment.

Abstract: According to an aspect, a method for managing augmented reality (AR) content in an AR environment includes obtaining image data associated with a scene of an AR environment, where the AR environment includes AR content positioned in a coordinate space of the AR environment. The method includes detecting a physical object from the image data, associating the physical object with the AR content, and storing coordinates of the AR content and information indicating that the physical object is associated with the AR content in an AR scene storage for future AR localization.

Abstract: According to an aspect, a method for managing augmented reality (AR) content in an AR environment includes obtaining image data associated with a scene of an AR environment, where the AR environment includes AR content positioned in a coordinate space of the AR environment. The method includes detecting a physical object from the image data, associating the physical object with the AR content, and storing coordinates of the AR content and information indicating that the physical object is associated with the AR content in an AR scene storage for future AR localization.

Abstract: According to an aspect, a method for creating a three-dimensional map for augmented reality (AR) localization includes obtaining a digital representation of a scene of an AR environment, where the digital representation has been captured by a computing device. The method includes identifying, using a machine learning (ML) model, a region of the digital representation having visual data identified as likely to change, and removing a portion of the digital representation that corresponds to the region of the digital representation to obtain a reduced digital representation, where the reduced digital representation is used to generate a three-dimensional (3D) map for the AR environment.

Abstract: A method includes receiving, from a device displaying a portion of a first frame of a streaming 360 degree video, an indication that a seek operation is triggered, the portion of the first frame being based on a view point of a user of the device, selecting a second frame of the streaming 360 degree video based on the seek operation, encoding a first portion of the second frame and a second portion of the second frame, the first portion of the second frame corresponding to the portion of the first frame, and communicating the encoded portion of the second frame to the device.

Abstract: Systems and methods for playing back content based on external content are provided. An example system and method may identify content based on an environment in which a content-playback device is located. The example system and method may also detect a playback-initiation action performed by a user of the content-playback device and identify a content-playback position based on the playback-initiation action. The example system and method may also trigger playback of the content on the content-playback device based on the identified content-playback position.

Abstract: Example implementations may relate to creation and control of channels. In particular, a computing device may receive a first channel-addition request indicating content from a first audio-provider service and may responsively send to a server an instruction to establish a first channel that provides access to content from the first audio-provider service via an application-program account. With this arrangement, a subsequent second channel-addition request may then similarly lead to establishment of a second channel that provides access to content from the second audio-provider service via the application-program account. After channel-additions, the device may determine a first selection of the added first channel and may responsively cause content from the first audio-provider service to be output by an audio output device.

Abstract: A method includes receiving, from a device displaying a portion of a first frame of a streaming 360 degree video, an indication that a seek operation is triggered, the portion of the first frame being based on a view point of a user of the device, selecting a second frame of the streaming 360 degree video based on the seek operation, encoding a first portion of the second frame and a second portion of the second frame, the first portion of the second frame corresponding to the portion of the first frame, and communicating the encoded portion of the second frame to the device.

Abstract: This document describes techniques and devices for an interactive cord with resistance touchpoints. An interactive cord includes an audio wire, and a fabric cover that covers the audio wire. The fabric cover includes at least one conductive thread integrated into the fabric cover to form at least one touchpoint at which the conductive thread makes contact with the audio wire when the touchpoint is pressed. The contact of the conductive thread with the audio wire creates a resistance value that is mapped to a control. The resistance value is based on a resistance of the conductive thread and a length of the conductive thread from a base of the interactive cord to the touchpoint. The resistance value is detectable by a controller implemented at a computing device that is coupled to the interactive cord. In response to detection of the resistance value, the controller triggers the control.

Abstract: This document describes techniques and devices for an interactive cord with resistance touchpoints. An interactive cord includes an audio wire, and a fabric cover that covers the audio wire. The fabric cover includes at least one conductive thread integrated into the fabric cover to form at least one touchpoint at which the conductive thread makes contact with the audio wire when the touchpoint is pressed. The contact of the conductive thread with the audio wire creates a resistance value that is mapped to a control. The resistance value is based on a resistance of the conductive thread and a length of the conductive thread from a base of the interactive cord to the touchpoint. The resistance value is detectable by a controller implemented at a computing device that is coupled to the interactive cord. In response to detection of the resistance value, the controller triggers the control.

Abstract: Systems and methods for playing back content based on external content are provided. An example system and method may identify content based on an environment in which a content-playback device is located. The example system and method may also detect a playback-initiation action performed by a user of the content-playback device and identify a content-playback position based on the playback-initiation action. The example system and method may also trigger playback of the content on the content-playback device based on the identified content-playback position.

Abstract: Example implementations may relate to creation and control of channels. In particular, a computing device may receive a first channel-addition request indicating content from a first audio-provider service and may responsively send to a server an instruction to establish a first channel that provides access to content from the first audio-provider service via an application-program account. With this arrangement, a subsequent second channel-addition request may then similarly lead to establishment of a second channel that provides access to content from the second audio-provider service via the application-program account. After channel-additions, the device may determine a first selection of the added first channel and may responsively cause content from the first audio-provider service to be output by an audio output device.