Abstract: Methods and apparatus for capturing motion from a self-tracking device are disclosed. In embodiments, a device self-tracks motion of the device relative to a first reference frame while recording motion of a subject relative to a second reference frame, the second reference frame being a reference frame relative to the device. In the embodiments, the subject may be a real object or, alternately, the subject may be a virtual subject and a motion of the virtual object may be recorded relative to the second reference frame by associating a position offset relative to the device with the position of the virtual object in the recorded motion. The motion of the subject relative to the first reference frame may be determined from the tracked motion of the device relative to the first frame and the recorded motion of the subject relative to the second reference frame.

Abstract: A holographic interaction device is described. In one or more implementations, an input device includes an input portion comprising a plurality of controls that are configured to generate signals to be processed as inputs by a computing device that is communicatively coupled to the controls. The input device also includes a holographic recording mechanism disposed over a surface of the input portion, the holographic recording mechanism is configured to output a hologram in response to receipt of light, from a light source, that is viewable by a user over the input portion.

Abstract: Techniques for implementing an integrative interactive space are described. In implementations, video cameras that are positioned to capture video at different locations are synchronized such that aspects of the different locations can be used to generate an integrated interactive space. The integrated interactive space can enable users at the different locations to interact, such as via video interaction, audio interaction, and so on. In at least some embodiments, techniques can be implemented to adjust an image of a participant during a video session such that the participant appears to maintain eye contact with other video session participants at other locations. Techniques can also be implemented to provide a virtual shared space that can enable users to interact with the space, and can also enable users to interact with one another and/or objects that are displayed in the virtual shared space.

Abstract: Example embodiments simultaneously acquire multiple different focus state images for a scene at a video rate. The focus state images are acquired from a static arrangement of static optical elements. The focus state images are suitable for and sufficient for determining the depth of an object in the scene using depth from defocus (DFD) processing. The depth of the object in the scene is determined from the focus state images using DFD processing. The static optical elements may be off-the-shelf components that are used without modification. The static elements may include image sensors aligned to a common optical path, a beam splitter in the common optical path, and telecentric lenses that correct light in multiple optical paths produced by the beam splitter. The multiple optical paths may differ by a defocus delta. Simultaneous acquisition of the multiple focus state images facilitates mitigating motion blur associated with conventional DFD processing.

Abstract: Remote depth sensing techniques are described via relayed depth from diffusion. In one or more implementations, a remote depth sensing system is configured to sense depth as relayed from diffusion. The system includes an image capture system including an image sensor and an imaging lens configured to transmit light to the image sensor through an intermediate image plane that is disposed between the imaging lens and the image sensor, the intermediate plane having an optical diffuser disposed proximal thereto that is configured to diffuse the transmitted light. The system also includes a depth sensing module configured to receive one or more images from the image sensor and determine a distance to one or more objects in an object scene captured by the one or more images using a depth by diffusion technique that is based at least in part on an amount of blurring exhibited by respective said objects in the one or more images.

Abstract: Global and local light detection techniques in optical sensor systems are described. In one or more implementations, a global lighting value is generated that describes a global lighting level for a plurality of optical sensors based on a plurality of inputs received from the plurality of optical sensors. An illumination map is generated that describes local lighting conditions of respective ones of the plurality of optical sensors based on the plurality of inputs received from the plurality of optical sensors. Object detection is performed using an image captured using the plurality of optical sensors along with the global lighting value and the illumination map.

Abstract: Examples relating to using non-visual feedback to alert a viewer of a display that a visual change has been triggered are disclosed. One disclosed example provides a method comprising using gaze tracking data from a gaze tracking system to determine that a viewer changes a gaze location. Based on determining that the viewer changes the gaze location, a visual change is triggered and non-visual feedback indicating the triggering of the visual change is provided to the viewer. If a cancel change input is received within a predetermined timeframe, the visual change is not displayed. If a cancel change input is not received within the timeframe, the visual change is displayed via the display.

Abstract: Object detection techniques for use in conjunction with optical sensors is described. In one or more implementations, a plurality of inputs are received, each of the inputs being received from a respective one of a plurality of optical sensors. Each of the plurality of inputs are classified using machine learning as to whether the inputs are indicative of detection of an object by a respective said optical sensor.

Abstract: Embodiments that relate to determining gaze locations are disclosed. In one embodiment a method includes shining light along an outbound light path to the eyes of the user wearing glasses. Upon detecting the glasses, the light is dynamically polarized in a polarization pattern that switches between a random polarization phase and a single polarization phase, wherein the random polarization phase includes a first polarization along an outbound light path and a second polarization orthogonal to the first polarization along a reflected light path. The single polarization phase has a single polarization. During the random polarization phases, glares reflected from the glasses are filtered out and pupil images are captured. Glint images are captured during the single polarization phase. Based on pupil characteristics and glint characteristics, gaze locations are repeatedly detected.

Abstract: Photos are shared among devices that are in close proximity to one another and for which there is a connection among the devices. The photos can be shared automatically, or alternatively based on various user inputs. Various different controls can also be placed on sharing photos to restrict the other devices with which photos can be shared, the manner in which photos can be shared, and/or how the photos are shared.

Abstract: Examples relating to using non-visual feedback to alert a viewer of a display that a visual change has been triggered are disclosed. One disclosed example provides a method comprising using gaze tracking data from a gaze tracking system to determine that a viewer changes a gaze location. Based on determining that the viewer changes the gaze location, a visual change is triggered and non-visual feedback indicating the triggering of the visual change is provided to the viewer. If a cancel change input is received within a predetermined timeframe, the visual change is not displayed. If a cancel change input is not received within the timeframe, the visual change is displayed via the display.

Abstract: A combination of three computational components may provide memory and computational efficiency while producing results with little latency, e.g., output can begin with the second frame of video being processed. Memory usage may be reduced by maintaining key frames of video and pose information for each frame of video. Additionally, only one global volumetric structure may be maintained for the frames of video being processed. To be computationally efficient, only depth information may be computed from each frame. Through fusion of multiple depth maps from different frames into a single volumetric structure, errors may average out over several frames, leading to a final output with high quality.

Abstract: Techniques for implementing an integrative interactive space are described. In implementations, video cameras that are positioned to capture video at different locations are synchronized such that aspects of the different locations can be used to generate an integrated interactive space. The integrated interactive space can enable users at the different locations to interact, such as via video interaction, audio interaction, and so on. In at least some embodiments, techniques can be implemented to adjust an image of a participant during a video session such that the participant appears to maintain eye contact with other video session participants at other locations. Techniques can also be implemented to provide a virtual shared space that can enable users to interact with the space, and can also enable users to interact with one another and/or objects that are displayed in the virtual shared space.

Abstract: Embodiments that relate to determining gaze locations are disclosed. In one embodiment a method includes shining light along an outbound light path to the eyes of the user wearing glasses. Upon detecting the glasses, the light is dynamically polarized in a polarization pattern that switches between a random polarization phase and a single polarization phase, wherein the random polarization phase includes a first polarization along an outbound light path and a second polarization orthogonal to the first polarization along a reflected light path. The single polarization phase has a single polarization. During the random polarization phases, glares reflected from the glasses are filtered out and pupil images are captured. Glint images are captured during the single polarization phase. Based on pupil characteristics and glint characteristics, gaze locations are repeatedly detected.

Abstract: Global and local light detection techniques in optical sensor systems are described. In one or more implementations, a global lighting value is generated that describes a global lighting level for a plurality of optical sensors based on a plurality of inputs received from the plurality of optical sensors. An illumination map is generated that describes local lighting conditions of respective ones of the plurality of optical sensors based on the plurality of inputs received from the plurality of optical sensors. Object detection is performed using an image captured using the plurality of optical sensors along with the global lighting value and the illumination map.

Abstract: A 3D silhouette sensing system is described which comprises a stereo camera and a light source. In an embodiment, a 3D sensing module triggers the capture of pairs of images by the stereo camera at the same time that the light source illuminates the scene. A series of pairs of images may be captured at a predefined frame rate. Each pair of images is then analyzed to track both a retroreflector in the scene, which can be moved relative to the stereo camera, and an object which is between the retroreflector and the stereo camera and therefore partially occludes the retroreflector. In processing the image pairs, silhouettes are extracted for each of the retroreflector and the object and these are used to generate a 3D contour for each of the retroreflector and object.

Abstract: Techniques for implementing an integrative interactive space are described. In implementations, video cameras that are positioned to capture video at different locations are synchronized such that aspects of the different locations can be used to generate an integrated interactive space. The integrated interactive space can enable users at the different locations to interact, such as via video interaction, audio interaction, and so on. In at least some embodiments, techniques can be implemented to adjust an image of a participant during a video session such that the participant appears to maintain eye contact with other video session participants at other locations. Techniques can also be implemented to provide a virtual shared space that can enable users to interact with the space, and can also enable users to interact with one another and/or objects that are displayed in the virtual shared space.

Abstract: Object detection techniques for use in conjunction with optical sensors is described. In one or more implementations, a plurality of inputs are received, each of the inputs being received from a respective one of a plurality of optical sensors. Each of the plurality of inputs are classified using machine learning as to whether the inputs are indicative of detection of an object by a respective said optical sensor.

Abstract: A holographic interaction device is described. In one or more implementations, an input device includes an input portion comprising a plurality of controls that are configured to generate signals to be processed as inputs by a computing device that is communicatively coupled to the controls. The input device also includes a holographic recording mechanism disposed over a surface of the input portion, the holographic recording mechanism is configured to output a hologram in response to receipt of light, from a light source, that is viewable by a user over the input portion.

Abstract: A system and method are disclosed for estimating camera motion of a visual input scene using points and lines detected in the visual input scene. The system includes a camera server comprising a stereo pair of calibrated cameras, a feature processing module, a trifocal motion estimation module and an optional adjustment module. The stereo pair of the calibrated cameras and its corresponding stereo pair of camera after camera motion form a first and a second trifocal tensor. The feature processing module is configured to detect points and lines in the visual input data comprising a plurality of image frames. The feature processing module is further configured to find point correspondence between detected points and line correspondence between detected lines in different views. The trifocal motion estimation module is configured to estimate the camera motion using the detected points and lines associated with the first and the second trifocal tensor.