Abstract: In one implementation, an apparatus includes a display having a front surface and a back surface. The display includes a plurality of pixel regions that emit light from the front surface to display a displayed image and a plurality of apertures that transmit light from the front surface to the back surface. The apparatus includes a camera disposed on a side of the back surface of the display. The camera is configured to capture a captured image. The apparatus includes a processor coupled to the display and the camera. The processor is configured to receive the captured image and apply a first digital filter to a first portion of the captured image and a second digital filter, different than the first digital filter, to a second portion of the captured image to reduce image distortion caused by the display.

Abstract: In one implementation, an apparatus includes a display having a front surface and a back surface. The display includes a plurality of pixel regions that emit light from the front surface to display a displayed image and a plurality of apertures that transmit light from the front surface to the back surface. The apparatus includes a camera disposed on a side of the back surface of the display. The camera is configured to capture a captured image. The apparatus includes a processor coupled to the display and the camera. The processor is configured to receive the captured image and apply a first digital filter to a first portion of the captured image and a second digital filter, different than the first digital filter, to a second portion of the captured image to reduce image distortion caused by the display.

Abstract: In one implementation, an apparatus includes a display having a front surface and a back surface. The display includes a plurality of pixel regions that emit light from the front surface to display a displayed image and a plurality of apertures that transmit light from the front surface to the back surface. The apparatus includes a camera disposed on a side of the back surface of the display. The camera is configured to capture a captured image. The apparatus includes a processor coupled to the display and the camera. The processor is configured to receive the captured image and apply a first digital filter to a first portion of the captured image and a second digital filter, different than the first digital filter, to a second portion of the captured image to reduce image distortion caused by the display.

Abstract: A device such as a stylus may have a color sensor. The color sensor may have a color sensing light detector having a plurality of photodetectors each of which measures light for a different respective color channel. The color sensor may also have a light emitter. The light emitter may have an adjustable light spectrum. The light spectrum may be adjusted during color sensing measurements using information such as ambient light color measurements made with a color ambient light sensor that has a plurality of photodetectors each of which measures light for a different respective color channel. An inertial measurement unit may be used to measure the angular orientation between the stylus and an external object during color measurements. Arrangements in which the light emitter is modulated during color sensing may also be used. Measurements from the stylus may be transmitted wirelessly to external equipment.

Abstract: In one implementation, an apparatus includes a display having a front surface and a back surface. The display includes a plurality of pixel regions that emit light from the front surface to display a displayed image and a plurality of apertures that transmit light from the front surface to the back surface. The apparatus includes a camera disposed on a side of the back surface of the display. The camera is configured to capture a captured image. The apparatus includes a processor coupled to the display and the camera. The processor is configured to receive the captured image and apply a first digital filter to a first portion of the captured image and a second digital filter, different than the first digital filter, to a second portion of the captured image to reduce image distortion caused by the display.

Abstract: A device such as a stylus may have a color sensor. The color sensor may have a color sensing light detector having a plurality of photodetectors each of which measures light for a different respective color channel. The color sensor may also have a light emitter. The light emitter may have an adjustable light spectrum. The light spectrum may be adjusted during color sensing measurements using information such as ambient light color measurements made with a color ambient light sensor that has a plurality of photodetectors each of which measures light for a different respective color channel. An inertial measurement unit may be used to measure the angular orientation between the stylus and an external object during color measurements. Arrangements in which the light emitter is modulated during color sensing may also be used. Measurements from the stylus may be transmitted wirelessly to external equipment.

Abstract: A device such as a stylus may have a color sensor. The color sensor may have a color sensing light detector having a plurality of photodetectors each of which measures light for a different respective color channel. The color sensor may also have a light emitter. The light emitter may have an adjustable light spectrum. The light spectrum may be adjusted during color sensing measurements using information such as ambient light color measurements made with a color ambient light sensor that has a plurality of photodetectors each of which measures light for a different respective color channel. An inertial measurement unit may be used to measure the angular orientation between the stylus and an external object during color measurements. Arrangements in which the light emitter is modulated during color sensing may also be used. Measurements from the stylus may be transmitted wirelessly to external equipment.

Abstract: In one implementation, an apparatus includes a display having a front surface and a back surface. The display includes a plurality of pixel regions that emit light from the front surface to display a displayed image and a plurality of apertures that transmit light from the front surface to the back surface. The apparatus includes a camera disposed on a side of the back surface of the display. The camera is configured to capture a captured image. The apparatus includes a processor coupled to the display and the camera. The processor is configured to receive the captured image and apply a first digital filter to a first portion of the captured image and a second digital filter, different than the first digital filter, to a second portion of the captured image to reduce image distortion caused by the display.

Abstract: In one implementation, an apparatus includes a display having a front surface and a back surface. The display includes a plurality of pixel regions that emit light from the front surface to display a displayed image and a plurality of apertures that transmit light from the front surface to the back surface. The apparatus includes a camera disposed on a side of the back surface of the display. The camera is configured to capture a captured image. The apparatus includes a processor coupled to the display and the camera. The processor is configured to receive the captured image and apply a first digital filter to a first portion of the captured image and a second digital filter, different than the first digital filter, to a second portion of the captured image to reduce image distortion caused by the display.

Abstract: A mixed reality system that includes a head-mounted display (HMD) that provides 3D virtual views of a user's environment augmented with virtual content. The HMD may include sensors that collect information about the user's environment (e.g., video, depth information, lighting information, etc.), and sensors that collect information about the user (e.g., the user's expressions, eye movement, hand gestures, etc.). The sensors provide the information as inputs to a controller that renders frames including virtual content based at least in part on the inputs from the sensors. The HMD may display the frames generated by the controller to provide a 3D virtual view including the virtual content and a view of the user's environment for viewing by the user.

Abstract: A method of authenticating a user of a wearable electronic device includes emitting light into a dorsal side of a forearm near a wrist of the user; receiving, using a light field camera, remissions of the light from the dorsal side of the forearm near the wrist of the user; generating a light field image from the remissions of the light; performing a synthetic focusing operation on the light field image to construct at least one image of at least one layer of the forearm near the wrist; extracting a set of features from the at least one image; determining whether the set of features matches a reference set of features; and authenticating the user based on the matching. In some embodiments, the method may further include compensating for a tilt of the light field camera prior to or while performing the synthetic focusing operation.

Abstract: A method of authenticating a user of a wearable electronic device includes emitting light into a dorsal side of a forearm near a wrist of the user; receiving, using a light field camera, remissions of the light from the dorsal side of the forearm near the wrist of the user; generating a light field image from the remissions of the light; performing a synthetic focusing operation on the light field image to construct at least one image of at least one layer of the forearm near the wrist; extracting a set of features from the at least one image; determining whether the set of features matches a reference set of features; and authenticating the user based on the matching. In some embodiments, the method may further include compensating for a tilt of the light field camera prior to or while performing the synthetic focusing operation.

Abstract: A method of authenticating a user of a wearable electronic device includes emitting light into a dorsal side of a forearm near a wrist of the user; receiving, using a light field camera, remissions of the light from the dorsal side of the forearm near the wrist of the user; generating a light field image from the remissions of the light; performing a synthetic focusing operation on the light field image to construct at least one image of at least one layer of the forearm near the wrist; extracting a set of features from the at least one image; determining whether the set of features matches a reference set of features; and authenticating the user based on the matching. In some embodiments, the method may further include compensating for a tilt of the light field camera prior to or while performing the synthetic focusing operation.

Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.

Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.

Abstract: A mixed reality system that includes a head-mounted display (HMD) that provides 3D virtual views of a user's environment augmented with virtual content. The HMD may include sensors that collect information about the user's environment (e.g., video, depth information, lighting information, etc.), and sensors that collect information about the user (e.g., the user's expressions, eye movement, hand gestures, etc.). The sensors provide the information as inputs to a controller that renders frames including virtual content based at least in part on the inputs from the sensors. The HMD may display the frames generated by the controller to provide a 3D virtual view including the virtual content and a view of the user's environment for viewing by the user.

Abstract: A method of authenticating a user of a wearable electronic device includes emitting light into a dorsal side of a forearm near a wrist of the user; receiving, using a light field camera, remissions of the light from the dorsal side of the forearm near the wrist of the user; generating a light field image from the remissions of the light; performing a synthetic focusing operation on the light field image to construct at least one image of at least one layer of the forearm near the wrist; extracting a set of features from the at least one image; determining whether the set of features matches a reference set of features; and authenticating the user based on the matching. In some embodiments, the method may further include compensating for a tilt of the light field camera prior to or while performing the synthetic focusing operation.

Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.

Abstract: A mixed reality system that includes a head-mounted display (HMD) that provides 3D virtual views of a user's environment augmented with virtual content. The HMD may include sensors that collect information about the user's environment (e.g., video, depth information, lighting information, etc.), and sensors that collect information about the user (e.g., the user's expressions, eye movement, hand gestures, etc.). The sensors provide the information as inputs to a controller that renders frames including virtual content based at least in part on the inputs from the sensors. The HMD may display the frames generated by the controller to provide a 3D virtual view including the virtual content and a view of the user's environment for viewing by the user.

Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.