Abstract: A vehicle camera system captures and transmits video to a user device, which includes a viewing device for playback of the captured video, such as virtual reality or augmented reality glasses. A rendering map is generated that indicates which pixels of the video frame (as identified by particular coordinates of the video frame) correspond to which coordinates of a virtual sphere in which a portion of the video frame is rendered for display. When a video frame is received, the rendering map is used to determine the texture values (e.g., colors) for coordinates in the virtual sphere, which is used to generate the display for the user. This technique reduces the rendering time when a user turns his or her head (e.g., while in virtual reality) and so it reduces motion and/or virtual reality sickness induced by the rendering lag.

Abstract: A three-dimensional shape parameter computation system and method for computing three-dimensional human head shape parameters from two-dimensional facial feature points. A series of images containing a user's face is captured. Embodiments of the system and method deduce the 3D parameters of the user's head by examining a series of captured images of the user over time and in a variety of head poses and facial expressions, and then computing an average. An energy function is constructed over a batch of frames containing 2D face feature points obtained from the captured images, and the energy function is minimized to solve for the head shape parameters valid for the batch of frames. Head pose parameters and facial expression and animation parameters can vary over each captured image in the batch of frames. In some embodiments this minimization is performed using a modified Gauss-Newton minimization technique using a single iteration.

Abstract: A method for operating an image processing device coupled to a color camera and a depth camera is provided. The method includes receiving a color image of a 3-dimensional scene from a color camera, receiving a depth map of the 3-dimensional scene from a depth camera, generating an aligned 3-dimensional face mesh from a plurality of color images received from the color camera indicating movement of a subject's head within the 3-dimensional scene and form the depth map, determining a head region based the depth map, segmenting the head region into a plurality of facial sections based on both the color image, depth map, and the aligned 3-dimensional face mesh, and overlaying the plurality of facial sections on the color image.

Abstract: A vehicle camera system captures and transmits video to a user device, which includes a viewing device for playback of the captured video, such as virtual reality or augmented reality glasses. A rendering map is generated that indicates which pixels of the video frame (as identified by particular coordinates of the video frame) correspond to which coordinates of a virtual sphere in which a portion of the video frame is rendered for display. When a video frame is received, the rendering map is used to determine the texture values (e.g., colors) for coordinates in the virtual sphere, which is used to generate the display for the user. This technique reduces the rendering time when a user turns his or her head (e.g., while in virtual reality) and so it reduces motion and/or virtual reality sickness induced by the rendering lag.

Abstract: A method for operating an image processing device coupled to a color camera and a depth camera is provided. The method includes receiving a color image of a 3-dimensional scene from a color camera, receiving a depth map of the 3-dimensional scene from a depth camera, generating an aligned 3-dimensional face mesh from a plurality of color images received from the color camera indicating movement of a subject's head within the 3-dimensional scene and form the depth map, determining a head region based the depth map, segmenting the head region into a plurality of facial sections based on both the color image, depth map, and the aligned 3-dimensional face mesh, and overlaying the plurality of facial sections on the color image.

Abstract: Methods and systems for face detection and tracking using an image-based capture device are disclosed herein. The method includes generating a depth image of a scene, generating a mask image from the depth image, and detecting a position of a face of a user in the scene using the mask image. The method also includes determining an intensity of the face using a first color channel of the mask image and adjusting a gain level of a first color channel of the image-based capture device directed at the scene to achieve a target intensity of the face.

Abstract: A three-dimensional shape parameter computation system and method for computing three-dimensional human head shape parameters from two-dimensional facial feature points. A series of images containing a user's face is captured. Embodiments of the system and method deduce the 3D parameters of the user's head by examining a series of captured images of the user over time and in a variety of head poses and facial expressions, and then computing an average. An energy function is constructed over a batch of frames containing 2D face feature points obtained from the captured images, and the energy function is minimized to solve for the head shape parameters valid for the batch of frames. Head pose parameters and facial expression and animation parameters can vary over each captured image in the batch of frames. In some embodiments this minimization is performed using a modified Gauss-Newton minimization technique using a single iteration.

Abstract: A three-dimensional shape parameter computation system and method for computing three-dimensional human head shape parameters from two-dimensional facial feature points. A series of images containing a user's face is captured. Embodiments of the system and method deduce the 3D parameters of the user's head by examining a series of captured images of the user over time and in a variety of head poses and facial expressions, and then computing an average. An energy function is constructed over a batch of frames containing 2D face feature points obtained from the captured images, and the energy function is minimized to solve for the head shape parameters valid for the batch of frames. Head pose parameters and facial expression and animation parameters can vary over each captured image in the batch of frames. In some embodiments this minimization is performed using a modified Gauss-Newton minimization technique using a single iteration.

Abstract: A three-dimensional shape parameter computation system and method for computing three-dimensional human head shape parameters from two-dimensional facial feature points. A series of images containing a user's face is captured. Embodiments of the system and method deduce the 3D parameters of the user's head by examining a series of captured images of the user over time and in a variety of head poses and facial expressions, and then computing an average. An energy function is constructed over a batch of frames containing 2D face feature points obtained from the captured images, and the energy function is minimized to solve for the head shape parameters valid for the batch of frames. Head pose parameters and facial expression and animation parameters can vary over each captured image in the batch of frames. In some embodiments this minimization is performed using a modified Gauss-Newton minimization technique using a single iteration.

Abstract: Methods and systems for face detection and tracking using an image-based capture device are disclosed herein. The method includes generating a depth image of a scene, generating a mask image from the depth image, and detecting a position of a face of a user in the scene using the mask image. The method also includes determining an intensity of the face using a first color channel of the mask image and adjusting a gain level of a first color channel of the image-based capture device directed at the scene to achieve a target intensity of the face.