Abstract: Techniques and systems are described for generating an anatomically-constrained local model and for performing performance capture using the model. The local model includes a local shape subspace and an anatomical subspace. In one example, the local shape subspace constrains local deformation of various patches that represent the geometry of a subject's face. In the same example, the anatomical subspace includes an anatomical bone structure, and can be used to constrain movement and deformation of the patches globally on the subject's face. The anatomically-constrained local face model and performance capture technique can be used to track three-dimensional faces or other parts of a subject from motion data in a high-quality manner. Local model parameters that best describe the observed motion of the subject's physical deformations (e.g., facial expressions) under the given constraints are estimated through optimization.

Abstract: A technique for fabricating a highlight hologram based on a digital object performs point sampling on the object and represents each sampled point as a geometric patch. A set of geometric patches corresponding to sampled points from the object are fabricated into a substrate. A paraboloid patch may be used for reflective substrates while a hyperboloid may be used for transmissive substrates. To avoid specifying overlapping patches, which are impractical to fabricate, certain of the sample points may be merged. An output set of grooves is saved and may be used to specify fabrication of a highlight hologram on the physical substrate.

Abstract: Approaches are described for generating a multiview autostereoscopic image from a stereo three-dimensional input image pair. A stereo to multiview rendering system receives a stereo three-dimensional image including a left image and a right image at a first view position and a second view position, respectively. The system generates a first input warp and a second input warp that maps the left image and the right image to a third and fourth positions, respectively, where the third and fourth positions lie between the first and second view positions. The system generates a plurality of output warps based on the first warp and the second warp. The system resamples each output warp in the plurality of output warps to create a plurality of partial output images. The system interleaves the plurality of partial output images to generate a composite output image.

Abstract: Methods and systems of reconstructing an eyelid are provided. A method of reconstructing an eyelid includes obtaining one or more images of the eyelid, generating one or more image input data for the one or more images of the eyelid, generating one or more reconstruction data for the one or more images of the eyelid, and reconstructing a spatio-temporal digital representation of the eyelid using the one or more input image data and the one or more reconstruction data.

Abstract: As described herein, an electronic device with a display screen may simulate the color diffusion that occurs in a physical painting process. For instance, the user may perform one or more actions that simulate a brushstroke on the display screen such as swiping a touch-sensitive area or dragging a cursor across the screen. The electronic device then calculates a geodesic distance between a pixel inside a region defined by the brushstroke and a pixel located outside this region based on the physical distance between the two pixels and a weighting factor that varies depending on whether an image boundary is between the two pixels. Based on the geodesic distance, the electronic device uses a color diffusion relationship that defines the effect of the color of the brushstroke on the pixel and a time delay controlling when the color of the brushstroke reaches the pixel in order to simulate color diffusion.

Abstract: Embodiments provide techniques for adjusting coloration of an image. A visual scene is captured using one or more camera devices, where the visual scene includes a reference object. Embodiments retrieve coloration information corresponding to the reference object. The coloration information generally describes coloration of the reference object under predefined conditions, such as neutral lighting conditions, desired lighting conditions, and so on. Additionally, a coloration difference is calculated between a depiction of the reference object within the captured visual scene and the retrieved coloration information. The coloration of the visual scene is adjusted based on the determined coloration difference.

Abstract: Techniques are disclosed for generating autostereoscopic video content. A multiscopic video frame is received that includes a first image and a second image. The first and second images are analyzed to determine a set of image characteristics. A mapping function is determined based on the set of image characteristics. At least a third image is generated based on the mapping function and added to the multiscopic video frame.

Abstract: Approaches are described for filtering a first image frame in a sequence of image frames. A tone-mapping pipeline applies a spatiotemporal filter to each pixel of the first image frame, based on a forward optical flow and a backward optical flow, to produce a base layer image. The tone-mapping pipeline applies a temporal filter to each pixel of the first image frame, based on the forward optical flow and the backward optical flow, to produce a temporally filtered frame. The tone-mapping pipeline produces a detail layer image based on the base layer image and the temporally filtered frame.

Abstract: System, method, and computer program product to perform an operation, comprising sampling a plurality of points and a plurality of segments of a curve on a surface of a three-dimensional model, storing each sampled point as a respective vertex of a plurality of vertices and each sampled segment as a respective half-edge in a curve network of the model surface, upon determining that a first half-edge and a second half-edge connect two of the plurality of vertices, generating a first halfchain connecting the first half-edge and the second half-edge, wherein each connected vertex comprises either a corner or an open endpoint, and upon determining that three consecutive halfchains form a loop comprising at least three corners, generating a first patch for a space enclosed by the loop, wherein the first patch is represented as a quad mesh with a respective set of vertices, faces, and half-edges.

Abstract: Techniques are provided for content-aware video retargeting. An interactive framework combines key frame-based constraint editing with numerous automatic algorithms for video analysis. This combination gives content producers a high level of control of the retargeting process. One component of the framework is a non-uniform, pixel-accurate warp to the target resolution that considers automatic as well as interactively-defined features. Automatic features comprise video saliency, edge preservation at the pixel resolution, and scene cut detection to enforce bilateral temporal coherence. Additional high level constraints can be added by the producer to achieve a consistent scene composition across arbitrary output formats. Advantageously, embodiments of the invention provide a better visual result for retargeted video when compared to using conventional techniques.

Abstract: Techniques are provided to model hair and skin. Multiscopic images are received that depict at least part of a subject having hair. The multiscopic images are analyzed to determine hairs depicted. Two-dimensional hair segments are generated that represent the hairs. Three-dimensional hair segments are generated based on the two-dimensional hair segments. A three-dimensional model of skin is generated based on the three-dimensional hair segments.

Abstract: LEDs that transmit or receive data in a VLC channel may create a flickering effect which is caused by the human eye being able to perceive the fluctuations of the light intensity in a LED. To prevent flickering such that the LED is perceived as being illuminated at a constant intensity, the LEDs may emit light based on a pattern of dark and energy slots. During the dark slots, the LEDs may be reversed bias to determine how much light is received. Based on comparing the light received, each VLC system may synchronize respective clocks such that the dark slots align. The VLC systems may then transmit a data frame preamble instructing the other VLC systems to cease synchronization and use the dark slots to receive the data from the data frame. While transmitting the data frame, the VLC systems may continue to use the energy slots to prevent flicker.

Abstract: A closed-loop control system for stereoscopic video capture is provided. At least two motorized lenses are positioned in accordance with specified parameters to capture spatially-disparate images of a scene. The motorized lenses focus light on a corresponding one of the at least two sensors, which generate image streams. One or more processors execute instructions to provide a stream analyzer and a control module. The stream analyzer receives the image streams from the sensors and analyzes the image streams and the specified parameters in real time; the stream analyzer then modifies the image streams and generates metadata. The control module then receives and analyzes the image streams and metadata and transmits updated parameters to a control mechanism that is coupled to the at least two motorized lenses. The control mechanism then modifies operation of the at least two motorized lenses in real time in accordance with the updated parameters.

Abstract: In an animation processing system, generating images to be viewable on a display using a computer that are generated based on scene geometry obtained from computer readable storage and animation data representing changes over time of scene geometry elements, but also images can be modified to include shading that is a function of positions of objects at other than the current instantaneous time for a frame render such that the motion effect shading would suggest motion of at least one of the elements to a viewer of the generated images. Motion effects provide, based on depiction parameters and/or artist inputs, shading that varies for at least some received animation data, received motion depiction parameters, for at least one pixel, a pixel color is rendered based on motion effect program output and at least some received scene geometry, such that the output contributes to features that would suggest the motion.

Abstract: Methods and systems for generating stereoscopic content with granular control over binocular disparity based on multi-perspective imaging from representations of light fields are provided. A reference image, a three-dimensional (“3D”) representation of a light field corresponding to the reference image, and a goal disparity image that indicates a goal binocular disparity for one or more pixels of the reference image may be received. For each pixel of an output image corresponding to the reference image, a point within the light field that is a closest match for the goal binocular disparity of a corresponding pixel of the goal disparity image may be determined. A stereoscopic image pair including the reference image and the output image may be generated.

Abstract: Techniques for displaying content using an augmented reality device are described. Embodiments provide a visual scene for display, the visual scene captured using one or more camera devices of the augmented reality device. Embodiments adjust physical display geometry characteristics of the visual scene to correct for optimal projection. Additionally, illumination characteristics of the visual scene are modified based on environmental illumination data to improve realism of the visual scene when it is displayed. Embodiments further adjust display characteristics of the visual scene to improve tone mapping output. The adjusted visual scene is then output for display on the augmented reality device.

Abstract: Embodiments provide techniques for creating a composite video stream. Additionally, a first selection of pixels from the first video stream and a second selection of pixels from the second video stream are received. Here, both the first selection of pixels and the second selection of pixels indicate pixels that are to be included in the composite video stream. Embodiments identify a plurality of spatiotemporal seams across the first video stream and the second video stream, based at least in part on the first selection of pixels and the second selection of pixels. The first video stream and the second video stream are then composited into the composite video stream, by joining frames from the first video stream and the second video stream at the identified plurality of spatiotemporal seams.

Abstract: Rendering 3D paintings can be done by compositing brush strokes embedded in space. Image elements are rendered into an image representable by a pixel array wherein at least some of the image elements correspond to simulated painting strokes. A method may include determining stroke positions in a 3D space, determining stroke orders, and for each pixel to be addressed, determining a pixel color value by determining strokes intersections with a view ray for that pixel, determining a depth order and a stroke order for intersecting fragments, each fragment having a color, alpha value, depth, and stroke order, assigning an intermediate color to each of the fragments, corresponding to a compositing of nearby fragments in stroke order, and assigning a color to the pixel that corresponds to a compositing of the fragments using the intermediate colors assigned to the fragments. The compositing may be done in depth order.

Abstract: Techniques are presented for controlling the amount of temporal noise in certain animation sequences. Sketchy animation sequences are received in an input in a digital form and used to create an altered version of the same animation with temporal coherence enforced down to the stroke level, resulting in a reduction of the perceived noise. The amount of reduction is variable and can be controlled via a single parameter to achieve a desired artistic effect.

Abstract: A computer-implemented method is provided for physical face cloning to generate a synthetic skin. Rather than attempt to reproduce the mechanical properties of biological tissue, an output-oriented approach is utilized that models the synthetic skin as an elastic material with isotropic and homogeneous properties (e.g., silicone rubber). The method includes capturing a plurality of expressive poses from a human subject and generating a computational model based on one or more material parameters of a material. In one embodiment, the computational model is a compressible neo-Hookean material model configured to simulate deformation behavior of the synthetic skin. The method further includes optimizing a shape geometry of the synthetic skin based on the computational model and the captured expressive poses. An optimization process is provided that varies the thickness of the synthetic skin based on a minimization of an elastic energy with respect to rest state positions of the synthetic skin.