Abstract: A system includes a computing device that includes a memory for storing instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes combining, in a nonlinear manner, a first set of vertex displacements that represent the difference between a first animated expression and a neutral animated expression with a second set of vertex displacements that represent the difference between a second animated expression and the neutral animated expression. The number of vertices associated with the first set of vertex displacements of the first animated expression is equivalent to the number of vertices associated with the second set of vertex displacements of the second animated expression.

Abstract: A method of compressing a deep image representation may include receiving a deep image, where the deep image may include multiple pixels, and where each pixel in the deep image may include multiple samples. The method may also include compressing the deep image by combining samples in each pixel that are associated with the same primitives. This process may be repeated on a pixel-by-pixel basis. Some embodiments may use primitive IDs to match pixels to primitives through the rendering and compositing process.

Abstract: A computer-implemented method for handling a modification of an animation having a constraint includes detecting a user modification of an animation that involves at least first and second objects, the first object constrained to the second object during a constrained period and non-constrained to the second object during a non-constrained period. The method includes, based on the user modification, selecting one of at least first and second compensation adjustments for the animation based on a compensation policy; and adjusting the animation according to the selected compensation adjustment.

Abstract: A method of generating unrecorded camera views may include receiving a plurality of 2-D video sequences of a subject in a real 3-D space, where each 2-D video sequence may depict the subject from a different perspective. The method may also include generating a 3-D representation of the subject in a virtual 3-D space, where a geometry and texture of the 3-D representation may be generated based on the 2D video sequences, and the motion of the 3-D representation in the virtual 3-D space is based on motion of the subject in the real 3-D space. The method may additionally include generating a 2-D video sequence of the motion of the 3D representation using a virtual camera in the virtual 3-D space where the perspective of the virtual camera may be different than the perspectives of the plurality of 2-D video sequences.

Abstract: An apparatus is disclosed which may serve as a target for calibrating a camera. The apparatus comprises one or more planar surfaces. The apparatus includes at least one fiducial marking on a planar surface. The set of all planar markings on the apparatus are distinguishable.

Abstract: Methods and systems are disclosed for calibrating a camera using a calibration target apparatus that contains at least one fiducial marking on a planar surface. The set of all planar markings on the apparatus are distinguishable. Parameters of the camera are inferred from at least one image of the calibration target apparatus. In some embodiments, pixel coordinates of identified fiducial markings in an image are used with geometric knowledge of the apparatus to calculate camera parameters.

Abstract: This document describes systems and techniques for representing computer-generated objects with a limit mesh. In general, the systems and techniques may generate an object's form or appearance based on an estimation of how the object would look if shapes that form the object were infinitely subdivided to create a smooth surface for the object. For example, when a user views a distance object in a video game, its shape can be estimated by calculating an approximation of what the object would look like if the polygons that make up the object were infinitely divided. As the user approaches the object, curving solutions that may be embedded in the object can be used to construct curves for the object so that it appears smooth.

Abstract: A system includes a visual data collector for collecting visual information from an image of one or more features of an object. The system also includes a physical data collector for collecting sensor information provided by at one or more sensors attached to the object. The system also includes a computer system that includes a motion data combiner for combining the visual information the sensor information. The motion data combiner is configured to determine the position of a representation of one or more of the feature in a virtual representation of the object from the combined visual information and sensor information. Various types of virtual representations may be provided from the combined information, for example, one or more poses (e.g., position and orientation) of the object may be represented.

Abstract: This disclosure describes technology, which can be implemented as a method, apparatus, and/or computer software embodied in a computer-readable medium, capable of deforming internal geometry within an outer geometry to account for light bending due to refraction, for example, for real-time modeling of a gaze direction of a 3D computer generated character's eyes. In some implementations, internal eye geometry can be deformed to capture the effect of refraction in real-time such that the gaze direction of a 3D character's eyes can be modeled during an animation phase prior to a rendering phase in the production of a computer generated motion picture.

Abstract: An animation rig is displayed in a plurality of reference poses. The animation rig has a pose deformation in each pose. A refinement deformation for each reference pose is received. Each refinement deformation is mapped to a respective point in n-dimensional space. A complex is built in the n-dimensional space. The complex includes a mesh of simplices, each corner of each simplex associated with one of the deformations.

Abstract: In one aspect, a method is performed by one or more processors. The method includes receiving a primary view image showing a scene from a view of a primary camera; depth values for one or more objects in the scene; and a camera offset between the primary camera and a secondary camera. The method further includes determining disparities for each pixel in the primary view image. The method further includes generating a secondary view image using the primary view image, the depth values, and the camera offset. At least some of the pixels are repositioned according to the determined disparities.

Abstract: A method of applying a post-render motion blur to an object may include receiving a first image of the object. The first image need not be motion blurred, and the first image may include a first pixel and rendered color information for the first pixel. The method may also include receiving a second image of the object. The second image may be motion blurred, and the second image may include a second pixel and a location of the second pixel before the second image was motion blurred. Areas that are occluded in the second image may be identified and colored using a third image rendering only those areas. Unoccluded areas of the second image may be colored using information from the first image.

Abstract: Systems and techniques for dynamically capturing and reconstructing lighting are provided. The systems and techniques may be based on a stream of images capturing the lighting within an environment as a scene is shot. Reconstructed lighting data may be used to illuminate a character in a computer-generated environment as the scene is shot. For example, a method may include receiving a stream of images representing lighting of a physical environment. The method may further include compressing the stream of images to reduce an amount of data used in reconstructing the lighting of the physical environment and may further include outputting the compressed stream of images for reconstructing the lighting of the physical environment using the compressed stream, the reconstructed lighting being used to render a computer-generated environment.

Abstract: A virtual director's viewfinder includes: a housing with a form factor of a director's viewfinder; at least one mechanical viewfinder control connected to the housing, the virtual director's viewfinder encoding a setting of the mechanical viewfinder control into digital setting information; at least one communication device for communication of the digital setting information to a computer system, and for receiving from the computer system an image generated using the digital setting information; and a display device that presents at least the image received from the computer system.

Abstract: A system includes a computing device that includes a memory configured to store instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes identifying one or more frames included in series of frames that represent a portion of an animated character. The identified one or more frames can be reconstructed from a portion of the series of frames absent the identified one or more frames. The method also includes removing the identified one or more frames from the series of frames to produce a reduced series of frames, and, transforming the reduced series of frames to produce an animation model that is capable of reconstructing the series of frames for an interactive virtual environment.

Abstract: A device includes a plurality of vertically aligned pairs of light sources. The device is configured to sequentially illuminate the vertically aligned pairs of light sources. The output illuminating can be captured by a camera having a rolling shutter delay to create an image having a rolling shutter artifact. The image can be analyzed to determine the rolling shutter delay of the camera.

Abstract: The present disclosure includes, among other things, systems, methods, and program products for estimating radiant exitance due to subsurface scattering. For example, one or more aspects of the subject matter described in this disclosure can be embodied in one or more methods that include distributing a plurality of sample points across the surface of a 3D object model to be rendered into a 2D image and determining a solid angle subtended by a first sample point and a second sample point relative to a region on the 3D object model. Depending on the determined solid angle relative to a threshold value, a previously determined subsurface scattering contribution for the region or a newly determined subsurface scattering contribution for the region may selectively be used for rendering a portion of the 2D image.

Abstract: Visually representing a composite graph of image functions includes providing a visual representation of a composite graph for an image, the visual representation including first items corresponding to respective image functions, and second items corresponding to containers for image functions, the image to be rendered by performing the image functions in an order defined by the composite graph. The user selects an expansion mode for presenting contents of a first container, each of the containers having a first expansion mode wherein the contents are displayed generally adjacent the visual representation, and a second expansion mode where the contents are displayed generally within the visual representation. A modified visual representation of the composite graph is provided in response to the input, wherein the modified visual representation maintains the order of the image functions defined by the composite graph and has the first container expanded according to the selected expansion mode.

Abstract: Among other aspects, on computer-implemented method includes: receiving at least one command in a computer system from a handheld device; positioning a virtual camera and controlling a virtual scene according to the command; and in response to the command, generating an output to the handheld device for displaying a view of the virtual scene as controlled on a display of the handheld device, the view captured by the virtual camera as positioned.

Abstract: A system includes a computing device that includes a memory configured to store instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes defining at least one of a location in a virtual scene and a time represented in a timeline as being associated with a performance of an animated character. The method also includes aggregating data that represents actions of the animation character for at least one of the defined location and the defined time. The method also includes presenting a user interface that includes a representation of the aggregated actions. The representation is editable to adjust at least one action included in the aggregation.

Abstract: A computer-implemented method for configuring stereo cameras includes detecting, in a computer system, a relocation of a manipulator in a virtual set. The manipulator defined in the computer system to specify an aspect of the virtual set. The method includes receiving an input in the computer system, the input specifying an aspect of a projection environment. The method includes configuring, using the relocation and the input, first and second cameras to capture a stereo view of a scene such that the stereo view is compatible for being projected in the projection environment.

Abstract: A method of applying a post-render motion blur to an object may include receiving a first image of the object. The first image need not be motion blurred, and the first image may include a first pixel and rendered color information for the first pixel. The method may also include receiving a second image of the object. The second image may be motion blurred, and the second image may include a second pixel and a location of the second pixel before the second image was motion blurred. The method may additionally include locating the first pixel in the first image using the location of the second pixel before the second image was motion blurred. The method may further include coloring the second pixel using the rendered color information for the first pixel.

Abstract: A computer program product tangibly embodied in a computer-readable storage medium includes instructions that when executed by a processor perform a method. The method includes identifying a frame of a video sequence, transforming a model into an initial guess for how the region appears in the frame, performing an exhaustive search of the frame, performing a plurality of optimization procedures, wherein at least one additional model parameter is taken into account as each subsequent optimization procedure is initiated. A system includes a computer readable storage medium, a graphical user interface, an input device, a model for texture and shape of the region, the model generated using the video sequence and stored in the computer readable storage medium, and a solver component.

Abstract: A computer-implemented method for providing retargeting of actor motion includes: receiving, in a computer system, an input generated by user selection of at least one of multiple real-time constraint scenarios, each real-time constraint scenario corresponding to a relationship between motion capture information from an actor and a position or movement of a character driven by the motion capture information; activating, based on the selected real-time constraint scenario, one or more of multiple limb solvers for the character which determines at least a first joint angle for a corresponding character limb; registering, for at least one character limb where the corresponding limb solver is not activated, a corresponding joint angle from the motion capture information as a second joint angle; and determining a root location for the character based on at least the first and second joint angles.

Abstract: Interaction between a 3D animation and a corresponding script includes: displaying a user interface that includes at least a 3D animation area and a script area, the 3D animation area including (i) a 3D view area for creating and playing a 3D animation and (ii) a timeline area for visualizing actions by one or more 3D animation characters, the script area comprising one or more objects representing lines from a script having one or more script characters; receiving a first user input corresponding to a user selecting at least one of the objects from the script area for assignment to a location in the timeline area; generating a timeline object at the location in response to the first user input, the timeline object corresponding to the selected object; and associating audio data with the generated timeline object, the audio data corresponding to a line represented by the selected object.

Abstract: A first geometry and a second geometry are accessed. They are positioned so that the first geometry and the second geometry at least in part intersect. The first geometry is divided into portions based on the intersection with the second geometry. At least a first portion of the first geometry is classified as being on one side of the second geometry. At least a second portion of the first geometry is classified as being on another side of the second geometry. At least a third portion of the first geometry is unclassified. The classifying includes comparing an angle weighted normal of a face with a property of the second geometry. The third portion is reclassified as either above the second geometry or below the second geometry according to the classification of a neighboring portion.

Abstract: Techniques for facial performance capture using an adaptive model are provided herein. For example, a computer-implemented method may include obtaining a three-dimensional scan of a subject and a generating customized digital model including a set of blendshapes using the three-dimensional scan, each of one or more blendshapes of the set of blendshapes representing at least a portion of a characteristic of the subject. The method may further include receiving input data of the subject, the input data including video data and depth data, tracking body deformations of the subject by fitting the input data using one or more of the blendshapes of the set, and fitting a refined linear model onto the input data using one or more adaptive principal component analysis shapes.

Abstract: A method of applying a post-render motion blur to an object may include receiving a first image of the object. The first image need not be motion blurred, and the first image may include a first pixel and rendered color information for the first pixel. The method may also include receiving a second image of the object. The second image may be motion blurred, and the second image may include a second pixel and a location of the second pixel before the second image was motion blurred. The method may additionally include locating the first pixel in the first image using the location of the second pixel before the second image was motion blurred. The method may further include coloring the second pixel using the rendered color information for the first pixel.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, are disclosed for animations that simulate skin deformation relative to a muscle. Displacement of a skin representation in response to deformation of a muscle representation and sliding of the skin representation over the deformed muscle representation can be determined quickly and interactively by monitoring changes in position and scale of defining ring elements of the muscle representation's geometry. The determined in displacements can be constrained to prevent portions of the displaced skin representation from sinking into the underlying muscle representation's geometry.

Abstract: In one general aspect, a method is described. The method includes generating a positional relationship between one or more support structures having at least one motion capture mark and at least one virtual structure corresponding to geometry of an object to be tracked and positioning the support structures on the object to be tracked. The support structures has sufficient rigidity that, if there are multiple marks, the marks on each support structure maintain substantially fixed distances from each other in response to movement by the object. The method also includes determining an effective quantity of ray traces between one or more camera views and one or more marks on the support structures, and estimating an orientation of the virtual structure by aligning the determined effective quantity of ray traces with a known configuration of marks on the support structures.

Abstract: A movie set can include light sources each producing a light corresponding to a light channel, at least one high-frame rate camera, and a controller connected to the light sources and camera to synchronize the camera and light sources. The number of light channels can be proportional to the frame rate. For example, if the filming frame rate is 120 frames per second (fps) and the playback frame rate is 24 fps, then 5 light channels can be used. In this example, for every one playback frame, 5 frames were filmed by the high frame rate camera. The controller modulates the light channels such that each of the 5 frames has different lighting characteristics. In post-production, contributions from each of the light channels can be included or excluded from the final frame. An optical flow algorithm can be used to stitch together frames with different dynamic light characteristics.

Abstract: A system includes a computing device that includes a memory configured to store instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes obtaining first data corresponding to a first simulation of matter in a space domain. The method also includes performing, using the first data, a second simulation that produces second data representative of particles in the space domain. The method also includes rasterizing the second data representative of the particles as defined by cells of a grid, wherein each cell has a common depth-to-size ratio, and, rendering an image of the particles from the rasterized second data.

Abstract: Among other aspects, on computer-implemented method includes: receiving at least one command in a computer system from a handheld device; positioning a virtual camera and controlling a virtual scene according to the command; and in response to the command, generating an output to the handheld device for displaying a view of the virtual scene as controlled on a display of the handheld device, the view captured by the virtual camera as positioned.

Abstract: In one aspect, a computer implemented method of motion capture, the method includes tracking the motion of a dynamic object bearing a pattern configured such that a first portion of the patterns is tracked at a first resolution and a second portion of the pattern is tracked at a second resolution. The method further includes causing data representing the motion to be stored to a computer readable medium.

Abstract: An apparatus includes a user wearable viewing device for assisting the user to view a representation of three dimensional content. The user wearable viewing device is assigned a unique identifier to provide a private viewing experience for the user.

Abstract: A system includes a computing device that includes a memory configured to store instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes projecting a first simulated ray from a moveable character within an interactive virtual environment to intersect one or more virtual surfaces represented in the environment. The method also includes rendering a graphical representation of the first simulated ray to identify the absence of an intersection with the one or more virtual surfaces represented in the interactive virtual environment.

Abstract: A system includes a computing device that includes a memory configured to store instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes representing animation states of a virtual character in editable graphical representations. Each animation state represents each individual action of the character for an instance in time. The method also includes storing data that represents one or more changes in the animation states of the virtual character from the editable graphical representations. A pose of the virtual character is reconstructable upon retrieval of the stored data.

Abstract: A computer-implemented method of controlling operations for execution includes displaying, in a display device area that controls a process of operations, a current container providing at least a first operation to the process, and a linear sequence representing parent containers of the current container organized in a hierarchy. The method includes receiving, in a system generating the display device area, a partial-performance request identifying a subset of the parent containers defined between the current container and a level in the hierarchy. The method includes performing, in response to the partial-performance request, the first operation and those of the operations that the subset of the parent containers provide to the process.

Abstract: A computer-implemented method includes selecting a subset of images from a set of captured images. A surface feature of one object is represented in the content of each selected subset image. The method also includes decomposing the surface feature content of each selected image to produce a model of representations of the object.

Abstract: A method of generating an animation rig for a three-dimensional (3-D) computing environment may include providing a rig generation environment that includes a library storing a plurality of blocks. Each of the plurality of blocks may represent particular rig elements and include information for generating the rig elements in the 3-D computing environment along with a first icon that is visually representative of the particular rig elements. The method may also include receiving and displaying two or more blocks in the rig generation environment, and receiving one or more graphical connections between the blocks. The method may additionally include generating the animation rig using the information of each of the two or more blocks and sending the animation rig from the rig generation environment to the 3-D computing environment.

Abstract: A computer-implemented method includes comparing content captured during one session and content captured during another session. A surface feature of an object represented in the content of one session corresponds to a surface feature of an object represented in the content of the other session. The method also includes substantially aligning the surface features of the sessions and combining the aligned content.

Abstract: A system includes a computing device that includes a memory configured to store instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes presenting a representation of three dimensional content on a single display. The three dimensional content representation is presented as a shared viewing experience for at least two viewers. The method also includes presenting a representation of two dimensional content on the single display to a first viewer of the at least two viewers. The two dimensional representation is presented as a private viewing experience for the first viewer and a second viewer of the at least two viewers is visually isolated from the presented two dimensional content representation.

Abstract: In one general aspect, a method is described. The method includes generating a positional relationship between one or more support structures having at least one motion capture mark and at least one virtual structure corresponding to geometry of an object to be tracked and positioning the support structures on the object to be tracked. The support structures has sufficient rigidity that, if there are multiple marks, the marks on each support structure maintain substantially fixed distances from each other in response to movement by the object. The method also includes determining an effective quantity of ray traces between one or more camera views and one or more marks on the support structures, and estimating an orientation of the virtual structure by aligning the determined effective quantity of ray traces with a known configuration of marks on the support structures.

Abstract: In one aspect, a method performed by one or more processes executing on a computer system, the method includes displaying in a user interface at least a portion of a 3D object model and one or more surface controls that collectively and implicitly define a control surface that is positioned in 3D space relative to, and capable of deforming, the 3D object model, each surface control having a single associated direction, directions of deformations of the control surface based on the associated directions. The method further includes receiving input from a human animator defining a change to the implicitly defined control surface relative to the 3D object model. The method further includes in response to the received input, updating the user interface to display the 3D object model as deformed by the changed implicitly defined control surface.

Abstract: A method is described that includes receiving, from a first device, input used to select a first object in a computer-generated environment. The first device has at least two degrees of freedom with which to control the selection of the first object. The method also includes removing, in response to the selection of the first object, at least two degrees of freedom previously available to a second device used to manipulating a second object in the computer-generated environment. The removed degrees of freedom correspond to the at least two degrees of freedom of the first device and specify an orientation of the second object relative to the selected first object. Additionally, the method includes receiving, from the second device, input including movements within the reduced degrees of freedom used to manipulate a position of the second object while maintaining the specified orientation relative to the selected first object.

Abstract: A system includes a computing device that includes a memory configured to store instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes receiving a model that represents the geometry of an animated character and represents a primary deformation applied the geometry of the character. The applied primary deformation is relative to a rest geometry of the character. The method also includes applying a secondary deformation to the geometry of the received character model. The applied secondary deformation is relative to the primary deformation applied to the geometry of the character.

Abstract: A motion library can be created by generating motion feature vectors for at least some of multiple frames of a video sequence using a 3D mesh, each motion feature vector corresponding to characteristics of the body deformation in one of the frames. The A user can select a subset of the frames. For each frame in the subset, the user can define settings for controls of an animation character, the settings selected by the user to correspond to the body deformation in the respective frame. Mappings are generated using the settings and the motion feature vectors, the mappings regulating the controls based on multiple motion feature vectors. The motion library can be used to generate an animation from an actor performance.

Abstract: A computer program product tangibly embodied in a computer-readable storage medium includes instructions that when executed by a processor perform a method. The method includes identifying a frame of a video sequence, transforming a model into an initial guess for how the region appears in the frame, performing an exhaustive search of the frame, performing a plurality of optimization procedures, wherein at least one additional model parameter is taken into account as each subsequent optimization procedure is initiated. A system includes a computer readable storage medium, a graphical user interface, an input device, a model for texture and shape of the region, the model generated using the video sequence and stored in the computer readable storage medium, and a solver component.

Abstract: A system includes a computing device that includes a memory configured to store instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes defining a first portion of a display as a workspace for presenting graphical elements that represent processing operations that define properties of an object. The method also includes defining a second portion of the display, different from the first portion, for presenting a graphical stack that uses the properties from the processing operations to define layers associated with the object. The method also includes simultaneously presenting the first and the second portion on the display.

Abstract: A system includes a computing device that includes a memory configured to store instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes identifying a portion of a representation of a character in an animation. The identified portion is associated with a feature of the character to be represented in a reconstructed version of the animation. The method also includes compressing the identified portion of the character representation and other portions of the character representation to produce a model of the character that is capable of reconstructing the animation.