Abstract: A method may include displaying, on one or more display devices in a virtual-reality environment, a visual representation of a 3-D virtual scene from the perspective of a subject location in the virtual-reality environment. The method may also include displaying, on the one or more display devices, a chroma-key background with the visual representation. The method may further include recording, using a camera, an image of the subject in the virtual-reality environment against the chroma-key background.

Abstract: Methods are disclosed for the computer generation of data for images that include hair, fur, or other strand-like material. A volume for the hair is specified, having a plurality of surfaces. A fluid flow simulation is performed within the volume, with a first surface of the volume being a source area through which fluid is simulated to enter the volume, and a second surface being an exit surface through which fluid is simulated as exiting the volume. The fluid flow simulation may be used to produce fluid flow lines, such as from a velocity vector field for the fluid. Fluid flow lines are selected, and image data of hairs that follow the fluid flow lines are generated. Other embodiments include generating animation sequences by generating images wherein the volume and surfaces vary between frames.

Abstract: A method may include receiving a plurality of objects from a 3-D virtual scene. The plurality of objects may be arranged in a hierarchy. The method may also include generating a plurality of identifiers for the plurality of objects. The plurality of identifiers may include a first identifier for a first object in the plurality of objects, and the identifier may be generated based on a position of the first object in the hierarchy. The method may additionally include performing a rendering operation on the plurality of objects to generate a deep image. The deep image may include a plurality of samples that correspond to the first object. The method may further include propagating the plurality of identifiers through the rendering operation such that each of the plurality of samples in the deep image that correspond to the first object are associated with the identifier.

Abstract: A method of rendering stereo images includes receiving a first image of an object comprising rendered color information. The first image of the object is rendered using a first camera position. The method also includes receiving a second image of the object. The second image of the object is rendered using a second camera position, and pixels in the second image comprise locations of corresponding pixels in the first image. The method additionally includes coloring the pixels in the second image using colors of the corresponding pixels in the first image. The second image and the third image are stereo images for a 3-D presentation.

Abstract: A handheld device includes: an input control configured to control and modify a virtual scene including a virtual camera; and a display that shows a representation of the controlled and modified virtual scene generated by the virtual camera. A system includes: a computer system configured to execute program instructions for generating a virtual scene including a virtual camera; and handheld device configured to communicate with the computer system for controlling and modifying the virtual scene, the handheld device comprising: an input control configured to control and modify the virtual scene; and a display that shows a representation of the controlled and modified virtual scene generated by the virtual camera.

Abstract: A multi-channel tracking pattern is provided along with techniques and systems for performing motion capture using the multi-channel tracking pattern. The multi-channel tracking pattern includes a plurality of shapes having different colors on different portions of the pattern. The portions with the unique shapes and colors allow a motion capture system to track motion of an object bearing the pattern across a plurality of video frames.

Abstract: A method includes receiving a first motion path for an object, where an orientation of the object is not aligned with the first motion path for the object for at least a portion of the first motion path. The method also includes receiving a first motion path for a virtual camera and determining a speed of the object along the first motion path for the object. The method additionally includes calculating a second motion path for the object based on the speed of the object along the first motion path for the object and the orientation of the object, where the second motion path of the object is aligned with second motion path. The method further includes calculating a second motion path for the virtual camera based on a difference between the first motion path of the object and the second motion path of the object.

Abstract: In one aspect, a computer implemented method of motion capture, the method includes tracking the motion of a dynamic object bearing a pattern configured such that a first portion of the patterns is tracked at a first resolution and a second portion of the pattern is tracked at a second resolution. The method further includes causing data representing the motion to be stored to a computer readable medium.

Abstract: Representing a connection between objects in a simulation includes identifying, on a source object to be used in a simulation process, a source point that is available for creating a connection to another object. On a target object, a target point is identified for use in creating the connection. At least one of the source object and the target object is a rigid object. A spring element having respective ends at the source point and the target point is created, and at least one property thereof is set. A system includes an animation module, and a simulation module that performs a simulation process involving creating, and setting a property of, a spring element having respective ends at a source point on the source object and a target point on the target object.

Abstract: Performance capture systems and techniques are provided for capturing a performance of a subject and reproducing an animated performance that tracks the subject's performance. For example, systems and techniques are provided for determining control values for controlling an animation model to define features of a computer-generated representation of a subject based on the performance. A method may include obtaining input data corresponding to a pose performed by the subject, the input data including position information defining positions on a face of the subject. The method may further include obtaining an animation model for the subject that includes adjustable controls that control the animation model to define facial features of the computer-generated representation of the face, and matching one or more of the positions on the face with one or more corresponding positions on the animation model.

Abstract: Methods and systems are disclosed for calibrating a camera using a calibration target apparatus that contains at least one fiducial marking on a planar surface. The set of all planar markings on the apparatus are distinguishable. Parameters of the camera are inferred from at least one image of the calibration target apparatus. In some embodiments, pixel coordinates of identified fiducial markings in an image are used with geometric knowledge of the apparatus to calculate camera parameters.

Abstract: A handheld device includes: an input control configured to control and modify a virtual scene including a virtual camera; and a display that shows a representation of the controlled and modified virtual scene generated by the virtual camera. A system includes: a computer system configured to execute program instructions for generating a virtual scene including a virtual camera; and handheld device configured to communicate with the computer system for controlling and modifying the virtual scene, the handheld device comprising: an input control configured to control and modify the virtual scene; and a display that shows a representation of the controlled and modified virtual scene generated by the virtual camera.

Abstract: A method includes receiving a first motion path for an object, where an orientation of the object is not aligned with the first motion path for the object for at least a portion of the first motion path. The method also includes receiving a first motion path for a virtual camera and determining a speed of the object along the first motion path for the object. The method additionally includes calculating a second motion path for the object based on the speed of the object along the first motion path for the object and the orientation of the object, where the second motion path of the object is aligned with second motion path. The method further includes calculating a second motion path for the virtual camera based on a difference between the first motion path of the object and the second motion path of the object.

Abstract: Performance capture systems and techniques are provided for capturing a performance of a subject and reproducing an animated performance that tracks the subject's performance. For example, systems and techniques are provided for determining control values for controlling an animation model to define features of a computer-generated representation of a subject based on the performance. A method may include obtaining input data corresponding to a pose performed by the subject, the input data including position information defining positions on a face of the subject. The method may further include obtaining an animation model for the subject that includes adjustable controls that control the animation model to define facial features of the computer-generated representation of the face, and matching one or more of the positions on the face with one or more corresponding positions on the animation model.

Abstract: Systems and techniques are provided for performing animation motion capture of objects within an environment. For example, a method may include obtaining input data including a three-dimensional point cloud of the environment. The three-dimensional point cloud is generated using a three-dimensional laser scanner including multiple laser emitters and multiple laser receivers. The method may further include obtaining an animation model for an object within the environment. The animation model includes a mesh, an animation skeleton rig, and adjustable controls that control the animation skeleton rig to define a position of one or more faces of the mesh. The method may further include determining a pose of the object within the environment. Determining a pose includes fitting the one or more faces of the mesh to one or more points of a portion of the three-dimensional point cloud. The portion of the three-dimensional point cloud corresponds to the object in the environment.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for animation. An animation application creates an animation environment. An input device receives input from a user to the animation application. An output device displays output to the user of the animation application. The animation application is configured to have a mode of operation that includes displaying, through the output device, a 3D animation view of the animation environment overlain by a 2D edit view of the animation environment.

Abstract: A stereoscopic camera calibration target includes: first illumination points on a first surface; second illumination points on a second surface, the first and second surfaces being planar, parallel to each other, and spaced from each other; and circuitry that sets a strobe frequency of the first and second illumination points. A method includes: moving a calibration target in front of a stereoscopic camera, the calibration target comprising first points on a first surface and second points on a second surface, the first and second surfaces being planar, parallel to each other, and spaced from each other; capturing, using the stereoscopic camera, an image sequence of the calibration target, the image sequence comprising pairs of left and right images of at least some of the first and second points; determining a calibration value for the stereoscopic camera using the image sequence; and processing the image sequence using the calibration value.

Abstract: A 3D animation environment that includes an animation object is generated. A movement speed is assigned to object the 3D animation environment. An animation path containing at least first and second waypoints is generated. An animation sequence is generated by identifying a first section of the animation path connected to the first waypoint. A first animation of the animation object is generated in which the animation object moves along the first section of the path at the movement speed. A spatial gap in the animation path is identified between the first and second waypoints. A second animation of the animation object is generated in which the animation object moves, by keyframe animation, from the first waypoint to the second waypoint. A third animation of the animation object is generated in which the animation object moves along at least a second portion of the path at the movement speed.

Abstract: A method of generating an animation rig for a three-dimensional (3-D) computing environment may include providing a rig generation environment that includes a library storing a plurality of blocks. Each of the plurality of blocks may represent particular rig elements and include information for generating the rig elements in the 3-D computing environment along with a first icon that is visually representative of the particular rig elements. The method may also include receiving and displaying two or more blocks in the rig generation environment, and receiving one or more graphical connections between the blocks. The method may additionally include generating the animation rig using the information of each of the two or more blocks and sending the animation rig from the rig generation environment to the 3-D computing environment.

Abstract: A method of generating an animation rig for a 3-D computing environment may include accessing a plurality of blocks, each representing a particular rig element, and including interfaces that govern interactions with other rig elements and information for generating a portion of the animation rig corresponding to the particular rig element represented by the block. The method may also include receiving a selection of a first block that includes a first interface and first information and represents a first rig element, and receiving a selection of a second block that includes a second interface and second information and represents a second rig element. The method may additionally include receiving an instruction to connect the first interface to the second interface, generating the animation rig using the first information and the second information, and providing the animation rig to the 3-D computing environment.

Abstract: Among other disclosed subject matter, a computer program product is tangibly embodied in a computer-readable storage medium and includes instructions that when executed by a processor perform a method for detecting collision between objects. The method includes identifying a first edge of a first object, and a second edge of a second object, presented on a display, the second object associated with a transformation. The method includes performing an inverse of the transformation on the first object while not performing the transformation on the second object. The method includes generating an output on the display that indicates whether the first and second objects collide, the output based on performing the inverse of the transformation.

Abstract: A system and method facilitating a user to manipulate a virtual reality (VR) environment are disclosed. The user may provide an input via a touch sensitive surface of a computing device associated with the user to bind a virtual object in the VR environment to the computing device. The user may then move and/or rotate the computing device to cause the bound virtual object to move and/or rotate in the VR environment accordingly. In some examples, the bound virtual object may cast a ray into the VR environment. The movement and/or rotation of the virtual object controlled by the computing device in those examples can change the direction of the ray. In some examples, the virtual object may include a virtual camera. In those examples, the user may move and/or rotate the virtual camera in the VR environment by moving and/or rotate the computing device.

Abstract: A system and method for controlling a view of a virtual reality (VR) environment via a computing device with a touch sensitive surface are disclosed. In some examples, a user may be enabled to augment the view of the VR environment by providing finger gestures to the touch sensitive surface. In one example, the user is enabled to call up a menu in the view of the VR environment. In one example, the user is enabled to switch the view of the VR environment displayed on a device associated with another user to a new location within the VR environment. In some examples, the user may be enabled to use the computing device to control a virtual camera within the VR environment and have various information regarding one or more aspects of the virtual camera displayed in the view of the VR environment presented to the user.

Abstract: The subject matter of this specification can be embodied in, among other things, a method that includes generating intermediate values from an evaluation of one or more static expressions within shader programming code that is configured to modify an appearance of an image, compressing the intermediate values based on a determination of which intermediate values are duplicative, and storing the compressed intermediate values in a buffer accessible to an image rendering application.

Abstract: In one general aspect, a method is described. The method includes generating a positional relationship between one or more support structures having at least one motion capture mark and at least one virtual structure corresponding to geometry of an object to be tracked and positioning the support structures on the object to be tracked. The support structures has sufficient rigidity that, if there are multiple marks, the marks on each support structure maintain substantially fixed distances from each other in response to movement by the object. The method also includes determining an effective quantity of ray traces between one or more camera views and one or more marks on the support structures, and estimating an orientation of the virtual structure by aligning the determined effective quantity of ray traces with a known configuration of marks on the support structures.

Abstract: Systems and techniques are provided for switching between different modes of a media content item. A media content item may include a movie that has different modes, such as a cinematic mode and an interactive mode. For example, a movie may be presented in a cinematic mode that does not allow certain user interactions with the movie. The movie may be switched to an interactive mode during any point of the movie, allowing a viewer to interact with various aspects of the movie. The movie may be displayed using different formats and resolutions depending on which mode the movie is being presented.

Abstract: A system includes a visual data collector for collecting visual information from an image of one or more features of an object. The system also includes a physical data collector for collecting sensor information provided by at one or more sensors attached to the object. The system also includes a computer system that includes a motion data combiner for combining the visual information the sensor information. The motion data combiner is configured to determine the position of a representation of one or more of the feature in a virtual representation of the object from the combined visual information and sensor information. Various types of virtual representations may be provided from the combined information, for example, one or more poses (e.g., position and orientation) of the object may be represented.

Abstract: Methods and systems efficiently apply known distortion, such as of a camera and lens, to source image data to produce data of an output image with the distortion. In an embodiment, an output image field is segmented into regions so that on each segment the distortion function is approximately linear, and segmentation data is stored in a quadtree. The distortion function is applied to the segmented image field to produce a segmented rendered distortion image (SRDI) and a corresponding look-up table. To distort a source image, a location in the output image field is selected, and the uniquely colored segment at the same location in the SRDI is found. The look-up table provides the local linear inverse of the distortion function, which is applied to determine from where in the source image to take image texture data for the distorted output image.

Abstract: A computer-implemented method for adjusting stereo images includes receiving a video sequence associated with a recorded setting of a stereoscopic camera, the video sequence comprising pairs of left and right images. The method includes processing the pairs of left and right images to reduce influence of a discrepancy between the recorded setting and an actual configuration of the stereoscopic camera.

Abstract: The subject matter of this specification can be embodied in, among other things, a method that includes determining a transform of a portion of constituent components of a source shape. The transform includes one or more states for the portion of constituent components of the source shape. The method also includes accessing a mapping function that associates the one or more states with one or more controls for a target shape, where the one or more controls configured for access by a user for manipulating constituent components of the target shape. The method includes outputting a transform for the target shape based on the one or more controls associated with the transformed constituent components of the source shape.

Abstract: A first input is received in an animation creation application having a user interface showing a virtual-space area for first objects, and a timeline area for second objects representing events. To generate the first input, a user presses and holds an input control while a cursor is over one of the first or second objects. It is determined whether the input control is held for at least a predefined duration. If so, a copy of the object is assigned to the cursor, and the copy is subsequently pasted at another location in the user interface upon the input control ceasing to be held after the cursor is moved to the other location. If the input control does not remain held for at least the predefined duration the copy of the object is not assigned to the cursor.

Abstract: Systems and techniques are provided for switching between different modes of a media content item. A media content item may include a movie that has different modes, such as a cinematic mode and an interactive mode. For example, a movie may be presented in a cinematic mode that does not allow certain user interactions with the movie. The movie may be switched to an interactive mode during any point of the movie, allowing a viewer to interact with various aspects of the movie. The movie may be displayed using different formats and resolutions depending on which mode the movie is being presented.

Abstract: Among other aspects, on computer-implemented method includes: receiving at least one command in a computer system from a handheld device; positioning a virtual camera and controlling a virtual scene according to the command; and in response to the command, generating an output to the handheld device for displaying a view of the virtual scene as controlled on a display of the handheld device, the view captured by the virtual camera as positioned.

Abstract: Techniques for facial performance capture using an adaptive model are provided herein. For example, a computer-implemented method may include obtaining a three-dimensional scan of a subject and a generating customized digital model including a set of blendshapes using the three-dimensional scan, each of one or more blendshapes of the set of blendshapes representing at least a portion of a characteristic of the subject. The method may further include receiving input data of the subject, the input data including video data and depth data, tracking body deformations of the subject by fitting the input data using one or more of the blendshapes of the set, and fitting a refined linear model onto the input data using one or more adaptive principal component analysis shapes.

Abstract: An apparatus is disclosed which may serve as a target for calibrating a camera. The apparatus comprises one or more planar surfaces. The apparatus includes at least one fiducial marking on a planar surface. The set of all planar markings on the apparatus are distinguishable.

Abstract: An apparatus is disclosed which may serve as a target for calibrating a camera. The apparatus includes a hollow body having an interior surface, an exterior surface and at least one window formed in the hollow body through which the interior surface is visible. A plurality of distinguishable fiducial markers are arranged in a predetermined pattern along the interior and exterior surface of the hollow body. The fiducial markers can be distinguishable such that, for any given image of the target captured by a camera, there is only one position, rotation, distortion value, and focal length for the camera of interest.

Abstract: A method may include receiving a plurality of objects from a 3-D virtual scene. The plurality of objects may be arranged in a hierarchy. The method may also include generating a plurality of identifiers for the plurality of objects. The plurality of identifiers may include a first identifier for a first object in the plurality of objects, and the identifier may be generated based on a position of the first object in the hierarchy. The method may additionally include performing a rendering operation on the plurality of objects to generate a deep image. The deep image may include a plurality of samples that correspond to the first object. The method may further include propagating the plurality of identifiers through the rendering operation such that each of the plurality of samples in the deep image that correspond to the first object are associated with the identifier.

Abstract: Among other disclosed subject matter, a system includes a first camera generating a live image of a scene, the first camera configured for being placed in a plurality of locations by robotic motion. The system includes a handheld device that includes a display device for continuously presenting the live image, wherein movement of the handheld device causes the handheld device to generate an output that controls the robotic motion.

Abstract: Methods are disclosed for the computer generation of data for images that include hair, fur, or other strand-like material. A volume for the hair is specified, having a plurality of surfaces. A fluid flow simulation is performed within the volume, with a first surface of the volume being a source area through which fluid is simulated to enter the volume, and a second surface being an exit surface through which fluid is simulated as exiting the volume. The fluid flow simulation may be used to produce fluid flow lines, such as from a velocity vector field for the fluid. Fluid flow lines are selected, and image data of hairs that follow the fluid flow lines are generated. Other embodiments include generating animation sequences by generating images wherein the volume and surfaces vary between frames.

Abstract: A movie set can include light sources each producing a light corresponding to a light channel, at least one high-frame rate camera, and a controller connected to the light sources and camera to synchronize the camera and light sources. The number of light channels can be proportional to the frame rate. For example, if the filming frame rate is 120 frames per second (fps) and the playback frame rate is 24 fps, then 5 light channels can be used. In this example, for every one playback frame, 5 frames were filmed by the high frame rate camera. The controller modulates the light channels such that each of the 5 frames has different lighting characteristics. In post-production, contributions from each of the light channels can be included or excluded from the final frame. An optical flow algorithm can be used to stitch together frames with different dynamic light characteristics.

Abstract: A method of applying a post-render motion blur to an object may include receiving a first image of the object. The first image need not be motion blurred, and the first image may include a first pixel and rendered color information for the first pixel. The method may also include receiving a second image of the object. The second image may be motion blurred, and the second image may include a second pixel and a location of the second pixel before the second image was motion blurred. Areas that are occluded in the second image may be identified and colored using a third image rendering only those areas. Unoccluded areas of the second image may be colored using information from the first image.

Abstract: In one aspect, a computer implemented method of motion capture, the method includes tracking the motion of a dynamic object bearing a pattern configured such that a first portion of the patterns is tracked at a first resolution and a second portion of the pattern is tracked at a second resolution. The method further includes causing data representing the motion to be stored to a computer readable medium.

Abstract: A system includes a computer system capable of representing one or more animated characters. The computer system includes a blendshape manager that combines multiple blendshapes to produce the animated character. The computer system also includes an expression manager to respectively adjust one or more control parameters associated with each of the plurality of blendshapes for adjusting an expression of the animated character. The computer system also includes a corrective element manager that applies one or more corrective elements to the combined blendshapes based upon at least one of the control parameters. The one or more applied corrective elements are adjustable based upon one or more of the control parameters absent the introduction of one or more additional control parameters.

Abstract: A system includes a visual data collector for collecting visual information from an image of one or more features of an object. The system also includes a physical data collector for collecting sensor information provided by at one or more sensors attached to the object. The system also includes a computer system that includes a motion data combiner for combining the visual information the sensor information. The motion data combiner is configured to determine the position of a representation of one or more of the feature in a virtual representation of the object from the combined visual information and sensor information. Various types of virtual representations may be provided from the combined information, for example, one or more poses (e.g., position and orientation) of the object may be represented.

Abstract: A system includes a computing device that includes a memory configured to store instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes defining a first portion of a display as a workspace for a graphical representation of a portion of a network of processing operations. The workspace provides a navigable view of the graphical representation for editing the displayed portion of the network. The method also includes defining at least a second portion of the display, different from first portion, for a compressed view of a portion of the network of processing operations external to the first portion of the display. The method also includes simultaneously presenting the first portion and the second portion on the display.

Abstract: A method may include displaying, on one or more display devices in a virtual-reality environment, a visual representation of a 3-D virtual scene from the perspective of a subject location in the virtual-reality environment. The method may also include displaying, on the one or more display devices, a chroma-key background with the visual representation. The method may further include recording, using a camera, an image of the subject in the virtual-reality environment against the chroma-key background.

Abstract: A method may include presenting a scene from linear content on one or more display devices in an immersive environment, and receiving, from a user within the immersive environment, input to change an aspect of the scene. The method may also include accessing 3-D virtual scene information previously used to render the scene, and changing the 3-D virtual scene information according to the changed aspect of the scene. The method may additionally include rending the 3-D virtual scene to incorporate the changed aspect, and presenting the rendered scene in real time in the immersive user environment.

Abstract: A system includes a computing device that includes a memory configured to store instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes producing an animation model from one or more representations of an object provided from a deformable likeness of the object. The one or more representations are based upon position information from a collection of images of the object captured by at least one camera. The method also includes refining the animation model to produce representations that substantially match the one or more representations provided by the deformable likeness of the object. Refining the animation model is based upon the position information from the collection of images of the object and one or more constraints.

Abstract: A system includes a computing device that includes a memory for storing instructions. The computing device also includes a processor configured to execute the instructions to perform a method that includes combining, in a nonlinear manner, a first set of vertex displacements that represent the difference between a first animated expression and a neutral animated expression with a second set of vertex displacements that represent the difference between a second animated expression and the neutral animated expression. The number of vertices associated with the first set of vertex displacements of the first animated expression is equivalent to the number of vertices associated with the second set of vertex displacements of the second animated expression.