Abstract: A performance capture system is provided to detect one or more active marker units in a live action scene. Active marker units emanate at least one wavelength of light that is captured by the performance capture system and used to detect the active markers in the scene. The system detects the presence of the light as a light patch in a capture frames and determines if the light patch represents light from an active marker unit. In some implementations, various active markers in a scene may emanate different wavelengths of light. For example, wavelengths of light from multi-emitting active marker units may be changed due to various conditions in the scene.

Abstract: Implementations facilitate the calibration of cameras in a live action scene. In some implementations, a system includes a base, where the base is configured to removably couple to a camera in the live action scene. The system further includes a plurality of extensions coupled to the base, where the plurality of extensions extends outwardly away from the base. The system further includes a plurality of reference points identifiable by a sensor, where the reference points are coupled to ends of corresponding extensions of the plurality of extensions, and where the plurality of extensions positions the plurality of reference points in a three-dimensional arrangement. Distances between pairs of reference points are based on one or more aspects of the extensions, and the distances between the pairs of reference points are used to compute reference point data generated from a performance capture system.

Abstract: Embodiments allow camera effects, such as imaging noise, to be included in a generation of a synthetic data set for use in training an artificial intelligence model to produce an image depth map. The image depth map can then be employed to assist in compositing live action images from an image capture device with computer generated images in real-time or near real-time. The two types of images (live action and computer generated) are composited accurately by using a depth map. In an embodiment, the depth map includes a “depth value” for each pixel in the live action image. In an embodiment, steps of one or more of feature extraction, matching, filtering or refinement can be implemented, at least in part, with an artificial intelligence (AI) computing approach using a deep neural network with training.

Abstract: Compositing is provided in which visual elements from different sources, including live action objects and computer graphic (CG) merged in a constant feed. Representative output images are produced during a live action shoot. The compositing system uses supplementary data, such as depth data of the live action objects for integration with CG items and light marker detection data for device calibration and performance capture. Varying capture times (e.g., exposure times) and processing times are tracked to align with corresponding incoming images and data.

Abstract: Embodiments allow live action images from an image capture device to be composited with computer generated images in real-time or near real-time. The two types of images (live action and computer generated) are composited accurately by using a depth map. In an embodiment, the depth map includes a “depth value” for each pixel in the live action image. In an embodiment, steps of one or more of feature extraction, matching, filtering or refinement can be implemented, at least in part, with an artificial intelligence (AI) computing approach using a deep neural network with training. A combination of computer-generated (“synthetic”) and live-action (“recorded”) training data is created and used to train the network so that it can improve the accuracy or usefulness of a depth map so that compositing can be improved.

Abstract: The present description relates to light patterns used in a live action scene of a visual production to encode information associated with objects in the scene, such as movement and position of the objects. A data capture system includes active markers that emit light of a particular wavelength in predefined strobing patterns. In some implementations, the active markers are instructed to emit an assigned signature pattern of light through a signal controller sending signals to a control unit. Various components are synchronized such that pulsing of light corresponds to time slices and particular frames captured by the performance capture system. The data representing the pattern is embedded in illuminated and blank frames. Frames showing the light pattern are analyzed to extract information about the active markers, such as identification of the active markers and objects to which they are attached.

Abstract: A performance capture system is provided to detect one or more active marker units in a live action scene. Active marker units emanate at least one wavelength of light that is captured by the performance capture system and used to detect the active markers in the scene. The system detects the presence of the light as a light patch in a capture frames and determines if the light patch represents light from an active marker unit. In some implementations, various active markers in a scene may emanate different wavelengths of light. For example, wavelengths of light from multi-emitting active marker units may be changed due to various conditions in the scene.

Abstract: An active marker apparatus is provided for securely affixing active markers to a wearable article of an object in a performance capture system. The active marker light components coupled to a strand, are inserted into a receptacle to position the active marker light components onto the wearable article. A gap is provided in a chamber between the active marker light component and an interior surface of a protrusion portion of the receptacle. At least a section of the protrusion portion permits light emitted from the active marker light component into the chamber, to diffuse in a manner that allows the light to be easily detected by a camera in the live action scene. Each active marker light component is locked into place in a respective receptacle by one or more channels that receive the strand. A cap fitting may further assist in securing the receptacle to the wearable article and optionally aid in visual detection of the receptacle.

Abstract: The present description relates to light patterns used in a live action scene of a visual production to encode information associated with objects in the scene, such as movement and position of the objects. A data capture system is enabled to differentiate between various groups of active markers attached to the objects in the scene. The groups of active markers emit light of a particular wavelength in strobing patterns predefined for the various groups. In some implementations, the groups are instructed to emit its assigned signature pattern of light through a signal controller transmitting an initial key signature predefined for the group, followed by pattern signals to a control unit. The data representing the pattern is captured in illuminated and blank frames. Frames showing the light pattern are analyzed to extract information about the groups of active markers, such as distinguishing the groups and identifying the objects to which they are attached.

Abstract: A sealed active marker apparatus of a performance capture system is described to provide protective housing for active marker light components coupled to a strand and attached via a receptacle, to an object, such as via a wearable article, in a live action scene. The receptacle includes a protrusion portion that permits at least one particular wavelength range of light emitted from the enclosed active marker light component, to diffuse in a manner that enables easy detection by a sensor device. A base portion interlocks with a bottom plate of the receptacle to secure the strand within one or more channels. A sealant material coating portions of the apparatus promotes an insulating environment for the active marker light component.

Abstract: Installation of active markers light components to a wearable article of an object in a performance capture system is provided. The active marker light components are inserted into a receptacle to position the active marker light components onto the wearable article. The active marker light component and a strand to which the active marker light component is coupled, are inserted into the receptacle. The strand may be engaged into a channel in the receptacle. A top portion of the receptacle is introduced through an opening in the wearable article from the inside of the wearable article to the outside and a base portion catches an inner side of the wearable article. A fitting may be rotatably secured around the top portion at the outer side of the wearable article, to press against the base portion through the wearable article. The active markers light components may be removed from the wearable article by rotating the fitting in a counter-direction and guiding the top portion back through the opening.

Abstract: Embodiments allow live action images from an image capture device to be composited with computer generated images in real-time or near real-time. The two types of images (live action and computer generated) are composited accurately by using a depth map. In an embodiment, the depth map includes a “depth value” for each pixel in the live action image. In an embodiment, steps of one or more of feature extraction, matching, filtering or refinement can be implemented, at least in part, with an artificial intelligence (AI) computing approach using a deep neural network with training. A combination of computer-generated (“synthetic”) and live-action (“recorded”) training data is created and used to train the network so that it can improve the accuracy or usefulness of a depth map so that compositing can be improved.

Abstract: Embodiments allow live action images from an image capture device to be composited with computer generated images in real-time or near real-time. The two types of images (live action and computer generated) are composited accurately by using a depth map. In an embodiment, the depth map includes a “depth value” for each pixel in the live action image. In an embodiment, steps of one or more of feature extraction, matching, filtering or refinement can be implemented, at least in part, with an artificial intelligence (AI) computing approach using a deep neural network with training. A combination of computer-generated (“synthetic”) and live-action (“recorded”) training data is created and used to train the network so that it can improve the accuracy or usefulness of a depth map so that compositing can be improved.

Abstract: The present description relates to light patterns used in a live action scene of a visual production to encode information associated with objects in the scene, such as movement and position of the objects. A data capture system includes active markers that emit light of a particular wavelength in predefined strobing patterns. In some implementations, the active markers are instructed to emit an assigned signature pattern of light through a signal controller sending signals to a control unit. Various components are synchronized such that pulsing of light corresponds to time slices and particular frames captured by the performance capture system. The data representing the pattern is embedded in illuminated and blank frames. Frames showing the light pattern are analyzed to extract information about the active markers, such as identification of the active markers and objects to which they are attached.

Abstract: A performance capture system is provided to detect one or more active marker units in a live action scene. Active marker units emanate at least one wavelength of light that is captured by the performance capture system and used to detect the active markers in the scene. The system detects the presence of the light as a light patch in a capture frames and determines if the light patch represents light from an active marker unit. In some implementations, various active markers in a scene may emanate different wavelengths of light. For example, wavelengths of light from multi-emitting active marker units may be changed due to various conditions in the scene.