Abstract: A system and associated methods perform a dynamic differentiated capture of a three-dimensional (3D) asset for non-uniform splat generation. The system partitions the 3D space of the 3D asset into multiple cells. The system selects a set of cells that encompass a volume of the 3D space in which one or more of the 3D asset primitives are defined. The system determines an amount of visual detail associated with the primitives in each selected cell, and positions virtual cameras in the 3D space to capture a first set of primitives that are associated with a first amount of visual detail from more perspectives than a second set of primitives that are associated with a second amount of visual detail that is less than the first amount of visual detail. The system generates images that non-uniformly capture the 3D asset from the positions of the virtual cameras.

Abstract: A system and associated methods optimize the streaming of three-dimensional (3D) content based on a pre-culled segmentation of the 3D content. The pre-culled segmentation involves performing server-side occlusion culling or preprocessing of the 3D content so that only the visible primitives within a requested field-of-view are streamed to client devices rather than all primitives within the requested field-of-view. The system segments the 3D content primitives to different tiles and filters each tile to differentiate a first subset of visible primitives in each tile from a second subset of non-visible primitives in each tile. The system receives a request for the 3D content, retrieves the one or more tiles with primitives positioned within the requested field-of-view, and streams the first subset of visible primitives from each tile of the one or more tiles in response to the request without the second subset of non-visible primitives.

Abstract: A splat generation system and associated methods perform splatting based on a controlled sampling of a three-dimensional (3D) asset. The controlled sampling ensures minimal coverage the 3D asset primitives and provides enhanced coverage for primitives that have greater detail or variation. The system projects rays in different directions from a set of the primitives towards a bounding volume, and detects points at which some of the projected rays intersect the bounding volume. The system defines virtual cameras in the 3D space of the 3D asset based on a set of the intersection points, and obtains the minimal and/or enhanced coverage based on one or more views of the primitives captured in images taken by the virtual cameras. The system generates a set of splats from the one or more views in the captured images with an acceptable amount of loss and with less total data than the 3D asset.

Abstract: A system and associated methods perform smooth temporally segmented encoding of dynamic unstructured spatial data or dynamic three-dimensional (3D) content. The system receives the primitives that define visual changes to the dynamic 3D content across multiple frames. The system determines a first set of primitives that remain unchanged for at least N frames and a second set of primitives that remain unchanged for less than the N frames. The system generates a first data stream that encodes the first set of primitives with temporal values at a reduced frame rate, and generates a second data stream that encodes the second set of primitives without temporal values at the desired frame rate. The system streams the first data stream and the second data stream in response to a request for the dynamic 3D content.

Abstract: A real-time streaming system and associated methods are provided for initially presenting three-dimensional (3D) content at a low-fidelity first encoding format so that the initial visualization of the 3D content on the requesting device is presented with no or insignificant delay and for streaming prioritized 3D assets within the initial visualization at higher fidelities in different encoding formats that preserve the real-time responsiveness of the system. The system generates and streams a two-dimensional (2D) image for a first presentation of the 3D content at a first fidelity. The system increases a fidelity of a prioritized 3D asset while the 3D content does not change by streaming 3D primitives for the prioritized 3D asset in a second encoding format that increases the fidelity of the prioritized 3D asset in a second presentation of the 3D content from the first fidelity to a greater second fidelity.

Abstract: Disclosed is an encoding system and associated methods for generating a graph-integrated tree-based representation of data that provides for direct lateral traversals of nodes in a each layer of the tree-based representation. The encoding system organizes data from a dataset to a tree-based representation with multiple layers and multiple nodes in each layer. The encoding system detects the nodes in each layer, and defines a graph structure that links the nodes in each layer for direct lateral access. The encoding system searches the tree-based representation in response to a query for a particular subset of the data by performing a single downward traversal to a particular layer with individual nodes that satisfy part of the query, and by laterally traversing the nodes in the particular layer using the graph structure to directly access a second node in the particular layer from a first node in the particular layer.

Abstract: An optimization system reduces the data encoded within a point cloud for streaming and/or rendering of a lossy representation of the point cloud. The optimization system generates a first optimized splat for a first visual characteristic of the point cloud by replacing a first set of points having a first common value for the first visual characteristic with a first replacement primitive, and generates a second optimized splat for a second visual characteristic of the point cloud by replacing a second set of points having a second common value for the second visual characteristic with a second replacement primitive. The optimization system provides the first optimized splat and the second optimized splat instead of the original points of the point cloud in response to a request to access the point cloud.

Abstract: A streaming system and associated methods provide adaptive streaming of point cloud data for out-of-order presentation of important visual features before less important visual features. The adaptive streaming includes retrieving the points for a requested point cloud, differentiating different sets of points that represent different features in the point cloud, prioritizing each set of points based on a feature that is represented by that set of points, and streaming the different sets of points across a data network to a client device in an order that is determined from the prioritization. Moreover, the adaptive streaming may dynamically select a different subset of points from each set of points and different subset of point data to stream with each selected subset of points to visually convey important detail of each feature without all the corresponding points or data that make up that feature in the point cloud.

Abstract: A splat generation system and associated methods generate a non-uniform splat representation of a three-dimensional (3D) asset based on a loss weighted sampling of reference images that capture the 3D asset from different viewpoints. The system performs a first training iteration to define a different set of splats to reconstruct the field-of-view captured by a different one of the reference images. The system associates an amount of loss to each reference image based on an amount of variation by which a set of splats trained on that reference image reconstructs the field-of-view of that reference image. The system selects a next image to train on based on the amount of loss associated with each of the reference images, and retrains the set of splats representing the field-of-view of the selected next image by adjusting one or more of those splats to increase the reconstructed field-of-view accuracy.

Abstract: Disclosed is Light Detection and Ranging (“LiDAR”)-based camera metering, exposure adjustment, and image postprocessing. The LiDAR-based exposure adjustment may include emitting a laser from an imaging device, obtaining one or more measurements based on the laser reflecting off one or more objects in a scene and returning to the imaging device, adjusting exposure settings of the imaging device based on the one or more measurements, and capturing an image of the scene using the exposure settings. The LiDAR-based image postprocessing may include receiving an image of a scene and measurements or outputs from a LiDAR scan of the scene, and performing different adjustments to color values, contrast, brightness, saturation, levels, and other visual characteristics of different sets of pixels in the image based on different distance, material property, and/or other measurements obtained by the LiDAR for objects represented by the different sets of pixels.

Abstract: A system and associated methods perform smooth temporally segmented encoding of dynamic unstructured spatial data or dynamic three-dimensional (3D) content. The system receives the primitives that define visual changes to the dynamic 3D content across multiple frames. The system determines a first set of primitives that remain unchanged for at least N frames and a second set of primitives that remain unchanged for less than the N frames. The system generates a first data stream that encodes the first set of primitives with temporal values at a reduced frame rate, and generates a second data stream that encodes the second set of primitives without temporal values at the desired frame rate. The system streams the first data stream and the second data stream in response to a request for the dynamic 3D content.