Abstract: A technique for reconstructing a three-dimensional scene from monocular video adaptively allocates an explicit sparse-dense voxel grid with dense voxel blocks around surfaces in the scene and sparse voxel blocks further from the surfaces. In contrast to conventional systems, the two-level voxel grid can be efficiently queried and sampled. In an embodiment, the scene surface geometry is represented as a signed distance field (SDF). Representation of the scene surface geometry can be extended to multi-modal data such as semantic labels and color. Because properties stored in the sparse-dense voxel grid structure are differentiable, the scene surface geometry can be optimized via differentiable volume rendering.

Abstract: Systems and methods are described for rendering complex surfaces or geometry. In at least one embodiment, neural signed distance functions (SDFs) can be used that efficiently capture multiple levels of detail (LODs), and that can be used to reconstruct multi-dimensional geometry or surfaces with high image quality. An example architecture can represent complex shapes in a compressed format with high visual fidelity, and can generalize across different geometries from a single learned example. Extremely small multi-layer perceptrons (MHLPs) can be used with an octree-based feature representation for the learned neural SDFs.

Abstract: One embodiment of a method for controlling a robot includes generating a representation of spatial occupancy within an environment based on a plurality of red, green, blue (RGB) images of the environment, determining one or more actions for the robot based on the representation of spatial occupancy and a goal, and causing the robot to perform at least a portion of a movement based on the one or more actions.

Abstract: Systems and methods are described for rendering complex surfaces or geometry. In at least one embodiment, neural signed distance functions (SDFs) can be used that efficiently capture multiple levels of detail (LODs), and that can be used to reconstruct multi-dimensional geometry or surfaces with high image quality. An example architecture can represent complex shapes in a compressed format with high visual fidelity, and can generalize across different geometries from a single learned example. Extremely small multi-layer perceptrons (MLPs) can be used with an octree-based feature representation for the learned neural SDFs.

Abstract: A manifold voxel mesh or surface mesh is manufacturable by carving a single block of material and a non-manifold mesh is not manufacturable. Conventional techniques for constructing or extracting a surface mesh from an input point cloud often produce a non-manifold voxel mesh. Similarly, extracting a surface mesh from a voxel mesh that includes non-manifold geometry produces a surface mesh that includes non-manifold geometry. To ensure that the surface mesh includes only manifold geometry, locations of the non-manifold geometry in the voxel mesh are detected and converted into manifold geometry. The result is a manifold voxel mesh from which a manifold surface mesh of the object may be extracted.

Abstract: In various examples, methods and systems are provided for estimating depth values for images (e.g., from a monocular sequence). Disclosed approaches may define a search space of potential pixel matches between two images using one or more depth hypothesis planes based at least on a camera pose associated with one or more cameras used to generate the images. A machine learning model(s) may use this search space to predict likelihoods of correspondence between one or more pixels in the images. The predicted likelihoods may be used to compute depth values for one or more of the images. The predicted depth values may be transmitted and used by a machine to perform one or more operations.

Abstract: A manifold voxel mesh or surface mesh is manufacturable by carving a single block of material and a non-manifold mesh is not manufacturable. Conventional techniques for constructing or extracting a surface mesh from an input point cloud often produce a non-manifold voxel mesh. Similarly, extracting a surface mesh from a voxel mesh that includes non-manifold geometry produces a surface mesh that includes non-manifold geometry. To ensure that the surface mesh includes only manifold geometry, locations of the non-manifold geometry in the voxel mesh are detected and converted into manifold geometry. The result is a manifold voxel mesh from which a manifold surface mesh of the object may be extracted.

Abstract: Systems and methods are described for rendering complex surfaces or geometry. In at least one embodiment, neural signed distance functions (SDFs) can be used that efficiently capture multiple levels of detail (LODs), and that can be used to reconstruct multi-dimensional geometry or surfaces with high image quality. An example architecture can represent complex shapes in a compressed format with high visual fidelity, and can generalize across different geometries from a single learned example. Extremely small multi-layer perceptrons (MLPs) can be used with an octree-based feature representation for the learned neural SDFs.

Abstract: Systems and methods are described for rendering complex surfaces or geometry. In at least one embodiment, neural signed distance functions (SDFs) can be used that efficiently capture multiple levels of detail (LODs), and that can be used to reconstruct multi-dimensional geometry or surfaces with high image quality. An example architecture can represent complex shapes in a compressed format with high visual fidelity, and can generalize across different geometries from a single learned example. Extremely small multi-layer perceptrons (MLPs) can be used with an octree-based feature representation for the learned neural SDFs.

Abstract: Systems and methods are described for rendering complex surfaces or geometry. In at least one embodiment, neural signed distance functions (SDFs) can be used that efficiently capture multiple levels of detail (LODs), and that can be used to reconstruct multi-dimensional geometry or surfaces with high image quality. An example architecture can represent complex shapes in a compressed format with high visual fidelity, and can generalize across different geometries from a single learned example. Extremely small multi-layer perceptrons (MLPs) can be used with an octree-based feature representation for the learned neural SDFs.

Abstract: Apparatuses, systems, and techniques are presented to determine distance for one or more objects. In at least one embodiment, a disparity network is trained to determine distance data from input stereoscopic images using a loss function that includes at least one of a gradient loss term and an occlusion loss term.

Abstract: Stereo matching generates a disparity map indicating pixels offsets between matched points in a stereo image pair. A neural network may be used to generate disparity maps in real time by matching image features in stereo images using only 2D convolutions. The proposed method is faster than 3D convolution-based methods, with only a slight accuracy loss and higher generalization capability. A 3D efficient cost aggregation volume is generated by combining cost maps for each disparity level. Different disparity levels correspond to different amounts of shift between pixels in the left and right image pair. In general, each disparity level is inversely proportional to a different distance from the viewpoint.

Abstract: The systems and methods discussed herein implement a volumetric approach to point cloud representation, compression, decompression, communication, or any suitable combination thereof. The volumetric approach can be used for both geometry and attribute compression and decompression, and both geometry and attributes can be represented by volumetric functions. To create a compressed representation of the geometry or attributes of a point cloud, a suitable set of volumetric functions are transformed, quantized, and entropy-coded. When decoded, the volumetric functions are sufficient to reconstruct the corresponding geometry or attributes of the point cloud.

Abstract: The systems and methods discussed herein implement a volumetric approach to point cloud representation, compression, decompression, communication, or any suitable combination thereof. The volumetric approach can be used for both geometry and attribute compression and decompression, and both geometry and attributes can be represented by volumetric functions. To create a compressed representation of the geometry or attributes of a point cloud, a suitable set of volumetric functions are transformed, quantized, and entropy-coded. When decoded, the volumetric functions are sufficient to reconstruct the corresponding geometry or attributes of the point cloud.

Abstract: The systems and methods discussed herein implement a volumetric approach to point cloud representation, compression, decompression, communication, or any suitable combination thereof The volumetric approach can be used for both geometry and attribute compression and decompression, and both geometry and attributes can be represented by volumetric functions. To create a compressed representation of the geometry or attributes of a point cloud, a suitable set of volumetric functions are transformed, quantized, and entropy-coded. When decoded, the volumetric functions are sufficient to reconstruct the corresponding geometry or attributes of the point cloud.

Abstract: A machine can be specially configured to generate, compress, decompress, store, access, communicate, or otherwise process a special data structure that represents a three-dimensional surface of an object. The data structure can be or include a pruned sparse voxel octree in which each node in the octree corresponds to a different block of the octree, and children of the node in the octree correspond to the smaller blocks that subdivide the block. Moreover, each block occupied by the surface or a portion thereof can define its enclosed surface or portion thereof explicitly or implicitly.

Abstract: Views of parametric surfaces are rendered. A set of parametric surface patches representing a parametric surface being rendered is projected onto a scene, producing a set of view-projected surface patches. Each view-projected surface patch is identified for either culling, subdivision or rendering. For patches which are identified for subdivision, the patches are recursively subdivided into sub-patches until for each sub-patch a prescribed screen-space projection of the sub-patch satisfies a prescribed screen-space error metric. Once the error metric is satisfied, the sub-patch is identified for rendering. Patches and sub-patches which have been identified for rendering are prepared and rendered.

Abstract: Views of parametric surfaces are rendered. A set of parametric surface patches representing a parametric surface being rendered is projected onto a scene, producing a set of view-projected surface patches. Each view-projected surface patch is identified for either culling, subdivision or rendering. For patches which are identified for subdivision, the patches are recursively subdivided into sub-patches until for each sub-patch a prescribed screen-space projection of the sub-patch satisfies a prescribed screen-space error metric. Once the error metric is satisfied, the sub-patch is identified for rendering. Patches and sub-patches which have been identified for rendering are prepared and rendered.

Abstract: Rendering an outline font. Rendering an outline font by adding Bezier control points to further define a contour of an outline font and applying an in or out test to determine if a pixel falls within the contour of an outline font.

Abstract: Surfaces defined by a Bézier tetrahedron, and in particular quadric surfaces, are rendered on programmable graphics hardware. Pixels are rendered through triangular sides of the tetrahedra and locations on the shapes, as well as surface normals for lighting evaluations, are computed using pixel shader computations. Additionally, vertex shaders are used to aid interpolation over a small number of values as input to the pixel shaders. Through this, rendering of the surfaces is performed independently of viewing resolution, allowing for advanced level-of-detail management. By individually rendering tetrahedrally-defined surfaces which together form complex shapes, the complex shapes can be rendered in their entirety.