Abstract: Techniques and systems are provided for image processing. For instance, a process can include generating a set of surface representation values for one or more surfaces visible in one or more first images using a first three dimensional scene reconstruction (3DR) algorithm; preprocessing the set of surface representation values to generate preprocessing information for a second 3DR algorithm; generating, by the second 3DR algorithm, a refined set of surface representation values based on the set of surface representation values and the preprocessing information; and outputting the refined set of surface representation values.

Abstract: Disclosed are systems and techniques for three-dimensional reconstruction (3DR) of a scene. For example, a computing device can extract two-dimensional (2D) features from image frames of a scene. Each of the image frames includes a respective view of the scene. The computing device can unproject the 2D features from a 2D space onto a three-dimensional (3D) space to obtain 3D features. The computing device can determine weights for the 3D features based on geometric information and visual similarity information for the image frames. Each of the weights is associated with a respective feature 3D of the 3D features. The computing device can determine voxel features for the image frames based on the weights for the 3D features. The computing device can further form the 3DR of the scene based on the voxel features.

Abstract: A processor-implemented method includes receiving neural network weights from an artificial neural network. The method also includes jointly embedding, by an encoder neural network, a first watermark into the neural network weights of the artificial neural network and a second watermark into an output of the artificial neural network to generate watermarked weights and watermarked output. The method further includes transmitting the watermarked weights and watermarked output. A processor-implemented method by a decoder neural network includes receiving watermarked neural network weights for an artificial neural network. The method also includes receiving a first private key. The method further includes decoding the watermarked neural network weights based on the first private key to obtain a first watermark.

Abstract: A device includes a memory configured to store a first model and a second model. The first model is configured to perform inference based on a first set of parameters corresponding to a first context. The device includes one or more processors configured to process, using the second model, the first set of parameters and input corresponding to a second context to generate an output of the second model. The one or more processors are also configured to update the first model to perform inference using an updated set of parameters based on the output of the second model.

Abstract: Aspects of the disclosure are directed to reordering a plurality of input block voxel indices in a cache memory. In accordance with one aspect, an apparatus including a create block configured to receive the plurality of input block voxel indices and configured to generate a reordered list based on the plurality of input block voxel indices; and an integrate block coupled to the create block, the integrate block configured to use the reordered list to deliver integrate depth data for generating a plurality of output block voxel indices. In accordance with one aspect, a method including reordering the plurality of input block voxel indices into a plurality of output block voxel indices using a separated set of input block voxel indices; and accessing the plurality of output block voxel indices to provide an augmented cache memory access.

Abstract: This disclosure provides systems, devices, apparatus, and methods, including computer programs encoded on storage media, for content adaptive 3D reconstruction. A graphics processor may compute a curvature of content in a block volume that represents a 3D scene, where the block volume includes a set of voxels. The graphics processor may select a number of the set of voxels in the block volume based on the computed curvature of the content. The graphics processor may output an indication of the selected number of the set of voxels in the block volume.

Abstract: Disclosed are systems and techniques for image processing. For example, a computing device can project a sample location (of a plurality of sample locations) of a block of a scene onto depth frames to determine pixel values for the sample location. Each of the depth frames corresponds to a pose and includes a depth prediction value corresponding to the sample location. The computing device can read the depth prediction values of the depth frames at the pixel values for the sample location on each depth frame of the depth frames. The computing device can obtain 3D points of the depth prediction values in a three-dimensional (3D) space. The computing device can then generate weighted depth prediction values by assigning a weight to each depth prediction value of the depth prediction values. The computing device can update the sample location based on the weighted depth prediction values.

Abstract: Methods, systems, and apparatuses are provided to automatically detect objects within images. For example, an image capture device may capture an image, and may apply a trained neural network to the image to generate an object value and a class value for each of a plurality of portions of the image. Further, the image capture device may determine, for each of the plurality of image portions, a confidence value based on the object value and the class value corresponding to each image portion. The image capture device may also detect an object within at least one image portion based on the confidence values. Further, the image capture device may output a bounding box corresponding to the at least one image portion. The bounding box defines an area of the image that includes one or more objects.

Abstract: Methods, systems, and apparatuses are provided to automatically detect objects within images. For example, an image capture device may capture an image, and may apply a trained neural network to the image to generate an object value and a class value for each of a plurality of portions of the image. Further, the image capture device may determine, for each of the plurality of image portions, a confidence value based on the object value and the class value corresponding to each image portion. The image capture device may also detect an object within at least one image portion based on the confidence values. Further, the image capture device may output a bounding box corresponding to the at least one image portion. The bounding box defines an area of the image that includes one or more objects.

Abstract: Systems and techniques are described herein for enabling a multi-user extended reality (XR) experience. In one illustrative example, a user device can receive, associated with a user from a host device, a message comprising a prompt to join an XR room hosted by the host device. The user device can connect to the host device based on the message. The user device can can obtain images of a three-dimensional (3D) scene of a physical environment and can transmit the images to the host device. The user device can receive synthetic content from the host device. A virtual representation of the user can be localized with respect to the XR room based on features in the images matched with features of the 3D scene of the physical environment. The user device can render the synthetic content for the XR room based on a pose of the apparatus.

Abstract: Systems and techniques are described for performing scalable voxel block selection. For example, a computing device can determine a fixed block configuration based on a storage size limitation. The computing device can select a plurality of blocks of the scene based on the fixed block configuration. The computing device can convert indices of the plurality of blocks associated with the fixed block configuration to indices of a plurality of blocks associated with a particular block configuration (of a plurality of block configurations) that corresponds to a particular 3DR application. The particular block configuration is different from the fixed block configuration.

Abstract: Systems and techniques are provided for registering three-dimensional (3D) images to deformable models.

Abstract: Systems and techniques are provided for modeling three-dimensional (3D) meshes using images. An example method can include receiving, via a neural network system, an image of a target and metadata associated with the image and/or a device that captured the image; determining, based on the image and metadata, first 3D mesh parameters of a first 3D mesh of the target, the first 3D mesh parameters and first 3D mesh corresponding to a first reference frame associated with the image and/or the device; and determining, based on the first 3D mesh parameters, second 3D mesh parameters for a second 3D mesh of the target, the second 3D mesh parameters and second 3D mesh corresponding to a second reference frame, the second reference frame including a 3D coordinate system of a real-world scene where the target is located.

Abstract: Media processing systems and techniques are described. A media processing system receives image data that represents an environment captured by an image sensor. The media processing system receives an indication of an object in the environment that is represented in the image data. The media processing system divides the image data into regions, including a first region and a second region. The object is represented in one of the plurality of regions. The media processing system modifies the image data to obscure the first region without obscuring the second region based on the object being represented in the one of the plurality of regions. The media processing system outputs the image data after modifying the image data. In some examples, the object is depicted in the first region and not the second region. In some examples, the object is depicted in the second region and not the first region.

Abstract: Methods, systems, and apparatuses are provided to automatically detect objects within images. For example, an image capture device may capture an image, and may apply a trained neural network to the image to generate an object value and a class value for each of a plurality of portions of the image. Further, the image capture device may determine, for each of the plurality of image portions, a confidence value based on the object value and the class value corresponding to each image portion. The image capture device may also detect an object within at least one image portion based on the confidence values. Further, the image capture device may output a bounding box corresponding to the at least one image portion. The bounding box defines an area of the image that includes one or more objects.

Abstract: Systems and techniques are provided for modeling three-dimensional (3D) meshes using images. An example method can include receiving, via a neural network system, an image of a target and metadata associated with the image and/or a device that captured the image; determining, based on the image and metadata, first 3D mesh parameters of a first 3D mesh of the target, the first 3D mesh parameters and first 3D mesh corresponding to a first reference frame associated with the image and/or the device; and determining, based on the first 3D mesh parameters, second 3D mesh parameters for a second 3D mesh of the target, the second 3D mesh parameters and second 3D mesh corresponding to a second reference frame, the second reference frame including a 3D coordinate system of a real-world scene where the target is located.

Abstract: Systems and techniques are provided for modeling three-dimensional (3D) meshes using multi-view image data. An example method can include determining, based on a first image of a target, first 3D mesh parameters for the target corresponding to a first coordinate frame; determining, based on a second image of the target, second 3D mesh parameters for the target corresponding to a second coordinate frame; determining third 3D mesh parameters for the target in a third coordinate frame, the third 3D mesh parameters being based on the first and second 3D mesh parameters and relative rotation and translation parameters of image sensors that captured the first and second images; determining a loss associated with the third 3D mesh parameters, the loss being based on the first and second 3D mesh parameters and the relative rotation and translation parameters; determining 3D mesh parameters based on the loss and third 3D mesh parameters.

Abstract: Systems and techniques are provided for registering three-dimensional (3D) images to deformable models.

Abstract: A method performed by an electronic device is described. The method includes generating a depth map of a scene external to a vehicle. The method also includes performing first processing in a first direction of a depth map to determine a first non-obstacle estimation of the scene. The method also includes performing second processing in a second direction of the depth map to determine a second non-obstacle estimation of the scene. The method further includes combining the first non-obstacle estimation and the second non-obstacle estimation to determine a non-obstacle map of the scene. The combining includes combining comprises selectively using a first reliability map of the first processing and/or a second reliability map of the second processing The method additionally includes navigating the vehicle using the non-obstacle map.

Abstract: A classifier for detecting objects in images can be configured to receive features of an image from a feature extractor. The classifier can determine a feature window based on the received features, and allows access by each decision tree of the classifier to only a predetermined area of the feature window. Each decision tree of the classifier can compare a corresponding predetermined area of the feature window with one or more thresholds. The classifier can determine an object in the image based on the comparisons. In some examples, the classifier can determine objects in a feature window based on received features, where the received features are based on color information for an image.