Abstract: Certain aspects of the present disclosure provide techniques for training and using an instance segmentation neural network to detect instances of a target object in an image. An example method generally includes generating, through an instance segmentation neural network, a first mask output from a first mask generation branch of the network. The method further includes generating, through the instance segmentation neural network, a second mask output from a second, parallel, mask generation branch of the network. The second mask output is typically of a lower resolution than the first mask output. The method further includes combining the first mask output and second mask output to generate a combined mask output. Based on the combined mask output, an output of the instance segmentation neural network is generated. One or more actions are taken based on the generated output.

Abstract: Systems and techniques are provided for generating disparity information from two or more images. For example, a process can include obtaining first disparity information corresponding to a pair of images, the pair of images including a first image of a scene and a second image of the scene. The process can include obtaining confidence information associated with the first disparity information. The process can include processing, using a machine learning network, the first disparity information and the confidence information to generate second disparity information corresponding to the pair of images. The process can include combining, based on the confidence information, the first disparity information with the second disparity information to generate a refined disparity map corresponding to the pair of images.

Abstract: Systems and techniques are provided for processing image data. For example, a process can include receiving an input image comprising a padded image including a plurality of padding pixels located around a perimeter of the frame of image data. A plurality of overlapping image patches can be generated, each corresponding to a respective portion of the padded image and having a resolution. A channel-wise grouped input can be generated from the plurality of overlapping image patches and having the resolution, and includes a corresponding channel for each respective channel of each overlapping image patch of the plurality of overlapping image patches. An image processing output can be generated based on processing the channel-wise grouped input using a plurality of grouped convolutional layers of a tiled neural network, wherein at least one grouped convolutional layer of the plurality of grouped convolutional layers does not perform internal padding.

Abstract: Systems and techniques are provided for processing image data corresponding to a scene. A process can include generating a planar distance map including a planar distance value for each pixel of at least one image corresponding to the scene. Planar segmentation is performed based on the planar distance map, a normal map corresponding to the at least one image, and positional encoding information of the planar distance map. A triangular mesh fragment is initialized based on sampling points from each planar segment of a plurality of planar segments from the planar segmentation. Ray-triangle intersections are determined based on performing ray casting for a reconstructed planar mesh including a plurality of triangular mesh fragments each corresponding to a different image. A planar reconstruction and segmentation machine learning network is optimized for the scene, based on training the planar reconstruction and segmentation machine learning network using one or more loss functions.

Abstract: An embodiment method includes performing first convolutional filtering on a first tensor constructed using a current frame and reference frames (or digital world reference images) of the current frame in a video, to generate a first estimated image of the current frame having a higher resolution than an image of the current frame. The method also includes performing second convolutional filtering on a second tensor constructed using the first estimated image and estimated reference images of the reference frames, to generate a second estimated image of the current having a higher resolution than the image of the current frame. The estimated reference images of the reference frames are reconstructed high resolution images of the reference images.

Abstract: Certain aspects of the present disclosure provide techniques for improved machine learning using gradient pruning, comprising computing, using a first batch of training data, a first gradient tensor comprising a gradient for each parameter of a parameter tensor for a machine learning model; identifying a first subset of gradients in the first gradient tensor based on a first gradient criteria; and updating a first subset of parameters in the parameter tensor based on the first subset of gradients in the first gradient tensor.

Abstract: Systems, methods, and non-transitory media are provided for providing spatiotemporal recycling networks (e.g., for video segmentation). For example, a method can include obtaining video data including a current frame and one or more reference frames. The method can include determining, based on a comparison of the current frame and the one or more reference frames, a difference between the current frame and the one or more reference frames. Based on the difference being below a threshold, the method can include performing semantic segmentation of the current frame using a first neural network. The semantic segmentation can be performed based on higher-spatial resolution features extracted from the current frame by the first neural network and lower-resolution features extracted from the one or more reference frames by a second neural network. The first neural network has a smaller structure and/or a lower processing cost than the second neural network.

Abstract: Systems and techniques are described herein for performing optical flow estimation between one or more frames. For example, a process can include determining a subset of pixels of at least one of a first frame and a second frame, and generating a mask indicating the subset of pixels. The process can include determining, based on the mask, one or more features associated with the subset of pixels of at least the first frame and the second frame. The process can include determining optical flow vectors between the subset of pixels of the first frame and corresponding pixels of a second frame. The process can include generating an optical flow map for the second frame using the optical flow vectors.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for operating a neural network using one or more selectable activation functions. The method generally includes generating an intermediate output of a neural network for an input into the neural network. One or more activation functions to apply to the intermediate output are selected. An output of the neural network is generated based on the selected one or more activation functions and the intermediate output, and one or more actions are taken based on the generated output.

Abstract: Disclosed are systems and techniques for capturing images (e.g., using a monocular image sensor) and detecting depth information. According to some aspects, a computing system or device can generate a feature representation of a current image and update accumulated feature information for storage in a memory based on a feature representation of a previous image and optical flow information of the previous image. The accumulated feature information can include accumulated image feature information associated with a plurality of previous images and accumulated optical flow information associated of the plurality of previous images. The computing system or device can obtain information associated with relative motion of the current image based on the accumulated feature information and the feature representation of the current image.

Abstract: Systems and techniques are provided for physical-based light estimation for inverse rendering of indoor scenes. For example, a computing device can obtain an estimated scene geometry based on a multi-view observation of a scene. The computing device can further obtain a light emission mask based on the multi-view observation of the scene. The computing device can also obtain an emitted radiance field based on the multi-view observation of the scene. The computing device can then determine, based on the light emission mask and the emitted radiance field, a geometry of at least one light source of the estimated scene geometry.

Abstract: Certain aspects of the present disclosure provide techniques for performing tabular convolution, including performing a tabularization operation on input data to generate a tabularized representation of the input data and performing a convolution operation using the tabularized representation of the input data to generate a convolution output.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for improved hidden Markov model (HMM)-based machine learning. A sequence of observations is accessed. A hidden Markov model (HMM) comprising a set of transition probabilities, a set of emission probabilities, a transition coefficient hyperparameter, and an emission coefficient hyperparameter is also accessed, and a first output inference from the HMM is generated based on the sequence of observations.

Abstract: Certain aspects of the present disclosure provide a method of processing video data. In one example, the method includes receiving input video data; sampling a first subset of clips from the input video data; providing the first subset of clips to a first component of a machine learning model to generate first output; sampling a second subset of clips from the input video data, wherein the second subset of clips comprises fewer clips than the first subset of clips; providing the second subset of clips to a second component of the machine learning model to generate a second output; aggregating the first output from the first component of the machine learning model with the second output from the second component of the machine learning model to generate aggregated output; and determining a characteristic of the input video data based on the aggregated output.

Abstract: Aspects presented herein may enable a passive positioning system, which may be a network entity or node, to be trained to identify multiple moving objects based on using training data for a single object. In one aspect, a network entity receives first RF channel data recorded by a set of devices for a coverage area during a first time period. The network entity trains an ML model based on the set of devices and the first RF channel data. The network entity receives second RF channel data recorded by the set of devices at a second time instance that is outside of the first time period. The network entity computes a number of moving objects in the coverage area at the second time instance based on the second RF channel data using the ML model.

Abstract: Disclosed are systems, apparatuses, processes, and computer-readable media for processing image data. For example, a process can include processing, for a first time step of a recurrent machine learning network, a first image of a plurality of images using a first subset of a group of distributed recurrent parameters to generate a first hidden state output associated with the first image. A process can include providing the first hidden state output as a recurrent state input to a second time step of the recurrent machine learning network. A process can include processing, for the second time step of the recurrent machine learning network, a second image of the plurality of images using the recurrent state input and a second subset of the group of distributed recurrent parameters to generate a second hidden state output associated with the second image.

Abstract: Systems and techniques are described for performing supervised learning (e.g., semi-supervised learning, self-supervised learning, and/or mixed supervision learning) for optical flow estimation. For example, a method can include obtaining an image associated with a sequence of images and generating an occluded image. The occluded image can include at least one of the image with an occlusion applied to the image and a different image of the sequence of images with the occlusion applied. The method can include determining a matching map based at least on matching areas of the image and the occluded image and, based on the matching map, determining a loss term associated with an optical flow loss prediction associated with the image and the occluded image. The loss term may include a matched loss and/or other loss. Based on the loss term, the method can include training a network configured to determine an optical flow between images.

Abstract: Certain aspects of the present disclosure provide techniques for improved domain adaptation in machine learning. A feature tensor is generated by processing input data using a feature extractor. A first set of logits is generated by processing the feature tensor using a domain-agnostic classifier, and a second set of logits is generated by processing the feature tensor using a domain-specific classifier. A loss is computed based at least in part on the first set of logits and the second set of logits, where the loss includes a divergence loss component. The feature extractor, the domain-agnostic classifier, and the domain-specific classifier are refined using the loss.

Abstract: Disclosed are systems, methods, and non-transitory media for performing passive radio frequency (RF) location detection operations. In some aspects, RF data, such as RF signals including channel state information (CSI), can be received from a wireless device. The RF data can be provided to a self-supervised machine-learning architecture that is configured to perform object location estimation.

Abstract: Systems and techniques are provided for processing sensor data. For example, a process can include determining, using a trained machine learning system, a predicted depth map for an image, the predicted depth map including a respective predicted depth value for each pixel of the image. The process can further include obtaining depth values for the image, the depth values including depth values for less than all pixels of the image from a tracker configured to determine the depth values based on one or more feature points between frames. The process can further include scaling the predicted depth map for the image using and the depth values. The output of the process can be scale-correct depth prediction values.