Abstract: Certain aspects of the present disclosure provide techniques and apparatus for efficient processing of visual content using machine learning models. An example method generally includes generating, from an input, an embedding tensor for the input. The embedding tensor for the input is projected into a reduced-dimensional space projection of the embedding tensor based on a projection matrix. An attention value for the input is derived based on the reduced-dimensional space projection of the embedding tensor and a non-linear attention function. A match, in the reduced-dimensional space, is identified between a portion of the input and a corresponding portion of a target against which the input is evaluated based on the attention value for the input. One or more actions are taken based on identifying the match.

Abstract: Disclosed are systems and techniques for capturing images (e.g., using an image capture) and performing reverse optical flow error correction. According to some aspects, a computing system or device can obtain first disparity information associated with a current image. The first disparity information estimates a first movement of a first feature to a first destination location in the current image. The computing system or device can warp the current image based on the first disparity information to obtain an estimated previous image, determine a confidence map associated with a confidence of the first disparity information based on a difference associated with the estimated previous image; and apply the confidence map to the first disparity information to generate updated first disparity information.

Abstract: Systems and techniques described herein relate to generating an inter-frame from a first and second frame. An apparatus includes a memory storing a first frame and a second frame; and a processor coupled to the memory and configured to: estimate at least one optical flow between the first frame and the second frame; generate, based on the at least one optical flow, at least one occlusion mask; generate, based on the at least one optical flow and the at least one occlusion mask, at least one weighting mask; generate, based on the at least one optical flow and the at least one weighting mask, at least one inter-frame optical flow; generate, based on the at least one inter-frame optical flow and at least one of the first frame or the second frame, at least one warped frame; and generate, based on the at least one warped frame, an inter-frame.

Abstract: Certain aspects of the present disclosure provide techniques for generating fine depth maps for images of a scene based on semantic segmentation and segment-based refinement neural networks. An example method generally includes generating, through a segmentation neural network, a segmentation map based on an image of a scene. The segmentation map generally comprises a map segmenting the scene into a plurality of regions, and each region of the plurality of regions is generally associated with one of a plurality of categories. A first depth map of the scene is generated through a first depth neural network based on a depth measurement of the scene. A second depth map of the scene is generated through a depth refinement neural network based on the segmentation map and the first depth map. One or more actions are taken based on the second depth map of the scene.

Abstract: Dimensional decomposition convolution systems and techniques are described. A system receives a tensor including a first number of dimensions. The system processes a variant of the tensor using a convolution function including a second number of dimensions to generate a processed tensor. The first number of dimensions is greater than the second number of dimensions. A plurality of dimensions from the first number of dimensions of the tensor are grouped into a dimension of the variant of the tensor to reduce dimensionality of the variant of the tensor to the second number of dimensions.

Abstract: Methods, systems, and devices for wireless communications are described. A transmitter may transmit control signaling indicating a common resource commonly allocated to a set of receivers, a dedicated resource allocated to a first receiver, and a ramification coding structure that indicates a common code segment and a dedicated code segment of a ramification codeword. The transmitter may determine the ramification coding structure based on common data for multiple receivers, a quantity of hierarchical levels, a machine learning model, or any combination thereof. The transmitter may encode common and dedicated data into at least one ramification codeword in a dynamic signaling format using the ramification coding structure, and transmit the at least one ramification codeword to the first receiver via the common and dedicated resources. Being aware of the ramification coding structure, the first receiver may decode the ramification codeword transmitted by the transmitter in the dynamic signaling format.

Abstract: Certain aspects of the present disclosure provide techniques for machine learning using basis decomposition, comprising receiving a first runtime record, where the first runtime record includes RF signal data collected in a physical space; processing the first runtime record using a plurality of basis machine learning (ML) models to generate a plurality of inferences; aggregating the plurality of inferences to generate a prediction comprising a plurality of coordinates; and outputting the prediction, where the plurality of coordinates indicate a location of a physical element in a physical space.

Abstract: Methods and systems for user-selected viewpoint rendering of a virtual meeting are provided herein. First image data generated by a first client device during a virtual meeting and second image data generated by a second client device during a virtual meeting is obtained. The first image data depicts object(s) captured from a first vantage point and the second image data depicts the object(s) captured from a second vantage point. A request is received from a third client device for third image data depicting the object(s) captured from a third vantage point. The third image data depicting the object(s) corresponding to the third vantage point is generated based on the first image data and the second image data. A rendering of the third image data is provided for presentation via a graphical user interface (GUI) of the third client device during the virtual meeting in accordance with the request.

Abstract: Certain aspects of the present disclosure provide techniques for improved machine learning using gradient pruning, comprising computing, using a first batch of training data, a first gradient tensor comprising a gradient for each parameter of a parameter tensor for a machine learning model; identifying a first subset of gradients in the first gradient tensor based on a first gradient criteria; and updating a first subset of parameters in the parameter tensor based on the first subset of gradients in the first gradient tensor.

Abstract: Systems and techniques are described herein for performing optical flow estimation between one or more frames. For example, a process can include determining a subset of pixels of at least one of a first frame and a second frame, and generating a mask indicating the subset of pixels. The process can include determining, based on the mask, one or more features associated with the subset of pixels of at least the first frame and the second frame. The process can include determining optical flow vectors between the subset of pixels of the first frame and corresponding pixels of a second frame. The process can include generating an optical flow map for the second frame using the optical flow vectors.

Abstract: Certain aspects of the present disclosure provide techniques for performing tabular convolution, including performing a tabularization operation on input data to generate a tabularized representation of the input data and performing a convolution operation using the tabularized representation of the input data to generate a convolution output.

Abstract: A method includes receiving, as input, reference audio data representing a reference audio signal captured by an audio input device. The method also includes receiving, as input, from a beamformer, spatially-filtered audio data representing an output of the beamformer, the beamformer configured to spatially filter, based on additional audio data captured by one or more additional audio input devices, the reference audio data to attenuate one or more interfering signals in the spatially-filtered audio data. The method processes, using a trained guided speech-enhancement network, the reference audio data and the spatially-filtered audio data to generate, as output, enhanced audio data, the guided speech-enhancement network processing the reference audio data and the spatially-filtered audio data to further attenuate, in the enhanced audio data, the one or more interfering signals attenuated by the beamformer.

Abstract: Systems and techniques are described for performing supervised learning (e.g., semi-supervised learning, self-supervised learning, and/or mixed supervision learning) for optical flow estimation. For example, a method can include obtaining an image associated with a sequence of images and generating an occluded image. The occluded image can include at least one of the image with an occlusion applied to the image and a different image of the sequence of images with the occlusion applied. The method can include determining a matching map based at least on matching areas of the image and the occluded image and, based on the matching map, determining a loss term associated with an optical flow loss prediction associated with the image and the occluded image. The loss term may include a matched loss and/or other loss. Based on the loss term, the method can include training a network configured to determine an optical flow between images.

Abstract: A computer-implemented method includes receiving a first input. The first input is interpolated based on a first shift along a first dimension and a second shift along a second dimension. A first output is generated based on the interpolated first input. The first output corresponds to a vectorized bilinear shift of the first input for use in place of grid sampling algorithms.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for messaging in a wireless communications system using neural networks. An example method generally includes receiving a first message and a second message, wherein the second message comprises a secret message to be hidden in the first message. The first message and second message are combined into a combined message. An emulation message is generated through an encoder neural network based on the combined message. The emulation message generally comprises a message decodable by a receiving device into the first message. The emulation message is output emulation message for transmission to the receiving device.

Abstract: A method including processing a stream of data in a sequence of tasks. The processing including receiving a first block of data of the stream of data, determining features associated with the first block of data, selecting, based on the features, one of a first a task to process the first block of data or a second task to process the first block of data and if the second task is selected, shift an output of the second task in time to align the output of the second task with a predicted output of the first task processing a second block of data of the stream of data.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for training a machine learning model. An example method generally includes partitioning a machine learning model into a plurality of partitions. A request to update a respective partition of the plurality of partitions in the machine learning model is transmitted to each respective participating device of a plurality of participating devices in a federated learning scheme, and the request may specify that the respective partition is to be updated based on unique data at the respective participating device. Updates to one or more partitions in the machine learning model are received from the plurality of participating devices, and the machine learning model is updated based on the received updates.

Abstract: Certain aspects of the present disclosure provide techniques for training and using machine learning models to predict locations of stationary and non-stationary objects in a spatial environment. An example method generally includes measuring, by a device, a plurality of signals within a spatial environment. Timing information is extracted from the measured plurality of signals. Based on a machine learning model, the measured plurality of signals within the spatial environment, and the extracted timing information, locations of stationary reflection points and locations of non-stationary reflection points in the spatial environment are predicted. One or more actions are taken by the device based on predicting the locations of stationary reflection points and non-stationary reflection points in the spatial environment.

Abstract: Systems and techniques are described herein for performing optical flow estimation for one or more frames. For example, a process can include determining an optical flow prediction associated with a plurality of frames. The process can include determining a position of at least one feature associated with a first frame and determining, based on the position of the at least one feature in the first frame and the optical flow prediction, a position estimate of a search area for searching for the at least one feature in a second frame. The process can include determining, from within the search area, a position of the at least one feature in the second frame.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for surrogated federated learning. A set of intermediate activations is received at a trusted server from a node device, where the node device generated the set of intermediate activations using a first set of layers of a neural network. One or more weights associated with a second set of layers of the neural network are refined using the set of intermediate activations, and one or more weight updates corresponding to the refined one or more weights are transmitted to a federated learning system.