Abstract: A device for processing sensor data is configured to receive the first frames from a first sensor; receive the second frames from a second sensor; perform a first feature extraction on the first frames using a first dynamic neural network to determine first features; perform a second feature extraction on the second frames using a second dynamic neural network to determine second features; determine a first delay associated with the first features; determine a second delay associated with the second features; modify a topology of the second dynamic neural network based on the first delay and the second delay; and use the second dynamic neural network with the modified topology to generate an output.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method includes receiving a first image frame from the first camera and a second image frame from the second camera. The method may also include determining a first set of optical flows between the first image frame and the second image frame and determining a second set of optical flows based on the first set of optical flows and positions of the first camera and the second camera on the vehicle. Position data may be determined for objects in an area surrounding the vehicle based on the second set of optical flows. Other aspects and features are also claimed and described.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method of image processing includes receiving image BEV features and receiving first radio detection and ranging (RADAR) BEV features. The first RADAR BEV features that are received are determined based on first RADAR data associated with a first data type. First normalized RADAR BEV features are determined, which includes rescaling the first RADAR BEV features using a first attention mechanism based on the image BEV features and the first RADAR BEV features. Fused data is determined that combines the first normalized RADAR BEV features and the image BEV features. Other aspects and features are also claimed and described.

Abstract: A method for selecting one or more Regions of Interest (RoIs) for human annotations includes obtaining sensor data generated by one or more sensors of a vehicle; applying at least one class-agnostic heuristic function to the sensor data to determine a presence and an approximate position of one or more objects in an RoI of the sensor data; selecting one or more RoIs having proposed annotations for the one or more objects for refinement by an annotator; and outputting the one or more selected RoIs.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method includes receiving a first image frame from a first camera; receiving second image frame from a second camera. The fields-of-view of the cameras partially overlap. A set of coordinates associated with pixel values of the first image frame and pixel values of the second image frame are determined. The set of coordinates correspond to an overlap of the first field-of-view and the second field-of-view. A first uncertainty window metric is determined based the set of coordinates and first uncertainty values. A second uncertainty window metric is determined based on the first uncertainty values and second uncertainty values associated with RADAR. Fused depth data is determined based on the set of coordinates, the RADAR data, and the first and second uncertainty window metrics. Other aspects and features are also claimed and described.

Abstract: A sensor data processing system includes various elements, including a perception unit that collects data representing positions of sensors on a vehicle and obtains environmental information around the vehicle via the sensors. The sensor data processing system also includes a feature fusion unit that combines the first environmental information from the sensors into first fused feature data representing first positions of objects around the vehicle, provides the first fused feature data to the object tracking unit, receives feedback for the first fused feature data from the object tracking unit, and combines second environmental information from the sensors using the feedback into second fused feature data representing second positions of objects around the vehicle. The sensor data processing system may then at least partially control operation of the vehicle using the second fused feature data.

Abstract: Disclosed are techniques for localization of an object. For example, a device can generate, based on sensor data obtained from sensor(s) associated with an object, a predicted map comprising predicted nodes associated with a predicted location of the object within an environment. The device can receive a high definition (HD) map comprising HD nodes associated with a HD location of the object within the environment. The device can further match the predicted nodes with the HD nodes to determine pair(s) of matched nodes between the predicted map and the HD map. The device can determine, based on a comparison between nodes in each pair of the pair(s) of matched nodes, a respective node score for each pair of the pair(s) of matched nodes. The device can determine, based on the respective node score for each pair of the pair(s) of matched nodes, a location of the object within the environment.

Abstract: Aspects presented herein may enable a UE to detect and identify a weather condition of an environment based on the sparsity of FFT/DWT coefficients derived from a set of range images associated with the environment. In one aspect, a UE converts a set of point clouds associated with an environment to a set of range images based on a spherical projection. The UE applies at least one of FFT or DWT to the set of range images to obtain a set of FFT coefficients or a set of DWT coefficients. The UE identifies a level of a condition for the environment based on a sparsity of the set of FFT coefficients or the set of DWT coefficients.

Abstract: An apparatus for multi-object tracking determines a current representation of a current object in a current image. The apparatus computes a joint Gaussian distribution between the current representation of the current object and a previous representation stored in one or more memory buffers, wherein the previous representation was determined from a previous image. The apparatus updates the one or more memory buffers based on the joint Gaussian distribution. For example, the apparatus determines whether to remove or replace the previous representation in the one or more memory buffers based on values of a covariance matrix of the joint Gaussian distribution.

Abstract: A system for navigation store, in a time-based buffer, a first set of frames acquired by a sensor; stores, in a distance-based buffer, a second set of frames acquired by the sensor, performs moving object segmentation on the first set of frames and the second set of frames to identify at least one moving object in a scene of the frames; predicts a trajectory of the at least one moving object; and performs a navigation function based on the predicted trajectory.

Abstract: In some aspects, a device may obtain, via a camera associated with the device, an image that includes one or more objects located within an area of the device. The device may generate a first three-dimensional output based at least in part on the image. The device may obtain, via an audio component associated with the device, an audio input associated with the one or more objects. The device may generate a second three-dimensional output based at least in part on the audio input. The device may detect the one or more objects based at least in part on the first three-dimensional output and the second three-dimensional output. Numerous other aspects are described.

Abstract: A method of processing image data includes receiving, with a frame correction machine-learning (ML) model executing on processing circuitry, an image frame captured from a first camera of a plurality of cameras; performing, with the frame correction ML model executing on the processing circuitry, image frame correction to generate a corrected image frame based on weights or biases of the frame correction ML model applied to two or more of: samples of the image frame, samples of previously captured image frames from the first camera, or samples from image frames from other cameras of the plurality of cameras; and performing, with the processing circuitry, post-processing based on the corrected image frame.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method of image processing includes receiving an image frame from an image sensor of a camera; receiving an indicator associated with a type of lens of the camera; determining a first tensor grid associated with the indicator, the first tensor grid including a plurality of image framework positions associated with the type of lens; and determining, using a machine learning model, a BEV feature map corresponding to the image frame based on features of the image frame and the first tensor grid. Other aspects and features are also claimed and described.

Abstract: Systems and techniques are described herein for determining road profiles. For instance, a method for determining a road profiles is provided. The method may include extracting image features from one or more images of an environment, wherein the environment includes a road; generating a segmentation mask based on the image features; determining a subset of the image features based on the segmentation mask; generating image-based three-dimensional features based on the subset of the image features; obtaining point-cloud-based three-dimensional features derived from a point cloud representative of the environment; combining the image-based three-dimensional features and the point-cloud-based three-dimensional features to generate combined three-dimensional features; and generating a road profile based on the combined three-dimensional features.

Abstract: This disclosure provides systems, methods, and devices that support image processing. In a first aspect, a method for multi-sensor fusion includes receiving first information indicative of a first set of BEV features of image data captured by an image sensor; receiving second information indicative of a second set of BEV features of non-image sensor data captured by a non-image sensor; and determining fused data that combines the image data and the non-image sensor data based on the first information, the second information, and third information indicative of differences between BEV features of training data and the first set of BEV features and the second set of BEV features. The BEV features of the training data include a third set of BEV features associated with the image sensor and a fourth set of BEV features associated with the non-image sensor. Other aspects and features are also claimed and described.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method of image processing includes receiving, by a processor, image data from a camera image sensor; receiving, by the processor, point cloud data from a light detection and ranging (LiDAR) sensor; generating, by the processor and using a first machine learning model, fused image data that combines the image data and the point cloud data; and determining, by the processor and using a second machine learning model, whether the fused image data satisfies a criteria based on whether a population risk function of the first machine learning model exceeds a threshold. Other aspects and features are also claimed and described.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method of image processing includes receiving a plurality of image frames by a computing device and using machine learning models to identify corrupted or occluded image frames. A first machine learning model may identify corrupted image frames, while a second machine learning model may identify partially occluded image frames. The method may further include generating updated versions of image frames captured by vehicle cameras, such as based on feature vectors from the first and second machine learning models. The feature vectors may be fused and provided to a third machine learning model to generate updated versions of occluded image frames. The method may further include determining vehicle control instructions based on the updated versions. Other aspects and features are also claimed and described.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method is provided that includes determining a first set of feature vectors for received images for a top view representation of an area surrounding a vehicle and a second set of feature vectors for a cylindrical representation of the area. The method may further include determining a first set of locations based on the first set of feature vectors and determining a second set of locations based on the second set of feature vectors. A third set of locations may be determined based on the first and second sets of locations, such as combining the first and second sets using a transformer attention process. Vehicle control instructions may then be determined based on the third set of locations. Other aspects and features are also claimed and described.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method of image processing includes receiving first kinematic information associated with a camera image sensor; receiving, by the processor, point cloud data from a light detection and ranging (LiDAR) sensor; generating, by the processor, first image data that is time-synchronized with the point cloud data based on the first kinematic information and a neural radiance fields (NeRF) model; and generating, by the processor, fused data that combines the first image data and the point cloud data. Other aspects and features are also claimed and described.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method is provided to train a machine learning model using image data and position data to identify contact points and ground surface normal vectors. Image data is received that depicts an object, and position data for the object is also received, such as point cloud position information for various points along the object's exterior surface. Two sets of labels may then be determined based on the position data, with one set identifying where the object contacts a ground surface and another identifying at least one normal vector for the ground surface. The machine learning model may then be trained based on both sets of labels to determine three-dimensional bounding boxes, normal maps, or combinations thereof. Other aspects and features are also claimed and described.