Abstract: A system for processing image data and position data, the system comprising: a memory for storing the image data and the position data; and processing circuitry in communication with the memory. The processing circuitry is configured to: apply a first encoder to extract, from the image data, a first set of features; apply a first decoder to determine, based on the first set of features, a first uncertainty score. Additionally, the processing circuitry is configured to apply a second encoder to extract, from the position data, a second set of features; apply a second decoder to determine, based on the second set of features, a second uncertainty score; and fuse the first set of features and the second set of features based on the first uncertainty score and the second uncertainty score.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method includes receiving an image frame representing a scene; receiving point cloud data representing the scene; determining first sets of image frame features; determining second sets of point cloud data features based on a plurality of voxels representing the point cloud data; determining a third set of features of the image frame based on a first set of features of the plurality of first sets of features of the image frame and a second set of features of the plurality of second sets of features of the point cloud data; and outputting fused data that combines the third set of features of the image frame and a fourth set of features of the point cloud data. Other aspects and features are also claimed and described.

Abstract: In some aspects, a device may obtain, via a camera associated with the device, an image that includes one or more objects located within an area of the device. The device may generate a first three-dimensional output based at least in part on the image. The device may obtain, via an audio component associated with the device, an audio input associated with the one or more objects. The device may generate a second three-dimensional output based at least in part on the audio input. The device may detect the one or more objects based at least in part on the first three-dimensional output and the second three-dimensional output. Numerous other aspects are described.

Abstract: This disclosure provides systems, methods, and devices that support image processing. In a first aspect, a method for multi-sensor fusion includes receiving first information indicative of a first set of BEV features of image data captured by an image sensor; receiving second information indicative of a second set of BEV features of non-image sensor data captured by a non-image sensor; and determining fused data that combines the image data and the non-image sensor data based on the first information, the second information, and third information indicative of differences between BEV features of training data and the first set of BEV features and the second set of BEV features. The BEV features of the training data include a third set of BEV features associated with the image sensor and a fourth set of BEV features associated with the non-image sensor. Other aspects and features are also claimed and described.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method is provided to train a machine learning model using image data and position data to identify contact points and ground surface normal vectors. Image data is received that depicts an object, and position data for the object is also received, such as point cloud position information for various points along the object's exterior surface. Two sets of labels may then be determined based on the position data, with one set identifying where the object contacts a ground surface and another identifying at least one normal vector for the ground surface. The machine learning model may then be trained based on both sets of labels to determine three-dimensional bounding boxes, normal maps, or combinations thereof. Other aspects and features are also claimed and described.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method is provided for determining the locations and bounding surfaces of objects depicted in image frames captured by fisheye image sensors attached to a vehicle. The method includes receiving raw fisheye image data from the sensor and using machine learning models to determine the locations and three-dimensional bounding surfaces of objects in the image frame. The bounding surfaces may be defined by three-dimensional polar coordinates representing portions of the viewing area of the fisheye image sensor. Control instructions for the vehicle may then be determined based on the bounding surfaces. In certain implementations, the bounding surfaces may be determined in three-dimensional polar coordinates. Other aspects and features are also claimed and described.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method is provided that includes receiving sensor data from a plurality of sensors on a vehicle and determining a three-dimensional representation of an area surrounding the vehicle by mapping the sensor data onto a three-dimensional surface. The plurality of sensors may include at least one perspective view sensor and at least one top view sensor, and the three-dimensional surface may include sensor data from the at least one perspective view sensor and sensor data from the at least one top view sensor. The method may further include determining, with a machine learning model, one or more characteristics of the area surrounding the vehicle based on the three-dimensional representation. Other aspects and features are also claimed and described.