Abstract: Techniques are provide for generating occupancy grids based on inputs from multiple heterogeneous sensors. An example method for generating an occupancy grid includes obtaining detection information from a plurality of heterogeneous sensors, generating a single measurement grid based on the detection information from the plurality of heterogeneous sensors, determining occupancy probabilities for a plurality of cells in the single measurement grid, and outputting the occupancy grid based at least in part on the occupancy probabilities.

Abstract: Techniques are provided for utilizing a dynamic occupancy grid (DoG) for tracking objects proximate to an autonomous or semi-autonomous vehicle. An example method for generating an object track list in a vehicle includes obtaining sensor information from one or more sensors on the vehicle, determining a first set of object data based at least in part on the sensor information and an object recognition process, generating a dynamic grid based on an environment proximate to the vehicle based at least in part on the sensor information, determining a second set of object data based at least in part on the dynamic grid, and outputting the object track list based on a fusion of the first set of object data and the second set of object data.

Abstract: A dynamic occupancy grid determination method includes: obtaining, at an apparatus, at least one radar-based occupancy grid based on radar sensor measurements, each of the at least one radar-based occupancy grid comprising a plurality of first cells, each cell of the plurality of first cells having a corresponding first occupancy probability and first velocity; obtaining, at the apparatus, at least one camera-based occupancy grid based on camera measurements, each of the at least one camera-based occupancy grid comprising a plurality of second cells, each cell of the plurality of second cells having a corresponding second occupancy probability and second velocity; and determining, at the apparatus, a dynamic occupancy grid by analyzing the at least one radar-based occupancy grid and the at least one camera-based occupancy grid.

Abstract: Techniques are provided for detecting objects proximate to a vehicle with multiple signal paths. An example method for generating object representations with multiple signal paths includes obtaining image information from at least one camera module disposed on a vehicle, obtaining target information from at least one radar module disposed on the vehicle, generating a first detection representation with a first signal path based on the image information and the target information, generating a second detection representation with a second signal path based on the image information and the target information, wherein the second signal path is different than the first signal path, and outputting the first detection representation and the second detection representation.

Abstract: A method, for determining a dynamic occupancy grid, includes: obtaining radar measurement data from at least one radar sensor of an apparatus; obtaining camera-derived data based on at least one image obtained by at least one camera of the apparatus; and determining the dynamic occupancy grid based on the radar measurement data and the camera-derived data.

Abstract: Example systems and techniques are described for training a machine learning model. A system includes memory configured to store image data captured by a plurality of cameras and one or more processors communicatively coupled to the memory. The one or more processors are configured to execute a machine learning model on the image data, the machine learning model including a plurality of layers. The one or more processors are configured to apply a non-linear mapping function to output of one layer of the plurality of layers to generate depth data. The one or more processors are configured to train the machine learning model based on the depth data to generate a trained machine learning model.

Abstract: Systems and techniques are described for performing object detection and tracking. For example, a tracking object can obtain an image comprising a target object at least partially in contact with a surface. The tracking object can obtain a plurality of two-dimensional (2D) keypoints based on one or more features associated with one or more portions of the target object in contact with the surface in the image. The tracking object can obtain information associated with a contour of the surface. Based on the plurality of 2D keypoints and the information associated with the contour of the surface, the tracking object can determine a three-dimensional (3D) representation associated with the plurality of 2D keypoints.

Abstract: An apparatus, method and computer-readable media are disclosed for processing images. For example, a method is provided for processing images for one or more visual perception tasks using a machine learning system including one or more transformer layers. The method includes: obtaining a plurality of input images associated with a plurality of spatial views of a scene; generating, using a machine learning-based encoder of a machine learning system, a plurality of features from the plurality of input images; and combining timing information associated with capture of the plurality of input images with at least one input of the machine learning system to synchronize the plurality of features in time.

Abstract: Example systems and techniques are described for controlling operation of a vehicle and training a machine learning model for controlling operation of a vehicle. A system includes memory configured to store point cloud data associated with the vehicle and one or more processors communicatively coupled to the memory. The one or more processors are configured to determine a depth map indicative of distance of one or more objects to the vehicle and control operation of a vehicle based on the depth map. The depth map is based on executing a machine learning model, the machine learning model being trained with a slice loss function determined from training point cloud data having a respective depth that is greater than the average depth for a set of points of the point cloud data plus a threshold.

Abstract: A processing system for cross-sensor calibration is configured to perform a first edge detection process on a camera image to generate an edge detected camera image, and perform a second edge detection process on a point cloud frame to generate an edge detected point cloud frame. The processing system projects the edge detected point cloud frame onto the edge detected camera image using an initial calibration matrix, determines an objective function representing an overlap of points in the edge detected point cloud frame and corresponding edge pixel values in the edge detected camera image, and determines a final calibration matrix based on the objective function.

Abstract: Certain aspects of the present disclosure provide techniques and apparatus for improved machine learning. Data from a source domain and data from a target domain is accessed. A set of machine learning models is trained, based on the data from the source domain and the data from the target domain, to generate depth outputs based on input images. Training the set of machine learning models includes: generating a discriminator output based at least in part on an input image frame from either the source domain or the target domain, generating an adversarial loss based on the discriminator output, and refining one or more machine learning models of the set of machine learning models based on the adversarial loss.

Abstract: An example device for processing image data includes a processing unit configured to: receive, from a camera of a vehicle, a first image frame at a first time and a second image frame at a second time; receive, from an odometry unit of the vehicle, a first position of the vehicle at the first time and a second position of the vehicle at a second time; calculate a pose difference value representing a difference between the second and first positions; form a pose frame having a size corresponding to the first and second image frames and sample values including the pose difference value; and provide the first and second image frames and the pose frame to a neural networking unit configured to calculate depth for objects in the first image frame and the second image frame, the depth for the objects representing distances between the objects and the vehicle.

Abstract: In some aspects, a device may receive point data associated with a cell of an occupancy grid for controlling a vehicle. The device may determine, based on the point data, a characteristic of the cell that is associated with an occupancy probability of the cell, wherein the occupancy probability is determined according to a first technique based on the point data. The device may configure, based on the characteristic, the occupancy probability for the cell, within the occupancy grid, according to a second technique. Numerous other aspects are described.

Abstract: Disclosed are systems, apparatuses, processes, and computer-readable media for processing image data. For example, an apparatus can compute initial embeddings from a plurality of images. The apparatus can construct a graph comprising nodes representing the initial embeddings. The apparatus can further perform, based on the graph, a plurality of message passing steps successively to generate final embeddings. The apparatus can classify, using a classification engine, one or more objects in each of the plurality of images based on the final embeddings. The apparatus can further compute a classification loss based on the classifying of the one or more objects.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a computing device receives a predicted depth map determined by a model and determines differences between the predicted depth map and a depth mask. The depth mask may be predetermined to include values indicating a probability that a corresponding region of the predicted depth map is a sky region. A first loss term for the predicted depth map is determined based on the differences and the model is trained based on the first loss term. Other aspects and features are also claimed and described.

Abstract: Systems and techniques are provided for detecting objects. For example, an apparatus may include: at least one memory and at least one processor coupled to the at least one memory. The at least one processor may be configured to determine, for a column of an image captured by a camera, a pixel associated with an object. The at least one processor may be further configured to obtain a distance between the camera and the object. The at least one processor may be further configured to determine, based on the distance, a probability of occupancy of a space relative to the camera.

Abstract: In some aspects, a device may obtain sensor data associated with identifying measured properties of an object in an environment. The device may detect a trigger event associated with at least one of the environment or the device. The device may modify, based on detecting the trigger event, one or more pre-processing operations associated with the sensor data for input to a neural network, and/or one or more post-processing operations associated with an object detection output of the neural network. The device may perform the one or more pre-processing operations associated with the sensor data to generate pre-processed sensor data. The device may generate the object detection output for the object based on detecting the object using the pre-processed sensor data as the input to the neural network. The device may perform the one or more post-processing operations using the object detection output. Numerous other aspects are described.

Abstract: Techniques and systems are provided for determining static occupancy. For example, an apparatus can be configured to determine one or more pixels associated with one or more static objects depicted in one or more images of a three-dimensional space. The apparatus can be configured to obtain a point map including a plurality of map points, the plurality of map points corresponding to a portion of the three-dimensional space. The apparatus can be configured to determine, based on the point map and the one or more pixels associated with the one or more static objects, a probability of occupancy by the one or more static objects in the portion of the three-dimensional space. The apparatus can be configured to combine information across multiple images of the three-dimensional space, and can determine probabilities of occupancy for all cells in a static occupancy grid that is associated with the three-dimensional space.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a computing device may receive a predicted depth map and a measured depth map and may determine a time difference between the predicted depth map and the measured depth map. A supervision loss term may be determined based on the time difference, such as by weighting the supervision loss term based on the time difference. The computing device may train a model based on the supervision loss term, such as a model that generated the predicted depth map. Other aspects and features are also claimed and described.

Abstract: In some aspects, a device may receive sensor data associated with a vehicle and a set of frames. The device may aggregate, using a first pose, the sensor data associated with the set of frames to generate an aggregated frame, wherein the aggregated frame is associated with a set of cells. The device may obtain an indication of a respective occupancy label for each cell from the set of cells, wherein the respective occupancy label includes a first occupancy label or a second occupancy label, and wherein a subset of cells from the set of cells are associated with the first occupancy label. The device may train, using data associated with the aggregated frame, a machine learning model to generate an occupancy grid, based on a loss function that only calculates a loss for respective cells from the subset of cells. Numerous other aspects are described.