Abstract: This document describes systems and techniques related to deep association for sensor fusion. For example, a model trained using deep machine learning techniques, may be used to generate an association score matrix that includes probabilities that tracks from different types of sensors are related to the same objects. This model may be trained using a convolutional recurrent neural network and include constraints not included in other training techniques. Focal loss can be used during training to compensate for imbalanced data samples and address difficult cases, and data expansion techniques can be used to increase the multi-sensor data space. Simple thresholding techniques can be applied to the association score matrix to generate an assignment matrix that indicates whether tracks from one sensor and tracks from another sensor match. In this manner, the track association process may be more accurate than current sensor fusion techniques, and vehicle safety may be increased.

Abstract: This document describes Kurtosis based pruning for sensor-fusion systems. Kurtosis based pruning minimizes a total quantity of comparisons performed when fusing together large sets of data. Multiple candidate radar tracks may possibly align with one of multiple candidate visual tracks. For each candidate vision track, a weight or other evidence of matching is assigned to each candidate radar track. An inverse of matching errors between each candidate vision and each candidate radar track contributes to this evidence, which may be normalized to produce, for each candidate vision track, a distribution associated with all candidate radar tracks. A Kurtosis or shape of this distribution is calculated. Based on the Kurtosis values, some candidate radar tracks are selected for matching and other remaining candidate radar tracks are pruned. The Kurtosis aids in determining how many candidates to retain and how many to prune.

Abstract: This document describes Kurtosis based pruning for sensor-fusion systems. Kurtosis based pruning minimizes a total quantity of comparisons performed when fusing together large sets of data. Multiple candidate radar tracks may possibly align with one of multiple candidate visual tracks. For each candidate vision track, a weight or other evidence of matching is assigned to each candidate radar track. An inverse of matching errors between each candidate vision and each candidate radar track contributes to this evidence, which may be normalized to produce, for each candidate vision track, a distribution associated with all candidate radar tracks. A Kurtosis or shape of this distribution is calculated. Based on the Kurtosis values, some candidate radar tracks are selected for matching and other remaining candidate radar tracks are pruned. The Kurtosis aids in determining how many candidates to retain and how many to prune.

Abstract: This document describes radar tracking with model estimates augmented by radar detections. An example tracker analyzes information derived using radar detections to enhance radar tracks having object measurements estimated from directly analyzing data cubes with a model (e.g., a machine-learning model). High-quality tracks with measurements to objects of importance can be quickly produced with the model. However, the model only estimates measurements for classes of objects its training or programming can recognize. To improve estimated measurements from the model, or even in some cases, to convey additional classes of objects, the tracker separately analyzes detections. Detections that consistently align to objects recognized by the model can update model-derived measurements conveyed initially in the tracks. Consistently observed detections that do not align to existing tracks may be used to establish new tracks for conveying more classes of objects than the model can recognize.

Abstract: This document describes methods and systems directed at history-based identification of incompatible tracks. The historical trajectory of tracks can be advantageous to accurately determine whether tracks originating from different sensors identify the same object or different objects. However, recording historical data of several tracks may consume vast amounts of memory or computing resources, and related computations may become complex. The methods and systems described herein enable a sensor fusion system of an automobile or other vehicle to consider historical data when associating and pairing tracks, without requiring large amounts of memory and without tying up other computing resources.

Abstract: This document describes a multiple hypothesis-based data fusion tracker. Each hypothesis aligns to a different pseudo measurement type. The fusion tracker automatically determines, using a predefined error covariance associated with the radar, which pseudo measurement type has a greater chance of being accurate for a current situation. The fusion tracker may rely on either one of two combined radar and vision calculations, or the fusion tracker may ignore the vision-based pseudo measurements and instead, rely on radar pseudo measurements alone. By selecting between three different bounding boxes, a vision angle based box, a vision lateral position based box, or a radar only based box, the fusion tracker can balance accuracy and speed when drawing, repositioning, or resizing bounding boxes, even under congested traffic or other high volume situations.

Abstract: This document describes Kurtosis based pruning for sensor-fusion systems. Kurtosis based pruning minimizes a total quantity of comparisons performed when fusing together large sets of data. Multiple candidate radar tracks may possibly align with one of multiple candidate visual tracks. For each candidate vision track, a weight or other evidence of matching is assigned to each candidate radar track. An inverse of matching errors between each candidate vision and each candidate radar track contributes to this evidence, which may be normalized to produce, for each candidate vision track, a distribution associated with all candidate radar tracks. A Kurtosis or shape of this distribution is calculated. Based on the Kurtosis values, some candidate radar tracks are selected for matching and other remaining candidate radar tracks are pruned. The Kurtosis aids in determining how many candidates to retain and how many to prune.