Abstract: In various examples, object track management for autonomous or semi-autonomous systems and applications is described herein. Systems and methods are disclosed that may limit the number of objects that are tracked based on one or more criteria. For instance, the number of objects that are tracked may be limited to a threshold number of objects when a number of detected objects exceeds a threshold. The systems and methods may use parameters associated with the detected objects to determine priority scores associated with the detected objects, and may then determine to only track the detected objects with the highest scores (e.g., high priority objects). As a result, latency and compute of the system may be reduced while still maintaining tracking with respect to safety-critical objects.

Abstract: In various examples, a technique for low-latency fusion and tracking of objects from multiple cameras is disclosed that includes determining a plurality of input objects and corresponding object characteristics, individual input objects and respective object characteristics being determined based at least on an image generated using a respective camera. The technique also includes identifying at least one subset of the plurality of input objects, the at least one subset corresponding to a respective physical object and comprising at least one input object that satisfies a similarity criterion. The technique further includes generating an output object associated with one or more smoothed object characteristics, the output object being generated based at least on two or more input objects included in the at least one subset of the plurality of input objects. The at least one subset corresponds to a physical object that is visible to two or more cameras.

Abstract: In various examples, disclosed techniques introduce a time-window based sensor measurement scheduling engine that determines an ordering of measurements received from multiple sensors. The measurements may be sorted by detection time and submitted in sorted order to a sensor fusion system. Upon receiving measurements from sensors, the scheduling engine determines a current time window between a most recently-submitted measurement and a most recent camera measurement. Any measurements from other sensors, such as RADAR, that are less than a threshold amount of time ahead of or behind the current time window can be extrapolated to a time in the current time window for comparison with camera measurements. The scheduling engine then sorts the selected measurements based on their timestamps in the current time window, and submits the selected measurements to the fusion system in sorted order. The system may then perform downstream operations, such as object tracking, using the sorted measurements.

Abstract: In various examples, systems and methods are disclosed relating to determining associations between objects represented in sensor data and predicted states of the objects in multi-sensor systems such as autonomous or semi-autonomous vehicle perception systems. Systems and methods are disclosed that employ neural network models, such as multi-layer perceptron (MLP) models or other deep neural network (DNN) models, in learning association costs between sensor measurements and predicted states of objects. During training, the systems and methods can generate data for updating parameters of the neural network models such that, during deployment, the neural network models can receive sensor data and predicted states, and provide corresponding association costs.

Abstract: In various examples, systems and methods for tracking objects and determining time-to-collision values associated with the objects are described. For instance, the systems and methods may use feature points associated with an object depicted in a first image and feature points associated with a second image to determine a scalar change associated with the object. The systems and methods may then use the scalar change to determine a translation associated with the object. Using the scalar change and the translation, the systems and methods may determine that the object is also depicted in the second image. The systems and methods may further use the scalar change and a temporal baseline to determine a time-to-collision associated with the object. After performing the determinations, the systems and methods may output data representing at least an identifier for the object, a location of the object, and/or the time-to-collision.