Abstract: Systems, methods and non-transitory computer-readable media are presented. These describe operations comprising obtaining, visual data acquired by a first image sensor and associated with a traffic scene and obtaining, audio data acquired by a first audio sensor and associated with the traffic scene. The first audio sensor and the first image sensor are mutually independent. The operations further comprise determining, based at least in part on the visual data and the audio data, embeddings for the traffic scene that represent a joint representation space for the different modalities; and determining, by a first machine learning model, and based at least in part on the embeddings, presence of an emergency vehicle in the traffic scene.

Abstract: A machine-learned architecture may use sensor data, data derived from the sensor data, and/or to determine a grid indicating objects that are occluded to one or more sensors of a vehicle and velocities associated therewith. This grid may be used to generate a candidate object detection associated with an occluded object and/or an object track associated with the occluded object. The candidate object detection and/or object track may be used to determine a predicted trajectory of the occluded object and, in cases where the occluded object is affirmed by a perception component of the vehicle to be a true positive, may be used to initialize a track associated with the newly disoccluded object.