Abstract: A fine-tuned model for few-shot object detection is output. A dataset of K-shot classes is created for fine-tuning a pretrained vision language model (VLM). Concept alignment is performed between the dataset of K-shot classes and the VLM. Fine-tuning is performed on the VLM using the dataset of K-shot classes with pseudo-negative federated loss to generate a few-shot object detection (FSOD) model. The FSOD model is output for use in object detection of the K-shot classes in image data received from one or more sensors.

Abstract: Methods for training a series of neural networks to output driving behavior control parameters is disclosed. The training dataset for the neural networks includes sensor-based vehicle driving recordings that may be categorized by geographical area, by qualitative driving behaviors, or by some combination, such that various training data subsets are used to train the series of neural networks. By learning either city-specific driving behavior control parameters, qualitative driving behavior specific driving behavior control parameters, or both, the resulting parameters may then be provided to a motion planning model for use in modeling predictive control for an autonomous vehicle. Rather than relying on XYZ trajectories of agent vehicles when planning future trajectories of the ego vehicle, the motion planning model is adaptive, due to the use of the learned driving behavior control parameters.

Abstract: Systems and methods for simultaneous map dynamic object reconstruction using LIDAR are disclosed. A method includes generating point cloud data of an environment using a LIDAR system, and generating annotated frames based thereon, the first and second frames corresponding to first and second time points at a particular direction of the LIDAR. Intermediate frames between the first and second annotated frames are generated, and coordinate frame transformations are conducted for objects within the frames to determine respective positions and orientations. First and second optimizations are performed for a mesh of a three-dimensional space and positions/orientations within the space. The dynamic scene is reconstructed based on the optimizations.

Abstract: Systems and methods of determining trajectories of an actor in an environment in which a vehicle is operating are provided. The method includes detecting an actor that may move within a scene in the environment by an object detection system of a vehicle in the environment, determining a kinematic history of the actor, and using context of the scene and the kinematic history of the actor to determine a plurality of reference polylines for the actor. The method further includes generating a contextual embedding of the kinematic history of the actor to generate a plurality of predicted trajectories of the actor, in which the generating conditions each of the predicted trajectories to correspond to one of the reference polylines. The method additionally includes using, by the vehicle, the plurality of predicted trajectories to plan movement of the vehicle.

Abstract: A method of generating data for machine learning (ML) models includes receiving, from one or more sensors, a sequence of samples that includes time-stamp information, extracting from the sequence of samples a snippet of a pre-defined length to generate a training dataset that includes a target sample derived from the sequence of samples, fine-tuning a pre-trained diffusion model to condition based on a context sample associated with the sequence of samples and corresponding time-stamp information, wherein the context sample associated with the sequence of samples is less than all samples of the sequence of samples, and in response to the fine-tuning the pre-trained diffusion model to reach convergence, outputting a final-predicted sample associated with the target sample, wherein the final-predicted sample was not in the sequence of samples.

Abstract: Systems and methods of determining trajectories of an actor in an environment in which a vehicle is operating are provided. The method includes, by an object detection system of a vehicle in an environment, detecting an actor that may move within a scene in the environment. The method further includes using context of the scene to determine a reference polyline for the actor and determining a kinematic history of the actor. The method additionally includes using the kinematic history to predict a waypoint, which is a predicted position of the actor at a conclusion of a waypoint time period, and identifying a segment of the reference polyline, the segment extending from a current location to a point along the reference polyline that is closest to the waypoint and determining a trajectory for the actor conditioned by the segment of the reference polyline.

Abstract: A method is disclosed for evaluating a classifier used to determine a traffic light signal state in images. The method includes, by a computer vision system of a vehicle, receiving at least one image of a traffic signal device of an imminent intersection. The traffic signal device includes a traffic signal face including one or more traffic signal elements. The method includes classifying, by a traffic light classifier (TLC), a classification state of the traffic signal face using labeled images correlated to the received at least one image. The classification state controls an operation of the vehicle at the intersection. The method includes evaluating a performance of the classifying of the classification state generated by the TLC. The evaluation is a label-free performance evaluation based on unlabeled images. The method includes training the TLC based on the evaluated performance.

Abstract: Various embodiments of a framework which allow, as an alternative to resource-taxing decompression, efficient computation of feature maps using a compressed content data subset, such as video, by exploiting the motion information, such as a motion vector, present in the compressed video. This framework allows frame-specific object recognition and action detection algorithms to be applied to compressed video and other media files by executing only on I-frames in a Group of Pictures and linearly interpolating the results. Training and machine learning increases recognition accuracy. Yielding significant computational gains, this approach accelerates frame-wise feature extraction I-frame/P-frame/P-frame videos as well as I-frame/P-frame/B-frame videos. The present techniques may also be used for segmentation to identify and label respective regions for objects in a video.

Abstract: Systems and methods for siren detection in a vehicle are provided. A method includes recording an audio segment, using a first audio recording device coupled to an autonomous vehicle, separating, using a computing device coupled to the autonomous vehicle, the audio segment into one or more audio clips, generating a spectrogram of the one or more audio clips, and inputting each spectrogram into a Convolutional Neural Network (CNN) run on the computing device. The CNN may be pretrained to detect one or more sirens present in spectrographic data. The method further includes determining, using the CNN, whether a siren is present in the audio segment, and if the siren is determined to be present in the audio segment, determining a course of action of the autonomous vehicle.

Abstract: Systems and methods of determining trajectories of an actor in an environment in which a vehicle is operating are provided. The method includes, by an object detection system of a vehicle in an environment, detecting an actor that may move within a scene in the environment. The method further includes using context of the scene to determine a reference polyline for the actor and determining a kinematic history of the actor. The method additionally includes using the kinematic history to predict a waypoint, which is a predicted position of the actor at a conclusion of a waypoint time period, and identifying a segment of the reference polyline, the segment extending from a current location to a point along the reference polyline that is closest to the waypoint and determining a trajectory for the actor conditioned by the segment of the reference polyline.

Abstract: Systems and methods of determining trajectories of an actor in an environment in which a vehicle is operating are provided. The method includes detecting an actor that may move within a scene in the environment by an object detection system of a vehicle in the environment, determining a kinematic history of the actor, and using context of the scene and the kinematic history of the actor to determine a plurality of reference polylines for the actor. The method further includes generating a contextual embedding of the kinematic history of the actor to generate a plurality of predicted trajectories of the actor, in which the generating conditions each of the predicted trajectories to correspond to one of the reference polylines. The method additionally includes using, by the vehicle, the plurality of predicted trajectories to plan movement of the vehicle.

Abstract: A method is disclosed for evaluating a classifier used to determine a traffic light signal state in images. The method includes, by a computer vision system of a vehicle, receiving at least one image of a traffic signal device of an imminent intersection. The traffic signal device includes a traffic signal face including one or more traffic signal elements. The method includes classifying, by a traffic light classifier (TLC), a classification state of the traffic signal face using labeled images correlated to the received at least one image. The classification state controls an operation of the vehicle at the intersection. The method includes evaluating a performance of the classifying of the classification state generated by the TLC. The evaluation is a label-free performance evaluation based on unlabeled images. The method includes training the TLC based on the evaluated performance.

Abstract: Various embodiments of a framework which allow, as an alternative to resource-taxing decompression, efficient computation of feature maps using a compressed content data subset, such as video, by exploiting the motion information, such as a motion vector, present in the compressed video. This framework allows frame-specific object recognition and action detection algorithms to be applied to compressed video and other media files by executing only on I-frames in a Group of Pictures and linearly interpolating the results. Training and machine learning increases recognition accuracy. Yielding significant computational gains, this approach accelerates frame-wise feature extraction I-frame/P-frame/P-frame videos as well as I-frame/P-frame/B-frame videos. The present techniques may also be used for segmentation to identify and label respective regions for objects in a video.

Abstract: Various embodiments of a framework which allow, as an alternative to resource-taxing decompression, efficient computation of feature maps using a compressed content data subset, such as video, by exploiting the motion information, such as a motion vector, present in the compressed video. This framework allows frame-specific object recognition and action detection algorithms to be applied to compressed video and other media files by executing only on I-frames in a Group of Pictures and linearly interpolating the results. Training and machine learning increases recognition accuracy. Yielding significant computational gains, this approach accelerates frame-wise feature extraction I-frame/P-frame/P-frame videos as well as I-frame/P-frame/B-frame videos. The present techniques may also be used for segmentation to identify and label respective regions for objects in a video.

Abstract: Various embodiments of a framework which allow, as an alternative to resource-taxing decompression, efficient computation of feature maps using a compressed content data subset, such as video, by exploiting the motion information, such as a motion vector, present in the compressed video. This framework allows frame-specific object recognition and action detection algorithms to be applied to compressed video and other media files by executing only on I-frames in a Group of Pictures and linearly interpolating the results. Training and machine learning increases recognition accuracy. Yielding significant computational gains, this approach accelerates frame-wise feature extraction I-frame/P-frame/P-frame videos as well as I-frame/P-frame/B-frame videos. The present techniques may also be used for segmentation to identify and label respective regions for objects in a video.