Abstract: A vehicle system includes at least one sensor configured to capture one or more images of a pedestrian positioned near a road, and a control module in communication with the at least one sensor. The control module is configured to determine one or more characteristics associated with the pedestrian positioned near to the road based on the one or more captured images, generate a trajectory prediction for the pedestrian, overlay the trajectory prediction for the pedestrian on a road segmentation of the road, and generate a road crossing prediction for the pedestrian based on the one or more characteristics and the overlayed trajectory prediction. The road crossing prediction forecasts whether the pedestrian will be on or off the road. Other example vehicle systems and methods for forecasting a future presence of a pedestrian on a road are also disclosed.

Abstract: A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations. The operations include training, based on a training distribution of a prediction model, a denoiser, the denoiser being a neural network, receiving an original distribution set including an image and image annotations, and executing, on the image and the image annotations, forward diffusion to define a noisy distribution set including a noisy image and noisy image annotations. The operations also include cleaning, by the trained denoiser, the noisy distribution set to define a cleaned distribution set including a cleaned image and cleaned image annotations, determining, based on a comparison of the cleaned distribution set with the original distribution set, a denoiser loss value, and generating, based on the denoiser loss value, a joint probability.

Abstract: A system includes: a traffic object detection module detecting traffic objects in an environment; an attention map highlighting module generating an attention map, highlighting relevant ones of the traffic objects or regions in which the relevant ones of the traffic objects are located; an image encoder, based on the attention map, encoding an image of the environment and generating an image embedding vector; a PLM module iteratively selecting and appending text to create a text message including selecting the text based on a score, the text message being a specific description of what is perceived in the environment; a text encoder encoding a portion of the text message created thus far to generate a text embedding vector; and a module, based on the image and text embedding vectors, to score the portion to generate the score, where the PLM module is configured to update the portion based on the score.

Abstract: Herein, a technology that facilitates the optimization of vision-language (VL) based classifiers with text embeddings is discussed. The technology includes tuning the VL-based classifier employing a pre-trained image encoder of a visual-language model (VLM) for imaging embedding of pre-classified images and a pre-trained textual encoder of the VLM for textual embedding of a set of differing textual sentences. The technology further includes determining an optimized set of differing textual sentences of a superset of textual sentences. The optimized set of differing textual sentences has a minimal classification loss of the VL-based classifier when classifying the pre-classified images.

Abstract: A system and method for operating a host vehicle. A first detection of a first reflection point from an object is received during a first time frame of a radar. A first position and a first Doppler frequency of the first detection are direction. The first position is updated to a first predicted position in a second time frame using the first Doppler frequency. Updating includes using an object-based component of the first Doppler frequency to shift the first detection from the first position to an intermediate position in the second time frame and using a vehicle-based component of the first Doppler frequency to shift the first detection from the intermediate position to the first predicted position. The prediction position is aggregated with a second detection of a second reflection point from the object, and the object is detected from the aggregation.

Abstract: A system for classifying a road crossing intention of a pedestrian includes a processor including a pretrained image encoder generating an image embedding based upon an input image. The system further includes a remote server receiving the image embedding. The remote server device further references a plurality of pretrained image and text embeddings each corresponding to either a positive road crossing intention or a negative road crossing intention. The remote server device further determines a plurality of proximity values evaluating whether the input image is closer to the positive road crossing intention or the negative road crossing intention, evaluating the image embedding against each of the pretrained embeddings. The remote server device further classifies a road crossing intention of the pedestrian based upon the plurality of proximity values. The system further includes generates a road crossing intention output based upon the road crossing intention of the pedestrian.

Abstract: A method includes performing object detection by a radar sensor system. The object detection includes generating and transmitting object-detection (OD) chirps on an OD channel of a plurality of radar channels employed by the radar sensor system. The method further includes monitoring unused radar channels for interference and determining that at least one of the unused radar channels is free of interference. The method still further includes performing object detection by generating and transmitting one or more OD chirps on the determined interference-free channel.

Abstract: A system of controlling operation of a vehicle includes a lidar unit and a radar unit configured to obtain measured lidar datapoints and a measured radar signal, respectively. A command unit is adapted to receive the measured lidar datapoints and the measured radar signal, the command unit including a processor and tangible, non-transitory memory on which instructions are recorded. The command unit is configured to identify respective objects in the measured lidar datapoints and assign a respective radar reflection intensity to the measured lidar datapoints in the respective objects. A synthetic radar signal is generated based in part on the radar reflection intensity. The command unit is configured to obtain an enhanced radar signal by adjusting the measured radar signal based on the synthetic radar reference signal.

Abstract: A system in a vehicle includes a transmitter of a radar system to transmit energy and a receiver of the radar system to receive reflections based on reflection of the energy transmitted by the transmitter by one or more objects. The system also includes a controller to process the reflections obtained during a predefined duration to estimate a range, azimuth angle, and relative velocity for each of the one or more objects by obtaining acceleration information from an inertial measurement unit (IMU) of the vehicle. An operation of the vehicle is controlled based on information about the one or more objects from the radar system.

Abstract: A vehicle system includes at least one sensor configured to capture one or more images of a pedestrian positioned near a road, and a control module in communication with the at least one sensor. The control module is configured to determine one or more characteristics associated with the pedestrian positioned near to the road based on the one or more captured images, generate a trajectory prediction for the pedestrian, overlay the trajectory prediction for the pedestrian on a road segmentation of the road, and generate a road crossing prediction for the pedestrian based on the one or more characteristics and the overlayed trajectory prediction. The road crossing prediction forecasts whether the pedestrian will be on or off the road. Other example vehicle systems and methods for forecasting a future presence of a pedestrian on a road are also disclosed.

Abstract: A method of performing doppler spectrum encoding for radar object detection. The method includes receiving a radar tensor including a doppler for range and angle combinations. An encoded vector is generated for each of the range and angle combinations. The encoded vector includes a reflection intensity, a frequency of the reflection intensity, and a width of a reflection intensity peak for a predetermined number of greatest reflection intensity peaks for each of the range and angle combinations. An encoded vector dataset is generated from the encoded vectors. Feature extraction is performed on the encoded vector dataset with a feature extraction neural network to generate an extracted feature dataset with extracted feature information for each of the range and angle combinations. Object detection is performed on the extracted feature dataset with an object detection neural network to generate an object detection output for identifying objects.

Abstract: A system for enhancing object detection includes at least one radar device configured to detect signals reflected by objects and a control module. The control module is configured to generate a radar spectrum based on the signals detected by the at least one radar device, partition the radar spectrum into static reflections and dynamic reflections separate from the static reflections, extract features from the static reflections with a first machine learning module, extract features from the dynamic reflections with a second machine learning module different than the first machine learning module, merge the extracted features from the static reflections and the extracted features from the dynamic reflections, and detect static objects and dynamic objects based on the merged features from the static reflections and the dynamic reflections. Other example systems and methods for enhancing object detection are also disclosed.

Abstract: A method that includes obtaining reflective radar signals regarding a scene monitored by a radar sensor system having an antenna array that is characterized by effecting a reflection-ghost offset in one or more domains, determining a reflective-intensity (RI) spectrum in three domains based on the reflective radar signals, producing a filtered RI spectrum by applying a trained convolutional neural network (CNN) to the RI spectrum by, at least in part, filtering the RI spectrum using one or more CNN kernels that incorporate the reflection-ghost offset; and detecting objects in the monitored scene based, at least in part, on the filtered RI spectrum.

Abstract: A method, system and vehicle that repetitively correct angle offsets in a synthetic aperture radar image of a vehicle while the vehicle is in motion by utilizing a radar system and a camera to determine accurate velocity of a measured object by matching angles of the object in the SAR image with angles of the object in the camera image, thereby reducing angle offsets of objects in the SAR image. The method includes obtaining an SAR image of another vehicle via a radar unit of the vehicle, obtaining a camera image of the other vehicle via a camera unit of the vehicle, determining an association between at least one object in the SAR image and a corresponding at least one object in the camera image, correcting a velocity estimation of the vehicle based on the determined association, and adjusting the SAR image based on the corrected velocity estimation.

Abstract: A system includes: a traffic object detection module detecting traffic objects in an environment; an attention map highlighting module generating an attention map, highlighting relevant ones of the traffic objects or regions in which the relevant ones of the traffic objects are located; an image encoder, based on the attention map, encoding an image of the environment and generating an image embedding vector; a PLM module iteratively selecting and appending text to create a text message including selecting the text based on a score, the text message being a specific description of what is perceived in the environment; a text encoder encoding a portion of the text message created thus far to generate a text embedding vector; and a module, based on the image and text embedding vectors, to score the portion to generate the score, where the PLM module is configured to update the portion based on the score.

Abstract: A system includes a transmitter of a radar system to transmit transmitted signals, and a receiver of the radar system to receive received signals based on reflection of one or more of the transmitted signals by one or more objects. The system also includes a processor to train a neural network with reference data obtained by simulating a higher resolution radar system than the radar system to obtain a trained neural network. The trained neural network enhances detection of the one or more objects based on obtaining and processing the received signals in a vehicle. One or more operations of the vehicle are controlled based on the detection of the one or more objects.

Abstract: A method for controlling automated vehicle acceleration and braking includes obtaining an image using at least one vehicle camera of a host vehicle, extracting machine learning model feature inputs based on the obtained image, detecting one or more objects in the obtained image, the one or more objects including at least one pedestrian, assigning attention weights to regions of the obtained image according to locations of the one or more objects in the obtained image, combining the attention weights with corresponding ones of the machine learning model feature inputs according to the regions of the obtained image, executing a machine learning model to generate a crossing intention prediction output associated with the at least one pedestrian, and in response to the crossing intention prediction output exceeding a crossing intention threshold, controlling automatic braking of the host vehicle according to a location of the at least one pedestrian.

Abstract: Herein, a technology that facilitates the optimization of vision-language (VL) based classifiers with text embeddings is discussed. The technology includes tuning the VL-based classifier employing a pre-trained image encoder of a visual-language model (VLM) for imaging embedding of pre-classified images and a pre-trained textual encoder of the VLM for textual embedding of a set of differing textual sentences. The technology further includes determining an optimized set of differing textual sentences of a superset of textual sentences. The optimized set of differing textual sentences has a minimal classification loss of the VL-based classifier when classifying the pre-classified images.

Abstract: A system for classifying a road crossing intention of a pedestrian includes a processor including a pretrained image encoder generating an image embedding based upon an input image. The system further includes a remote server receiving the image embedding. The remote server device further references a plurality of pretrained image and text embeddings each corresponding to either a positive road crossing intention or a negative road crossing intention. The remote server device further determines a plurality of proximity values evaluating whether the input image is closer to the positive road crossing intention or the negative road crossing intention, evaluating the image embedding against each of the pretrained embeddings. The remote server device further classifies a road crossing intention of the pedestrian based upon the plurality of proximity values. The system further includes generates a road crossing intention output based upon the road crossing intention of the pedestrian.

Abstract: A system of controlling operation of a vehicle includes one or more sensors operatively connected to the vehicle. The sensors are configured to obtain respective data of a scene and include a radar unit. A command unit is adapted to receive the respective data and includes a processor and tangible, non-transitory memory on which instructions are recorded. The command unit is configured to determine an orientation angle of a pedestrian in the scene, a Doppler frequency of the pedestrian and a distance of the pedestrian from a border of a road, based in part on the respective data. The orientation angle is based on a heading of the pedestrian relative to a direction of a road. The command unit is configured to designate a status of the pedestrian as either crossing or not crossing based on the distance, the orientation angle and the Doppler frequency of the pedestrian.