Abstract: A method of updating parameters of an occupancy estimation model. The method includes receiving images that are two-dimensional. An occupancy estimation model is utilized to generate a voxel based on the images. An occupancy loss is determined by comparing the voxel to an occupancy ground truth corresponding to the voxel. An object loss is determined by comparing the voxel to an object ground truth. The object loss is combined with the occupancy loss to determine a total loss for the voxel. Parameters of the occupancy estimation model are updated to reduce the total loss determined.

Abstract: Herein, a technology that facilitates the optimization of vision-language (VL) based classifiers with text embeddings is discussed. The technology includes tuning the VL-based classifier employing a pre-trained image encoder of a visual-language model (VLM) for imaging embedding of pre-classified images and a pre-trained textual encoder of the VLM for textual embedding of a set of differing textual sentences. The technology further includes determining an optimized set of differing textual sentences of a superset of textual sentences. The optimized set of differing textual sentences has a minimal classification loss of the VL-based classifier when classifying the pre-classified images.

Abstract: A method of training a disparity estimation network. The method includes obtaining an eye-gaze dataset having first images with at least one gaze direction associated with each of the first images. A gaze prediction neural network is trained based on the eye-gaze dataset to develop a model trained to provide a gaze prediction for an external image. A depth database is obtained that includes second images having depth information associated with each of the second images. A disparity estimation neural network for object detection is trained based on an output from the gaze prediction neural network and an output from the depth database.

Abstract: A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations. The operations include generating, at an occupancy estimation network, an occupancy probability at one or more voxels, predicting, via a gaze prediction model, a gaze direction of a gaze prediction, and identifying, based on the predicted gaze direction, an object of interest. The operations also include generating, via an occupancy estimation application, a gaze saliency map based on the gaze direction and identified object of interest, and updating, based on the determined gaze direction and the gaze saliency map, the occupancy probability of the one or more voxels.

Abstract: A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations. The operations include generating, at an occupancy estimation network, an occupancy probability at one or more voxels, predicting, via a gaze prediction model, a gaze direction of a gaze prediction, and identifying, based on the predicted gaze direction, an object of interest. The operations also include generating, via an occupancy estimation application, a gaze saliency map based on the gaze direction and identified object of interest, and updating, based on the determined gaze direction and the gaze saliency map, the occupancy probability of the one or more voxels.

Abstract: A multi-objective dense open-vocabulary system includes an image encoder and a classifier. The image encoder includes a summarization contrastive language image pre-training (CLIP) head trained on supervised losses from unlabeled and labeled image data. The summarization CLIP head loses open-vocabulary capabilities as capacity grows, and offsets the loss with pseudo-labels generated by a dense CLIP head. The summarization CLIP head is operational to receive captured images from a source device, and generate image embeddings based on current images. The classifier is operational to receive one or more targets from a text encoder, receive the plurality of image embeddings from the summarization CLIP, classify the plurality of image embeddings to identify one or more output images that contain the one or more targets, and present the one or more output images to the destination device.

Abstract: Examples described herein provide a method that includes receiving a first image captured by a camera of a vehicle at a first time t and receiving a second image captured by the camera of the vehicle at a second time t-1. The method further includes projecting each of a plurality of world voxels to the camera at the first time t and the second time t-1. The method further includes aggregating voxel features for the plurality of world voxels for the first image and the second image. The method further includes training an occupancy classifier using the aggregated voxel features.

Abstract: A system for image retrieval includes a processing device connected to a database configured to store a set of images. The processing device includes a computer vision model including a text encoder configured to extract textual features and a vision encoder configured to extract image features, and generate embeddings used for image retrieval tasks, and a summarization module configured to be trained using a targeted dataset, the summarization module configured to restrict a number of queries per image that are learnable by the computer vision model to a selected number.

Abstract: A system for classifying a road crossing intention of a pedestrian includes a processor including a pretrained image encoder generating an image embedding based upon an input image. The system further includes a remote server receiving the image embedding. The remote server device further references a plurality of pretrained image and text embeddings each corresponding to either a positive road crossing intention or a negative road crossing intention. The remote server device further determines a plurality of proximity values evaluating whether the input image is closer to the positive road crossing intention or the negative road crossing intention, evaluating the image embedding against each of the pretrained embeddings. The remote server device further classifies a road crossing intention of the pedestrian based upon the plurality of proximity values. The system further includes generates a road crossing intention output based upon the road crossing intention of the pedestrian.

Abstract: A method of updating parameters of an occupancy estimation model. The method includes receiving images that are two-dimensional. An occupancy estimation model is utilized to generate a voxel based on the images. An occupancy loss is determined by comparing the voxel to an occupancy ground truth corresponding to the voxel. An object loss is determined by comparing the voxel to an object ground truth. The object loss is combined with the occupancy loss to determine a total loss for the voxel. Parameters of the occupancy estimation model are updated to reduce the total loss determined.

Abstract: A method of analyzing includes obtaining a dataset having images of an area surrounding a vehicle and identifying at least one object in each image of the images. Eye-gaze information directed to an operator of the vehicle is obtained from an eye-gaze monitoring system. The eye-gaze information includes an eye-gaze direction of the operator corresponding to each of the images. A subset of images from the images is identified for performing additional data labeling based on a relationship between the eye-gaze direction to the at least one object identified in each image of the images.

Abstract: A method, system and vehicle that repetitively correct angle offsets in a synthetic aperture radar image of a vehicle while the vehicle is in motion by utilizing a radar system and a camera to determine accurate velocity of a measured object by matching angles of the object in the SAR image with angles of the object in the camera image, thereby reducing angle offsets of objects in the SAR image. The method includes obtaining an SAR image of another vehicle via a radar unit of the vehicle, obtaining a camera image of the other vehicle via a camera unit of the vehicle, determining an association between at least one object in the SAR image and a corresponding at least one object in the camera image, correcting a velocity estimation of the vehicle based on the determined association, and adjusting the SAR image based on the corrected velocity estimation.

Abstract: A system for a host vehicle operating on a road surface includes a polarimetric camera, a global positioning system (“GPS”) receiver, a compass, and an electronic control unit (“ECU”). The camera collects polarimetric image data of a drive scene, including a potential driving path on the road surface. The ECU receives the polarimetric image data, estimates the Sun location using the GPS receiver and compass, and computes an ideal representation of the road surface using the Sun location. The ECU normalizes the polarimetric image data such that the road surface has a normalized representation in the drive scene, i.e., an angle of linear polarization (“AoLP”) and degree of linear polarization (“DoLP”) equal predetermined fixed values. The ECU executes a control action using the normalized representation.

Abstract: A detection system for a host vehicle includes a camera, global positioning system (“GPS”) receiver, compass, and electronic control unit (“ECU”). The camera collects polarimetric image data forming an imaged drive scene inclusive of a road surface illuminated by the Sun. The GPS receiver outputs a present location of the vehicle as a date-and-time-stamped coordinate set. The compass provides a directional heading of the vehicle. The ECU determines the Sun's location relative to the vehicle and camera using an input data set, including the present location and directional heading. The ECU also detects a specular reflecting area or areas on the road surface using the polarimetric image data and Sun's location, with the specular reflecting area(s) forming an output data set. The ECU then executes a control action aboard the host vehicle in response to the output data set.

Abstract: A system includes a transmitter of a radar system to transmit transmitted signals, and a receiver of the radar system to receive received signals based on reflection of one or more of the transmitted signals by one or more objects. The system also includes a processor to train a neural network with reference data obtained by simulating a higher resolution radar system than the radar system to obtain a trained neural network. The trained neural network enhances detection of the one or more objects based on obtaining and processing the received signals in a vehicle. One or more operations of the vehicle are controlled based on the detection of the one or more objects.

Abstract: A system in a vehicle includes an image sensor to obtain images in an image sensor coordinate system and a depth sensor to obtain point clouds in a depth sensor coordinate system. Processing circuitry implements a neural network to determine a validation state of a transformation matrix that transforms the point clouds in the depth sensor coordinate system to transformed point clouds in the image sensor coordinate system. The transformation matrix includes rotation parameters and translation parameters.

Abstract: A method of controlling a color display screen aboard a motor vehicle includes performing, via a host computer, a color calibration test of a user of the motor vehicle in which the user is subjected to a calibrated set of color-coded test information. The method includes receiving a color perception response of the user to the calibrated set of color-coded test information via the host computer. Additionally, the method includes mapping a reduced visual gamut of the user via the host computer using the color perception response, and then commanding adjustment of user-specific color settings of the motor vehicle using the reduced visual gamut to thereby accommodate a color perception deficiency of the user.

Abstract: A method for deblurring a blurred image includes dividing the blurred image into overlapping regions each having a size and an offset from neighboring overlapping regions along a first direction as determined by a period of a ringing artifact in the blurred image, or by obtained blur characteristics relating to the blurred image and/or attributable to the optical system, or by a detected cause capable of producing the blur characteristics, stacking the overlapping regions to produce a stacked output, wherein the overlapping regions are sequentially organized along the first direction, convolving the stacked output through a first convolutional neural network (CNN) to produce a first CNN output having reduced blur as compared to the stacked output, and assembling the first CNN output into a re-assembled image, and processing the re-assembled image through a second CNN to produce a deblurred image having reduced residual artifacts as compared to the re-assembled image.

Abstract: Herein, a technology that facilitates the optimization of vision-language (VL) based classifiers with text embeddings is discussed. The technology includes tuning the VL-based classifier employing a pre-trained image encoder of a visual-language model (VLM) for imaging embedding of pre-classified images and a pre-trained textual encoder of the VLM for textual embedding of a set of differing textual sentences. The technology further includes determining an optimized set of differing textual sentences of a superset of textual sentences. The optimized set of differing textual sentences has a minimal classification loss of the VL-based classifier when classifying the pre-classified images.

Abstract: A method for open-vocabulary query-based dense retrieval is provided. The method includes monitoring camera data including an image related to an object and referencing a set of queries, each of the queries describing a candidate object to be updated by a remote server device. An encoder of an open-vocabulary pre-trained vision-language model system is utilized to initialize a predefined embedding for each query, and a classifier is initialized by mapping the predefined embeddings to weights of the classifier. The method further includes applying a dense open-vocabulary image encoder on the camera data to create a mass of dense embeddings including a set of spatially-arranged embeddings for the image, each including a matrix including embedding vectors. The classifier is utilized by applying the classifier to the plurality of embedding vectors to classify the object within the operating environment as an identified object. The method further includes publishing the identified object.