Abstract: A method for training a neural network including receiving a plurality of images of a driver's field of view, generating a depth information, a driver's gaze probability and a known object indication for each of the plurality of images, estimating a probability of an unknown object within each of the plurality of images in response to the depth information, the driver's gaze probability and the known object indication, generating a plurality of annotated images in response to annotating each of the plurality of images having the probability of the unknown object exceeding a threshold probability to identify the unknown object, wherein each of the plurality of annotated images is annotated to identify the unknown object, and training the neural network in response to the plurality of annotated images.

Abstract: A method of updating parameters of an occupancy estimation model. The method includes receiving images that are two-dimensional. An occupancy estimation model is utilized to generate a voxel based on the images. An occupancy loss is determined by comparing the voxel to an occupancy ground truth corresponding to the voxel. An object loss is determined by comparing the voxel to an object ground truth. The object loss is combined with the occupancy loss to determine a total loss for the voxel. Parameters of the occupancy estimation model are updated to reduce the total loss determined.

Abstract: A system for measuring vehicle parameters includes an inertial measurement unit configured to measure an inertial parameter of a vehicle, and a virtual measurement unit configured to receive a plurality of measured parameters from one or more other sensors of the vehicle, and emulate the inertial parameter by combining the plurality of measured parameters. The system also includes a controller configured to control vehicle operation based on at least one of the measured inertial parameter and the emulated inertial parameter.

Abstract: A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations. The operations include generating, at an occupancy estimation network, an occupancy probability at one or more voxels, predicting, via a gaze prediction model, a gaze direction of a gaze prediction, and identifying, based on the predicted gaze direction, an object of interest. The operations also include generating, via an occupancy estimation application, a gaze saliency map based on the gaze direction and identified object of interest, and updating, based on the determined gaze direction and the gaze saliency map, the occupancy probability of the one or more voxels.

Abstract: A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations. The operations include generating, at an occupancy estimation network, an occupancy probability at one or more voxels, predicting, via a gaze prediction model, a gaze direction of a gaze prediction, and identifying, based on the predicted gaze direction, an object of interest. The operations also include generating, via an occupancy estimation application, a gaze saliency map based on the gaze direction and identified object of interest, and updating, based on the determined gaze direction and the gaze saliency map, the occupancy probability of the one or more voxels.

Abstract: Examples described herein provide a method that includes receiving a first image captured by a camera of a vehicle at a first time t and receiving a second image captured by the camera of the vehicle at a second time t-1. The method further includes projecting each of a plurality of world voxels to the camera at the first time t and the second time t-1. The method further includes aggregating voxel features for the plurality of world voxels for the first image and the second image. The method further includes training an occupancy classifier using the aggregated voxel features.

Abstract: A method of updating parameters of an occupancy estimation model. The method includes receiving images that are two-dimensional. An occupancy estimation model is utilized to generate a voxel based on the images. An occupancy loss is determined by comparing the voxel to an occupancy ground truth corresponding to the voxel. An object loss is determined by comparing the voxel to an object ground truth. The object loss is combined with the occupancy loss to determine a total loss for the voxel. Parameters of the occupancy estimation model are updated to reduce the total loss determined.

Abstract: A vehicle control system for camera-based vehicle navigation includes at least one vehicle camera configured to capture an image of a front view from a vehicle, a global positioning system (GPS) receiver configured to obtain a current location of the vehicle, a vehicle user interface including a display, and a vehicle control module. The vehicle control module is configured to obtain the current location of the vehicle via the GPS receiver, identify a sequence of vehicle navigation steps to a target destination, capture the image via the at least one vehicle camera, process the image with a machine learning model to detect multiple objects in the image, rank the multiple objects according to landmark scoring criteria indicative of an object recognition likelihood by a driver of the vehicle, and display a highest ranked one of the multiple objects in association with a next vehicle navigation step.

Abstract: A system for a host vehicle operating on a road surface includes a polarimetric camera, a global positioning system (“GPS”) receiver, a compass, and an electronic control unit (“ECU”). The camera collects polarimetric image data of a drive scene, including a potential driving path on the road surface. The ECU receives the polarimetric image data, estimates the Sun location using the GPS receiver and compass, and computes an ideal representation of the road surface using the Sun location. The ECU normalizes the polarimetric image data such that the road surface has a normalized representation in the drive scene, i.e., an angle of linear polarization (“AoLP”) and degree of linear polarization (“DoLP”) equal predetermined fixed values. The ECU executes a control action using the normalized representation.

Abstract: A system for vehicle trajectory planning. The system may include a polarized camera system configured to generate polarized images of a roadway to be driven over with a vehicle, a prior controller configured to generate a prior for the roadway based at least in part on the polarized images, and a driving assistance system configured to provide a driving assistance according to the prior.

Abstract: A detection system for a host vehicle includes a camera, global positioning system (“GPS”) receiver, compass, and electronic control unit (“ECU”). The camera collects polarimetric image data forming an imaged drive scene inclusive of a road surface illuminated by the Sun. The GPS receiver outputs a present location of the vehicle as a date-and-time-stamped coordinate set. The compass provides a directional heading of the vehicle. The ECU determines the Sun's location relative to the vehicle and camera using an input data set, including the present location and directional heading. The ECU also detects a specular reflecting area or areas on the road surface using the polarimetric image data and Sun's location, with the specular reflecting area(s) forming an output data set. The ECU then executes a control action aboard the host vehicle in response to the output data set.

Abstract: A system in a vehicle includes an image sensor to obtain images in an image sensor coordinate system and a depth sensor to obtain point clouds in a depth sensor coordinate system. Processing circuitry implements a neural network to determine a validation state of a transformation matrix that transforms the point clouds in the depth sensor coordinate system to transformed point clouds in the image sensor coordinate system. The transformation matrix includes rotation parameters and translation parameters.

Abstract: A disclosure is presented for estimating height of an object using ultrasonic sensors carried onboard a vehicle. The height may be estimated by generating ultrasonic distance measurements based on a time of flight associated with ultrasonic reflections detected with the ultrasonic sensors, generating a three-dimensional (3D) occupancy grid to volumetrically represent at least a portion of an environment within field of view of the ultrasonic sensors, the 3D occupancy grid including a plurality of volumetric pixels (voxels), assigning a count value to each of the voxels based on the distance measurements, and estimating the height according to a relative spatial relationship between the vehicle and the voxel having the count value with a greatest value.

Abstract: A system for a host vehicle operating on a road surface includes a polarimetric camera, a global positioning system (“GPS”) receiver, a compass, and an electronic control unit (“ECU”). The camera collects polarimetric image data of a drive scene, including a potential driving path on the road surface. The ECU receives the polarimetric image data, estimates the Sun location using the GPS receiver and compass, and computes an ideal representation of the road surface using the Sun location. The ECU normalizes the polarimetric image data such that the road surface has a normalized representation in the drive scene, i.e., an angle of linear polarization (“AoLP”) and degree of linear polarization (“DoLP”) equal predetermined fixed values. The ECU executes a control action using the normalized representation.

Abstract: A detection system for a host vehicle includes a camera, global positioning system (“GPS”) receiver, compass, and electronic control unit (“ECU”). The camera collects polarimetric image data forming an imaged drive scene inclusive of a road surface illuminated by the Sun. The GPS receiver outputs a present location of the vehicle as a date-and-time-stamped coordinate set. The compass provides a directional heading of the vehicle. The ECU determines the Sun's location relative to the vehicle and camera using an input data set, including the present location and directional heading. The ECU also detects a specular reflecting area or areas on the road surface using the polarimetric image data and Sun's location, with the specular reflecting area(s) forming an output data set. The ECU then executes a control action aboard the host vehicle in response to the output data set.

Abstract: A system for vehicle trajectory planning. The system may include a polarized camera system configured to generate polarized images of a roadway to be driven over with a vehicle, a prior controller configured to generate a prior for the roadway based at least in part on the polarized images, and a driving assistance system configured to provide a driving assistance according to the prior.

Abstract: A free space estimation and visualization system for a host vehicle includes a camera configured to collect red-green-blue (“RGB”)-polarimetric image data of drive environs of the host vehicle, including a potential driving path. An electronic control unit (“ECU”) receives the RGB-polarimetric image data and estimates free space in the driving path by processing the RGB-polarimetric image data via a run-time neural network. Control actions are taken in response to the estimated free space. A method for use with the visualization system includes collecting RGB and lidar data of target drive scenes and generating, via a first neural network, pseudo-labels of the scenes. The method includes collecting RGB-polarimetric data via a camera and thereafter training a second neural network using the RGB-polarimetric data and pseudo-labels. The second neural network is used in the ECU to estimate free space in the potential driving path.

Abstract: Presented are intelligent vehicle systems for off-road driving incident prediction and assistance, methods for making/operating such systems, and vehicles networking with such systems. A method for operating a motor vehicle includes a system controller receiving geolocation data indicating the vehicle is in or entering off-road terrain. Responsive to the vehicle geolocation data, the controller receives, from vehicle-mounted cameras, camera-generated images each containing the vehicle's drive wheel(s) and/or the off-road terrain's surface. The controller receives, from a controller area network bus, vehicle operating characteristics data and vehicle dynamics data for the motor vehicle. The camera data, vehicle operating characteristics data, and vehicle dynamics data is processed via a convolutional neural network backbone to predict occurrence of a driving incident on the off-road terrain within a prediction time horizon.

Abstract: Methods, systems, and apparatuses to build an explainable user output to receive input feature data by a neural network of multiple layers of an original classifier; determine a semantic function to label data samples with semantic categories; determine a semantic accuracy for each layer of the original classifier within the neural network; compare each layer based on results from the comparison of the semantic accuracy; designate a layer based on an amount of computed semantic accuracy; extend the designated layer by a category branch to the neural network to extract semantic data samples from the semantic content to train a set of new connections of an explainable classifier to compute a set of output explanations with an accuracy measure associated each output explanation for each semantic category of the plurality of semantic categories, and compare the accuracy measure for each output explanation to generate the output explanation in a user understandable format.

Abstract: Vision-based airbag enablement may include capturing two-dimensional images of a passenger, segmenting the image, classifying the image, and determining seated height of the passenger from the image. Enabling or disabling deployment of the airbag may be controlled based at least in part upon the determined seated height.