Abstract: A method of training a disparity estimation network. The method includes obtaining an eye-gaze dataset having first images with at least one gaze direction associated with each of the first images. A gaze prediction neural network is trained based on the eye-gaze dataset to develop a model trained to provide a gaze prediction for an external image. A depth database is obtained that includes second images having depth information associated with each of the second images. A disparity estimation neural network for object detection is trained based on an output from the gaze prediction neural network and an output from the depth database.

Abstract: A system includes a transmitter of a radar system to transmit transmitted signals, and a receiver of the radar system to receive received signals based on reflection of one or more of the transmitted signals by one or more objects. The system also includes a processor to train a neural network with reference data obtained by simulating a higher resolution radar system than the radar system to obtain a trained neural network. The trained neural network enhances detection of the one or more objects based on obtaining and processing the received signals in a vehicle. One or more operations of the vehicle are controlled based on the detection of the one or more objects.

Abstract: A method, system and vehicle that repetitively correct angle offsets in a synthetic aperture radar image of a vehicle while the vehicle is in motion by utilizing a radar system and a camera to determine accurate velocity of a measured object by matching angles of the object in the SAR image with angles of the object in the camera image, thereby reducing angle offsets of objects in the SAR image. The method includes obtaining an SAR image of another vehicle via a radar unit of the vehicle, obtaining a camera image of the other vehicle via a camera unit of the vehicle, determining an association between at least one object in the SAR image and a corresponding at least one object in the camera image, correcting a velocity estimation of the vehicle based on the determined association, and adjusting the SAR image based on the corrected velocity estimation.

Abstract: A system for analyzing images includes a processing device includes a receiving module configured to receive an image, and an analysis module configured to apply the received image to a machine learning network and classify one or more features in the received image, the machine learning network configured to propagate image data through a plurality of convolutional layers, each convolutional layer of the plurality of convolutional layers including a plurality of filter channels, the machine learning network including a bottleneck layer configured to recognize an image feature based on a shape of an image component, The system also includes an output module configured to output characterization data that includes a classification of the one or more features.

Abstract: A method for deblurring a blurred image includes dividing the blurred image into overlapping regions each having a size and an offset from neighboring overlapping regions along a first direction as determined by a period of a ringing artifact in the blurred image, or by obtained blur characteristics relating to the blurred image and/or attributable to the optical system, or by a detected cause capable of producing the blur characteristics, stacking the overlapping regions to produce a stacked output, wherein the overlapping regions are sequentially organized along the first direction, convolving the stacked output through a first convolutional neural network (CNN) to produce a first CNN output having reduced blur as compared to the stacked output, and assembling the first CNN output into a re-assembled image, and processing the re-assembled image through a second CNN to produce a deblurred image having reduced residual artifacts as compared to the re-assembled image.

Abstract: A vehicle having an imaging sensor that is arranged to monitor a field-of-view (FOV) that includes a travel surface proximal to the vehicle is described. Detecting the travel lane includes capturing a FOV image of a viewable region of the travel surface. The FOV image is converted, via an artificial neural network, to a plurality of feature maps. The feature maps are projected, via an inverse perspective mapping algorithm, onto a BEV orthographic grid. The feature maps include travel lane segments and feature embeddings, and the travel lane segments are represented as line segments. The line segments are concatenated for the plurality of grid sections based upon the feature embeddings to form a predicted lane. The concatenation, or clustering is accomplished via the feature embeddings.

Abstract: A system in a vehicle includes an image sensor to obtain images in an image sensor coordinate system and a depth sensor to obtain point clouds in a depth sensor coordinate system. Processing circuitry implements a neural network to determine a validation state of a transformation matrix that transforms the point clouds in the depth sensor coordinate system to transformed point clouds in the image sensor coordinate system. The transformation matrix includes rotation parameters and translation parameters.

Abstract: A system and method to adjust an augmented reality (AR) image involve a front-facing camera in a vehicle. The camera obtains an image that includes a road surface ahead of the vehicle. The system also includes a processor to detect lane lines of the road surface in three dimensions, to perform interpolation using the lane lines to determine a ground plane of the road surface, and to adjust the AR image to obtain an adjusted AR image such that all points of the adjusted AR image correspond with points on the ground plane.

Abstract: A system for analyzing images includes a processing device includes a receiving module configured to receive an image, and an analysis module configured to apply the received image to a machine learning network and classify one or more features in the received image, the machine learning network configured to propagate image data through a plurality of convolutional layers, each convolutional layer of the plurality of convolutional layers including a plurality of filter channels, the machine learning network including a bottleneck layer configured to recognize an image feature based on a shape of an image component, The system also includes an output module configured to output characterization data that includes a classification of the one or more features.

Abstract: A system and method may include capturing a multi-channel polarimetric image and a multi-channel RGB image of a scene by a color polarimetric imaging camera. A multi-channel hyperspectral image may be synthesized from the multi-channel RGB image and concatenated with the multi-channel polarimetric image to create an integrated polarimetric-hyperspectral image. Scene properties within the integrated polarimetric-hyperspectral image may be disentangled.

Abstract: A system for notifying a user of a vehicle includes a receiving module configured to receive detection data related to an environment around the vehicle, the detection data including an image of at least a portion of the environment. The system also includes an analysis module configured to detect an object in the image and determine a level of salience of the object to the user, an image enhancement module configured to apply one or more saliency features to a region of the image corresponding to the object, the one or more saliency features including an adjustment to an image attribute in the region, the adjustment based on the level of salience and configured to draw attention of the user to the region without occluding the region, and a display module configured to present a display including the image and the one or more saliency features to the user.

Abstract: A system and method may include capturing a multi-channel polarimetric image and a multi-channel RGB image of a scene by a color polarimetric imaging camera. A multi-channel hyperspectral image may be synthesized from the multi-channel RGB image and concatenated with the multi-channel polarimetric image to create an integrated polarimetric-hyperspectral image. Scene properties within the integrated polarimetric-hyperspectral image may be disentangled.

Abstract: A system and method to adjust an augmented reality (AR) image involve a front-facing camera in a vehicle. The camera obtains an image that includes a road surface ahead of the vehicle. The system also includes a processor to detect lane lines of the road surface in three dimensions, to perform interpolation using the lane lines to determine a ground plane of the road surface, and to adjust the AR image to obtain an adjusted AR image such that all points of the adjusted AR image correspond with points on the ground plane.

Abstract: The present application relates to a method and apparatus for generating a three-dimensional point cloud generation using a polarimetric camera in a drive assistance system equipped vehicle including a camera configured to capture a color image for a field of view and a polarimetric data of the field of view, a processor configured to perform a neural network function in response to the color image and the polarimetric data to generate a depth map of the field of view, and a vehicle controller configured a perform an advanced driving assistance function and to control a vehicle movement in response to the depth map.

Abstract: A vehicle having an imaging sensor that is arranged to monitor a field-of-view (FOV) that includes a travel surface proximal to the vehicle is described. Detecting the travel lane includes capturing a FOV image of a viewable region of the travel surface. The FOV image is converted, via an artificial neural network, to a plurality of feature maps. The feature maps are projected, via an inverse perspective mapping algorithm, onto a BEV orthographic grid. The feature maps include travel lane segments and feature embeddings, and the travel lane segments are represented as line segments. The line segments are concatenated for the plurality of grid sections based upon the feature embeddings to form a predicted lane. The concatenation, or clustering is accomplished via the feature embeddings.

Abstract: Systems and methods to perform image-based three-dimensional (3D) lane detection involve obtaining known 3D points of one or more lane markings in an image including the one or more lane markings. The method includes overlaying a grid of anchor points on the image. Each of the anchor points is a center of i concentric circles. The method also includes generating an i-length vector and setting an indicator value for each of the anchor points based on the known 3D points as part of a training process of a neural network, and using the neural network to obtain 3D points of one or more lane markings in a second image.

Abstract: The present application relates to a method and apparatus for generating a three-dimensional point cloud generation using a polarimetric camera in a drive assistance system equipped vehicle including a camera configured to capture a color image for a field of view and a polarimetric data of the field of view, a processor configured to perform a neural network function in response to the color image and the polarimetric data to generate a depth map of the field of view, and a vehicle controller configured a perform an advanced driving assistance function and to control a vehicle movement in response to the depth map.

Abstract: Systems and methods to perform image-based three-dimensional (3D) lane detection involve obtaining known 3D points of one or more lane markings in an image including the one or more lane markings. The method includes overlaying a grid of anchor points on the image. Each of the anchor points is a center of i concentric circles. The method also includes generating an i-length vector and setting an indicator value for each of the anchor points based on the known 3D points as part of a training process of a neural network, and using the neural network to obtain 3D points of one or more lane markings in a second image.

Abstract: A vehicle, system for operating a vehicle and method of navigating a vehicle. The system includes a sensor and a multi-layer convolutional neural network. The sensor generates an image indicative of a road scene of the vehicle. The multi-layer convolutional neural network generates a plurality of feature maps from the image via a first processing pathway, projects at least one of the plurality of feature maps onto a defined plane relative to a defined coordinate system of the road scene to obtain at least one projected feature map, applies a convolution to the at least one projected feature map in a second processing pathway to obtain a final feature map, and determines lane information from the final feature map. A control system adjusts operation of the vehicle using the lane information.

Abstract: An occupant monitoring apparatus is provided. The apparatus includes a laser illuminator configured to emit a laser to illuminate an occupant; and a sensor configured to generate an image of the occupant illuminated by the laser.