Abstract: A plurality of images can be acquired from a plurality of sensors and a plurality of flattened patches can be extracted from the plurality of images. An image location in the plurality of images and a sensor type token identifying a type of sensor used to acquire an image in the plurality of images from which the respective flattened patch was acquired can be added to each of the plurality of flattened patches. The flattened patches can be concatenated into a flat tensor and add a task token indicating a processing task to the flat tensor, wherein the flat tensor is a one-dimensional array that includes two or more types of data. The flat tensor can be input to a first deep neural network that includes a plurality of encoder layers and a plurality of decoder layers and outputs transformer output. The transformer output can be input to a second deep neural network that determines an object prediction indicated by the token and the object predictions can be output.

Abstract: The disclosure is generally directed to systems and methods for inference quality determination of a deep neural network (DNN) without requiring ground truth information for use in driver-assisted vehicles, including receiving an image frame from a source; applying a normal inference DNN model to the image frame to produce a first inference with a first bounding box using a normal inference DNN model; applying a deep inference DNN model to a plurality of filtered versions of the image frame to produce a plurality of deep inferences with a plurality of bounding boxes; comparing the plurality of bounding boxes to identify a cluster condition of the plurality of bounding boxes; and determining an inference quality of the image frame of the normal inference DNN model as a function of the cluster condition.

Abstract: A system is disclosed that includes a computer and memory, the memory including instructions to acquire images, including a first image and a second image of an object attached to a platform that is moving and determine a first real world location of a fiducial marker and a second location of the fiducial marker. A center of rotation for the object can be determined by tracking the first and second real world locations of the fiducial marker and an angle of an axis the object with respect to an axis of the platform can be determined based on the center of rotation, the tracked locations of the fiducial marker, and calibration data.

Abstract: A computer includes a processor and a memory storing instructions executable by the processor to receive image data from a camera, generate a depth map from the image data, detect an object in the image data, apply a bounding box circumscribing the object to the depth map, mask the depth map by setting depth values for pixels in the bounding box in the depth map to a depth value of a closest pixel in the bounding box, and determine a distance to the object based on the masked depth map. The closest pixel is closest to the camera of the pixels in the bounding box.

Abstract: A first image can be acquired from a first sensor included in a vehicle and input to a deep neural network to determine a first bounding box for a first object. A second image can be acquired from the first sensor. Input latitudinal and longitudinal motion data from second sensors included in the vehicle corresponding to the time between inputting the first image and inputting the second image. A second bounding box can be determined by translating the first bounding box based on the latitudinal and longitudinal motion data. The second image can be cropped based on the second bounding box. The cropped second image can be input to the deep neural network to detect a second object. The first image, the first bounding box, the second image, and the second bounding box can be output.

Abstract: Embodiments describe a system configured with a brain machine interface (BMI) system implemented in a vehicle for performing vehicle functions using electrical impulses from motor cortex activity in a user's brain. The system uses fuzzy states for increased robustness. The fuzzy states are defined by sets of Gaussian kernel-type membership functions that are defined for steering and velocity action function states. The membership functions define fuzzy states that provide overlapping control tiers for increasing and decreasing vehicle functionality. An autonomous vehicle may perform control and governance of transitions between membership functions that may overlap, resulting in smooth transitioning between the states.

Abstract: A computer includes a processor and a memory, and the memory stores instructions executable by the processor to receive an image frame from a camera, generate a feature map from the image frame, generate a depth map from the feature map, classify an object in the image frame based on the feature map, and estimate a distance to the object based on the depth map and based on an input to generating the feature map.

Abstract: Embodiments describe a vehicle configured with a brain machine interface (BMI) for a vehicle computing system to control vehicle functions using electrical impulses from motor cortex activity in a user's brain. A BMI training system trains the BMI device to interpret neural data generated by a motor cortex of a user and correlate the neural data to a vehicle control command associated with a neural gesture emulation function. A BMI system onboard the vehicle may receive a neural data feed of neural data from the user using the trained BMI device, determine, a user intention for a control instruction to control a vehicle infotainment system using the neural data feed, and perform an action based on the control instruction. The vehicle may further include a headrest configured as a Human Machine Interface (HMI) device that reads the electrical impulses without invasive electrode connectivity.

Abstract: A system includes a computer including a processor and a memory, the memory storing instructions executable by the processor to generate a synthetic image by adjusting respective color values of one or more pixels of a reference image based on a specified meteorological optical range from a vehicle sensor to simulated fog, and input the synthetic image to a machine learning program to train the machine learning program to identify a meteorological optical range from the vehicle sensor to actual fog.

Abstract: Systems and methods for estimation of vehicle hitchball location are disclosed. A plurality of image frames may be received from a rear-facing camera of a vehicle. The rear-facing camera may be directed at a front of a trailer that is coupled to the vehicle at a hitchball. An approximate lateral location of the hitchball coupled to the vehicle may be determined by obtaining a plurality of cropped images along a hitch drawbar coupled to the vehicle by stepping along the vertical direction, and performing a stepwise lateral scan, centered at the hitch drawbar.

Abstract: A plurality of images can be acquired from a plurality of sensors and a plurality of flattened patches can be extracted from the plurality of images. An image location in the plurality of images and a sensor type token identifying a type of sensor used to acquire an image in the plurality of images from which the respective flattened patch was acquired can be added to each of the plurality of flattened patches. The flattened patches can be concatenated into a flat tensor and add a task token indicating a processing task to the flat tensor, wherein the flat tensor is a one-dimensional array that includes two or more types of data. The flat tensor can be input to a first deep neural network that includes a plurality of encoder layers and a plurality of decoder layers and outputs transformer output. The transformer output can be input to a second deep neural network that determines an object prediction indicated by the token and the object predictions can be output.

Abstract: The disclosure is generally directed to systems and methods for inference quality determination of a deep neural network (DNN) without requiring ground truth information for use in driver-assisted vehicles, including receiving an image frame from a source; applying a normal inference DNN model to the image frame to produce a first inference with a first bounding box using a normal inference DNN model; applying a deep inference DNN model to a plurality of filtered versions of the image frame to produce a plurality of deep inferences with a plurality of bounding boxes; comparing the plurality of bounding boxes to identify a cluster condition of the plurality of bounding boxes; and determining an inference quality of the image frame of the normal inference DNN model as a function of the cluster condition.

Abstract: A computer includes a processor and a memory storing instructions executable by the processor to receive image data from a camera, generate a depth map from the image data, detect an object in the image data, apply a bounding box circumscribing the object to the depth map, mask the depth map by setting depth values for pixels in the bounding box in the depth map to a depth value of a closest pixel in the bounding box, and determine a distance to the object based on the masked depth map. The closest pixel is closest to the camera of the pixels in the bounding box.

Abstract: A training system for a deep neural network and method of training is disclosed. The system and/or method may comprise: receiving, from an eye-tracking system associated with a sensor, an image frame captured while an operator is controlling a vehicle; receiving, from the eye-tracking system, eyeball gaze data corresponding to the image frame; and iteratively training the deep neural network to determine an object of interest depicted within the image frame based on the eyeball gaze data. The deep neural network generates at least one feature map and determine a proposed region corresponding to the object of interest within the at least one feature map based on the eyeball gaze data.

Abstract: A training system for a neural network system and method of training is disclosed. The method may comprise: receiving, from a sensor, an image frame captured while an operator is controlling a vehicle; using an eye-tracking system associated with the sensor, monitoring the eyes of the operator to determine eyeball gaze data; determining, from the image frame, a plurality of pedestrians; and iteratively training the neural network system to determine, from among the plurality of pedestrians, the one or more target pedestrians using the eyeball gaze data and an answer dataset that is based on the eyeball gaze data, wherein the determined one or more target pedestrians have a relatively-higher probability of collision with the vehicle than a remainder of the plurality of pedestrians.

Abstract: A first image can be acquired from a first sensor included in a vehicle and input to a deep neural network to determine a first bounding box for a first object. A second image can be acquired from the first sensor. Input latitudinal and longitudinal motion data from second sensors included in the vehicle corresponding to the time between inputting the first image and inputting the second image. A second bounding box can be determined by translating the first bounding box based on the latitudinal and longitudinal motion data. The second image can be cropped based on the second bounding box. The cropped second image can be input to the deep neural network to detect a second object. The first image, the first bounding box, the second image, and the second bounding box can be output.

Abstract: A system, including a processor and a memory, the memory including instructions to be executed by the processor train a deep neural network based on plurality of real-world images, determine the accuracy of the deep neural network is below a threshold based on identifying one or more physical features by the deep neural network, including one or more object types, in the plurality of real-world images and generate a plurality of synthetic images based on the accuracy of the deep neural network is below a threshold based on identifying the one or more physical features using a photo-realistic image rendering software program and a generative adversarial network. The instructions can include further instructions to retrain the deep neural network based on the plurality of real-world images and the plurality of synthetic images and output the retrained deep neural network.

Abstract: A system, including a processor and a memory, the memory including instructions to be executed by the processor to train a deep neural network based on a plurality of real-world images, determine the accuracy of the deep neural network is below a threshold based on identifying one or more physical features by the deep neural network, including one or more object types, in the plurality of real-world images and generate a plurality of synthetic images based on the accuracy of the deep neural network is below a threshold based on identifying the one or more physical features using a photo-realistic image rendering software program and a generative adversarial network. The instructions can include further instructions to retrain the deep neural network based on the plurality of real-world images and the plurality of synthetic images and output the retrained deep neural network.

Abstract: A sensor-fusion approach of using Brain Machine Interface (BMI) to gain a higher resolution perspective of chassis input control is described according to the present disclosure. Traditional chassis control inputs, such as steering wheel, brake and driver state monitoring sensors can calculate input but often cannot well predict intent. By interpreting well known motor command signals, it can become clear how much chassis input the driver was intending to provide. The BMI may monitor motor cortex to identity when a muscular movement is imminent, such as the movement of the arms to grasp the steering wheel. This combination would enable faster and more precise intent calculation. Additionally, information from driver wearable devices may be used to supplement the determination. This allows for a faster response and well as better integration with the driver.

Abstract: Embodiments describe a system configured with a brain machine interface (BMI) system implemented in a vehicle for performing vehicle functions using electrical impulses from motor cortex activity in a user's brain. The system uses fuzzy states for increased robustness. The fuzzy states are defined by sets of Gaussian kernel-type membership functions that are defined for steering and velocity action function states. The membership functions define fuzzy states that provide overlapping control tiers for increasing and decreasing vehicle functionality. An autonomous vehicle may perform control and governance of transitions between membership functions that may overlap, resulting in smooth transitioning between the states.