Abstract: Methods, systems, and computer-readable mediums storing computer executable code for visual recognition implementing a triplet loss function are provided. The method include receiving an image generated from an image source associated with a vehicle. The method may also include analyzing the image based on a convolutional neural network. The convolutional neural network may apply both a triplet loss function and a softmax loss function to the image to determine classification logits. The method may also include classifying the image into a predetermined class distribution based upon the determined classification logits. The method may also include instructing the vehicle to perform a specific task based upon the classified image.

Abstract: Systems and techniques for autonomous vehicle decision making may include training an autonomous vehicle decision making database by capturing an image including a first training object and a second training object during a training phase. The first training object may be classified as a first class and the second training object may be classified as a second class based on a driver gaze location associated with a driver of the vehicle. The database may be built based on classification of the first training object and the second training object. The autonomous vehicle decision making database may be utilized to classify a first object as a first class and a second object as a second class during an operation phase. A processor may perform a first computation associated with the first object based on the classification of the first object and the classification of the second object.

Abstract: Systems and methods for generating adaptive instructions for a driver. The system includes a memory that stores instructions for generating adaptive instructions for a driver. The system also includes a processor configured to execute the instructions. The instructions cause the processor to: monitor one or more factors including a gaze of the driver, a physiological signal of the driver, information related to the operation of a vehicle, and information related to an external environment of the vehicle; analyze the one or more factors to determine when and how to provide the adaptive instructions to the driver; generate the adaptive instructions based on the analysis; and provide the adaptive instructions to the driver via a vehicle user interface.

Abstract: A system and method for learning naturalistic driving behavior based on vehicle dynamic data that include receiving vehicle dynamic data and image data and analyzing the vehicle dynamic data and the image data to detect a plurality of behavioral events. The system and method also include classifying at least one behavioral event as a stimulus-driven action and building a naturalistic driving behavior data set that includes annotations that are based on the at least one behavioral event that is classified as the stimulus-driven action. The system and method further include controlling a vehicle to be autonomously driven based on the naturalistic driving behavior data set.

Abstract: A system and method for learning naturalistic driving behavior based on vehicle dynamic data that include receiving vehicle dynamic data and image data and analyzing the vehicle dynamic data and the image data to detect a plurality of behavioral events. The system and method also include classifying at least one behavioral event as a stimulus-driven action and predicting at least one behavioral event as a goal-oriented action based on the stimulus-driven action. The system and method additionally include building a naturalistic driving behavior data set that includes annotations that are based on the at least one behavioral event that is classified as the stimulus-driven action. The system and method further include controlling a vehicle to be autonomously driven based on the naturalistic driving behavior data set.

Abstract: Systems and methods for generating adaptive instructions for a driver. The system includes a memory that stores instructions for generating adaptive instructions for a driver. The system also includes a processor configured to execute the instructions. The instructions cause the processor to: monitor one or more factors including a gaze of the driver, a physiological signal of the driver, information related to the operation of a vehicle, and information related to an external environment of the vehicle; analyze the one or more factors to determine when and how to provide the adaptive instructions to the driver; generate the adaptive instructions based on the analysis; and provide the adaptive instructions to the driver via a vehicle user interface.

Abstract: A system and method for learning and executing naturalistic driving behavior that include classifying a driving maneuver as a goal-oriented action or a stimulus-driven action based on data associated with a trip of a vehicle. The system and method also include determining a cause associated with the driving maneuver classified as a stimulus-driven action and determining an attention capturing traffic related object associated with the driving maneuver. The system and method additionally include building a naturalistic driving behavior data set that includes at least one of: an annotation of the driving maneuver based on a classification of the driving maneuver, an annotation of the cause, and an annotation of the attention capturing traffic object. The system and method further include controlling the vehicle to be autonomously driven based on the naturalistic driving behavior data set.

Abstract: Systems and methods for analyzing a driver's brake behavior are provided. In some aspects, the system includes a memory that stores instructions for executing processes for analyzing the driver's brake behavior, and a processor configured to execute the instructions. In aspects, the processes include: receiving data from one or more multi-modal sensors, determining whether each braking event of a plurality of braking events is a hard braking event; analyzing the one or more physiological signals of the driver to determine whether each of the hard braking events is a reactive hard braking event or an intended hard braking event; applying a Lasso regression analysis and a classification framework for each of the hard braking events to identify features that indicate whether a respective hard braking event is a reactive hard braking event or an intended hard braking event; and generating a signal notifying a vehicle of the features.

Abstract: Presence of an object can be indicated on a display of a vehicle. An avatar can be displayed on the display of the vehicle indicating a travel direction of the vehicle. Presence of the object can be detected within a path in the travel direction of the vehicle, where the path corresponds to an area on the display of the vehicle. A beam can be drawn from the avatar to the object as an alert of the presence of the object.

Abstract: Systems and methods for analyzing a driver's brake behavior are provided. In some aspects, the system includes a memory that stores instructions for executing processes for analyzing the driver's brake behavior, and a processor configured to execute the instructions. In aspects, the processes include: receiving data from one or more multi-modal sensors, determining whether each braking event of a plurality of braking events is a hard braking event; analyzing the one or more physiological signals of the driver to determine whether each of the hard braking events is a reactive hard braking event or an intended hard braking event; applying a Lasso regression analysis and a classification framework for each of the hard braking events to identify features that indicate whether a respective hard braking event is a reactive hard braking event or an intended hard braking event; and generating a signal notifying a vehicle of the features.

Abstract: Presence of an object can be indicated on a display of a vehicle. Multiple indicators can be displayed on the display of the vehicle, where each indicator corresponds to one of multiple portions of a drawing area associated with the display. Presence of the object can be detected within a travel direction of the vehicle. Location information of the object can be associated with one of the multiple portions of the drawing area. One of the multiple indicators that corresponds to the one of the multiple portions of the drawing area can be highlighted on the display as an alert of the presence of the object.

Abstract: The systems and methods provided herein are directed to the uploading and transmission of vehicle data to a remote system when a physiological event for a driver has been detected using one or more sensors. Information such as the driver's heart rate, temperature, voice inflection or facial expression may be monitored to detect the physiological event. Vehicle data, such as gathering or control system data, may be sent once the event has been detected. Selected vehicle data associated with the event or all data during the time of the event may be sent. After receiving the vehicle data, the remote system may process or store it where it may be used to modify automated driving functionalities.

Abstract: A method and vehicle system for reducing driver distractions caused by a head-up display (HUD). The HUD projects one or more graphic elements onto a windshield in view of a driver of the vehicle. The vehicle system determines a content status of the one or more projected graphic elements. The vehicle system detects one or more objects within an environment of the vehicle. The vehicle system determines whether the content status of the one or more projected graphic elements satisfies a driver distraction criterion. The vehicle then notifies the driver of the one or more detected objects when the content status of the one or more projected graphic element satisfies the driver distraction criterion.

Abstract: In one or more embodiments, driver awareness may be calculated, inferred, or estimated utilizing a saliency model, a predictive model, or an operating environment model. An awareness model including one or more awareness scores for one or more objects may be constructed based on the saliency model or one or more saliency parameters associated therewith. A variety of sensors or components may detect one or more object attributes, saliency, operator attributes, operator behavior, operator responses, etc. and construct one or more models accordingly. Examples of object attributes associated with saliency or saliency parameters may include visual characteristics, visual stimuli, optical flow, velocity, movement, color, color differences, contrast, contrast differences, color saturation, brightness, edge strength, luminance, a quick transient (e.g., a flashing light, an abrupt onset of a change in intensity, brightness, etc.).

Abstract: A method and system are disclosed for recognizing speech errors, such as in a spoken short messages, using an audio input device to receive an utterance of a short message, using an automated speech recognition module to generate a text sentence corresponding to the utterance, generating an N-best list of predicted error sequences for the text sentence using a linear-chain conditional random field (CRF) module, where each word of the text sentence is assigned a label in each of the predicted error sequences, and each label is assigned a probability score. The predicted error sequence labels are rescored using a metacost matrix module, the best rescored error sequence from the N-best list of predicted error sequences is selected using a Recognition Output Voting Error Reduction (ROVER) module, and a dialog action is executed by a dialog action module based on the best rescored error sequence and the dialog action policy.

Abstract: A method for identifying a Point of Interest (POI) of a driver of a vehicle comprises: receiving voice data; receiving head movement data of the driver; determining a location of the vehicle and POIs around the location when the voice data is received; identifying potential POIs of the driver by analyzing the head movement data and a timing of the voice data using Gaussian Process Regression (GPR); and identifying the POI of the driver using a plurality of: a speed of the vehicle, density of surrounding POIs and visual salience.

Abstract: In one or more embodiments, driver awareness may be calculated, inferred, or estimated utilizing a saliency model, a predictive model, or an operating environment model. An awareness model including one or more awareness scores for one or more objects may be constructed based on the saliency model or one or more saliency parameters associated therewith. A variety of sensors or components may detect one or more object attributes, saliency, operator attributes, operator behavior, operator responses, etc. and construct one or more models accordingly. Examples of object attributes associated with saliency or saliency parameters may include visual characteristics, visual stimuli, optical flow, velocity, movement, color, color differences, contrast, contrast differences, color saturation, brightness, edge strength, luminance, a quick transient (e.g., a flashing light, an abrupt onset of a change in intensity, brightness, etc.).

Abstract: A method and system are disclosed for recognizing speech errors, such as in a spoken short messages, using an audio input device to receive an utterance of a short message, using an automated speech recognition module to generate a text sentence corresponding to the utterance, generating an N-best list of predicted error sequences for the text sentence using a linear-chain conditional random field (CRF) module, where each word of the text sentence is assigned a label in each of the predicted error sequences, and each label is assigned a probability score. The predicted error sequence labels are rescored using a metacost matrix module, the best rescored error sequence from the N-best list of predicted error sequences is selected using a Recognition Output Voting Error Reduction (ROVER) module, and a dialog action is executed by a dialog action module based on the best rescored error sequence and the dialog action policy.