Abstract: A system and method for estimation of a driver state based on eye gaze includes, capturing and sending, using an outward looking camera situated in a vehicle, a first video stream of surrounding environment to a neural controller. The neural controller, based on the first video stream, generates an expected gaze distribution. Using an inward looking camera situated in the vehicle, the camera captures and sends a second video stream of a face of a driver to an eye tracker controller, where based on the second video stream, the eye tracker controller extracts a plurality of gaze directions. A gaze distribution module generates, based on the plurality of gaze directions, an actual gaze distribution. A distance distribution controller, based on a difference between the expected gaze distribution and the actual gaze distribution, generates a distance measure where a determination is made that the distance measure exceeds a threshold.

Abstract: A system for vehicle trajectory planning. The system may include a polarized camera system configured to generate polarized images of a roadway to be driven over with a vehicle, a prior controller configured to generate a prior for the roadway based at least in part on the polarized images, and a driving assistance system configured to provide a driving assistance according to the prior.

Abstract: A system and method for limited rate context-based eye gaze encoding includes generating, based on an outward looking camera situated in a vehicle, a first video stream of a surrounding environment to a controller. The controller generates a scene description based on the first video stream and selects a corresponding scene from a predefined list of known scenes. Further, the controller, based on the selected corresponding scene, selects a codebook of encoding and decoding parameters from a plurality of predefined codebooks. Based on an inward looking camera situated in the vehicle, a second video stream of a face of a driver to an eye tracker controller is generated, where the eye tracker controller estimates a gaze direction of the driver. Encoded data comprising the estimated gaze direction of the driver is transmitted over a bandwidth limited channel to a decoder, based on the selected codebook, the encoded data.

Abstract: A method of training a disparity estimation network. The method includes obtaining an eye-gaze dataset having first images with at least one gaze direction associated with each of the first images. A gaze prediction neural network is trained based on the eye-gaze dataset to develop a model trained to provide a gaze prediction for an external image. A depth database is obtained that includes second images having depth information associated with each of the second images. A disparity estimation neural network for object detection is trained based on an output from the gaze prediction neural network and an output from the depth database.

Abstract: A system and method for error estimation in an eye gaze tracking system in a vehicle may include an operator monitoring system providing measured eye gaze information corresponding to an object outside the vehicle and an external object monitoring system providing theoretical eye gaze information and an error in the measured eye gaze information based upon the measured eye gaze information and the theoretical eye gaze information.

Abstract: A system and method for error estimation in an eye gaze tracking system in a vehicle may include an operator monitoring system providing measured eye gaze information corresponding to an object outside the vehicle and an external object monitoring system providing theoretical eye gaze information and an error in the measured eye gaze information based upon the measured eye gaze information and the theoretical eye gaze information.

Abstract: A system for notifying a user of a vehicle includes a receiving module configured to receive detection data related to an environment around the vehicle, the detection data including an image of at least a portion of the environment. The system also includes an analysis module configured to detect an object in the image and determine a level of salience of the object to the user, an image enhancement module configured to apply one or more saliency features to a region of the image corresponding to the object, the one or more saliency features including an adjustment to an image attribute in the region, the adjustment based on the level of salience and configured to draw attention of the user to the region without occluding the region, and a display module configured to present a display including the image and the one or more saliency features to the user.

Abstract: A system for notifying a user of a vehicle includes a receiving module configured to receive detection data, and a threat analysis module configured to receive object detection data related to a detected object in an environment around the vehicle, acquiring a predicted trajectory of the detected object, and determine whether the detected object constitutes a threat based on the predicted trajectory of the detected object and a vehicle trajectory. The system also includes a threat display module configured to determine an operating scenario based on a user attentiveness, a field of view, an operating mode, and a threat level, and generate a notification to the user representing the threat, the notification including a visual representation of the detected object and a visual indicator of the predicted trajectory of the detected object. At least one of the visual representation and the visual indicator is customized based on the operating scenario.

Abstract: A vehicle, system and method of adjusting a mirror of a vehicle. A system for adjusting a mirror of a vehicle is disclosed. The system includes a calibration a calibration marker disposed on the vehicle, a camera, a motor and a processor. The calibration marker forms a calibration image onto a face of an occupant of the vehicle via reflection through the mirror. The camera obtains a camera image including the calibration image and the face of the occupant. The processor determines from the camera image an initial location of the calibration image at the face, determines a calibrated setting of the mirror that places the calibration image at a calibration location, and operates the motor to adjust the mirror to the calibrated setting.

Abstract: Systems and methods to mitigate an effect of an errant signal on an image sensor of a vehicle involve collecting light and separating the light to obtain signals at different wavelengths. A method includes determining an intensity of the light at each of the different wavelengths, and identifying the errant signal based on the intensity of the light exceeding a threshold value at an errant signal wavelength among the different wavelengths. The errant signal is mitigated using the controller.

Abstract: An occupant monitoring apparatus is provided. The apparatus includes a laser illuminator configured to emit a laser to illuminate an occupant; and a sensor configured to generate an image of the occupant illuminated by the laser.

Abstract: An interactive driving system and method that improves a driving experience by allowing a passenger to interactively monitor and evaluate certain driving maneuvers in real-time with an augmented reality interface. The input provided by the passenger via the augmented reality interface is recorded for later processing and analysis. For instance, the passenger input may be used to generate driving reports for later review by the driver and passenger (e.g., when the passenger is teaching a new driver how to drive) or for the manufacturer of autonomous or semi-autonomous vehicles (e.g., when the manufacturer is seeking actionable feedback for improving various self-driving algorithms). In addition, the interactive driving system and method entertain the passenger with the augmented reality interface, which can be delivered in a game-like fashion via a computer program product installed on their own personal electronic device (e.g., a tablet, phone, laptop, etc.).

Abstract: Examples of techniques for vehicle communication between a first vehicle and a second vehicle are disclosed. An example method includes displaying, on an external vehicle display of the first vehicle, vehicle data. The method further includes capturing, by a camera of the second vehicle, an image of the external vehicle display. The method further includes processing, by a processing device associated with the second vehicle, the image to extract the vehicle data about the first vehicle. The method further includes determining, by the processing device, a position of the first vehicle relative to a position of the second vehicle based at least in part on the vehicle data. The method further includes controlling, by the processing device associated with the second vehicle, the second vehicle based at least in part on the position of the first vehicle relative to the position of the second vehicle.

Abstract: Methods and systems are provided for a speech system of a vehicle. In particular, a method is taught for associating a speech utterance with a voice command in response to a failed voice control attempt followed by a successfully voice control attempt.

Abstract: Methods and systems are provided for a speech system of a vehicle. In particular, a method is taught for associating a speech utterance with a voice command in response to a failed voice control attempt followed by a successfully voice control attempt.

Abstract: A system and method of performing automatic speech recognition (ASR) includes: receiving speech at a vehicle microphone; communicating the received speech to an ASR system; measuring an amount of time that elapses while speech is received; selecting a point-of-interest (POI) context or an address context based on the measured amount of received time; and processing the received speech using a POI context-based grammar when a POI context is selected or an address-based grammar when an address context is selected.

Abstract: Methods and systems are provided for recovering from an error in a speech recognition system. In one embodiment, a method includes: receiving, by a processor, a first command recognized from a first speech utterance by a first language model; receiving, by the processor, a second command recognized from the first speech utterance by a second language model; determining, by the processor, at least one of similarities and dissimilarities between the first command and the second command; processing, by the processor, the first command and the second command with at least one rule of an error model based on the similarities and dissimilarities to determine a root cause; and selectively executing a recovery process based on the root cause.

Abstract: A system for serving a shared-ride user, including a non-transitory storage component and a hardware-based processing unit performing module functions. The storage includes a user-input-interface module that receives, from a machine interface, user-input data regarding a user/co-passenger interaction. The storage includes a ride-sharing module determining, based on input data, an identity or account for the co-passenger, and an output module performing an action based on the identity or account. In another aspect, the storage includes a commerce module determining, based on the input data, a service or product indicated in the interaction and the output action is based on the service or product. In another aspect, the storage includes a social-media module accessing a social-media resource to, using the input data, determine an identity or account of the co-passenger, a product or service indicated by the co-passenger or the user in the interaction, or a schedule of the co-passenger.

Abstract: Methods and systems are provided for managing speech of a speech system. In one embodiment, a method includes: receiving, by a processor, relative information comprising graph data from at least one relative data datasource; processing, by a processor, the graph data of the relative information to determine at least one of an association and a relationship associated with an element defined in the speech system; and storing, by a processor, the at least one of association and relationship as relative slot data for use by at least one of a speech recognition method and a dialog management method.

Abstract: An automated vehicle includes a control system configured to selectively transfer the automated vehicle from an automated control mode to a manual control mode based on a transfer-of-control criterion. The transfer-of-control criterion is based on a current context of the automated vehicle and a dataset of previous transfer-of-control events and previous contexts associated with the operation of a plurality of additional automated vehicles.