Abstract: Embodiments describe a system configured with a brain machine interface (BMI) system implemented in a vehicle for performing vehicle functions using electrical impulses from motor cortex activity in a user's brain. The system uses fuzzy states for increased robustness. The fuzzy states are defined by sets of Gaussian kernel-type membership functions that are defined for steering and velocity action function states. The membership functions define fuzzy states that provide overlapping control tiers for increasing and decreasing vehicle functionality. An autonomous vehicle may perform control and governance of transitions between membership functions that may overlap, resulting in smooth transitioning between the states.

Abstract: Tracking an object included in a reflective surface by detecting the reflective surface included in one or more images included in a plurality of images by determining a location of the reflective surface in pixel coordinates and tracking the location of the reflective surface in the plurality of images. Real world locations of the object can be determined based on attributes of the reflective surface including a geometric class, extrinsic properties that relate the reflective surface to an environment, intrinsic properties describing the reflective surface without referring to the environment and calibration properties of the sensor.

Abstract: Embodiments describe a vehicle configured with a brain machine interface (BMI) for a vehicle computing system to control vehicle functions using electrical impulses from motor cortex activity in a user's brain. A BMI training system trains the BMI device to interpret neural data generated by a motor cortex of a user and correlate the neural data to a vehicle control command associated with a neural gesture emulation function. A BMI system onboard the vehicle may receive a neural data feed of neural data from the user using the trained BMI device, determine, a user intention for a control instruction to control a vehicle infotainment system using the neural data feed, and perform an action based on the control instruction. The vehicle may further include a headrest configured as a Human Machine Interface (HMI) device that reads the electrical impulses without invasive electrode connectivity.

Abstract: Example embodiments described in this disclosure are generally directed to detecting drowsiness in a driver of a vehicle. In an example method, a driver drowsiness detection system receives a motor cortex signal from a brain activity monitoring element attached to the driver. The brain activity monitoring element can be a cortical implant, for example. The driver drowsiness detection system evaluates the motor cortex signal to identify an anatomical part of the driver (eyes, for example) that is associated with a brain activity. The driver drowsiness detection system then uses a drowsiness detection device placed in the vehicle for evaluating a physical activity of the anatomical part. The evaluation may be carried out by using a camera directed upon the driver's eyes, for example. The driver drowsiness detection system determines a drowsiness state of the driver based on the evaluation and assigns a sleep risk score.

Abstract: A training system for a deep neural network and method of training is disclosed. The system and/or method may comprise: receiving, from an eye-tracking system associated with a sensor, an image frame captured while an operator is controlling a vehicle; receiving, from the eye-tracking system, eyeball gaze data corresponding to the image frame; and iteratively training the deep neural network to determine an object of interest depicted within the image frame based on the eyeball gaze data. The deep neural network generates at least one feature map and determine a proposed region corresponding to the object of interest within the at least one feature map based on the eyeball gaze data.

Abstract: A training system for a neural network system and method of training is disclosed. The method may comprise: receiving, from a sensor, an image frame captured while an operator is controlling a vehicle; using an eye-tracking system associated with the sensor, monitoring the eyes of the operator to determine eyeball gaze data; determining, from the image frame, a plurality of pedestrians; and iteratively training the neural network system to determine, from among the plurality of pedestrians, the one or more target pedestrians using the eyeball gaze data and an answer dataset that is based on the eyeball gaze data, wherein the determined one or more target pedestrians have a relatively-higher probability of collision with the vehicle than a remainder of the plurality of pedestrians.

Abstract: A sensor-fusion approach of using Brain Machine Interface (BMI) to gain a higher resolution perspective of chassis input control is described according to the present disclosure. Traditional chassis control inputs, such as steering wheel, brake and driver state monitoring sensors can calculate input but often cannot well predict intent. By interpreting well known motor command signals, it can become clear how much chassis input the driver was intending to provide. The BMI may monitor motor cortex to identity when a muscular movement is imminent, such as the movement of the arms to grasp the steering wheel. This combination would enable faster and more precise intent calculation. Additionally, information from driver wearable devices may be used to supplement the determination. This allows for a faster response and well as better integration with the driver.

Abstract: Embodiments describe a system configured with a brain machine interface (BMI) system implemented in a vehicle for performing vehicle functions using electrical impulses from motor cortex activity in a user's brain. The system uses fuzzy states for increased robustness. The fuzzy states are defined by sets of Gaussian kernel-type membership functions that are defined for steering and velocity action function states. The membership functions define fuzzy states that provide overlapping control tiers for increasing and decreasing vehicle functionality. An autonomous vehicle may perform control and governance of transitions between membership functions that may overlap, resulting in smooth transitioning between the states.

Abstract: A training system for a deep neural network and method of training is disclosed. The system and/or method may comprise: receiving, from an eye-tracking system associated with a sensor, an image frame captured while an operator is controlling a vehicle; receiving, from the eye-tracking system, eyeball gaze data corresponding to the image frame; and iteratively training the deep neural network to determine an object of interest depicted within the image frame based on the eyeball gaze data. The deep neural network generates at least one feature map and determine a proposed region corresponding to the object of interest within the at least one feature map based on the eyeball gaze data.

Abstract: Example embodiments described in this disclosure are generally directed to detecting drowsiness in a driver of a vehicle. In an example method, a driver drowsiness detection system receives a motor cortex signal from a brain activity monitoring element attached to the driver. The brain activity monitoring element can be a cortical implant, for example. The driver drowsiness detection system evaluates the motor cortex signal to identify an anatomical part of the driver (eyes, for example) that is associated with a brain activity. The driver drowsiness detection system then uses a drowsiness detection device placed in the vehicle for evaluating a physical activity of the anatomical part. The evaluation may be carried out by using a camera directed upon the driver's eyes, for example. The driver drowsiness detection system determines a drowsiness state of the driver based on the evaluation and assigns a sleep risk score.

Abstract: Embodiments describe a vehicle configured with a brain machine interface (BMI) for a vehicle computing system to control vehicle functions using electrical impulses from motor cortex activity in a user's brain. A BMI training system trains the BMI device to interpret neural data generated by a motor cortex of a user and correlates the neural data to a vehicle control command associated with a neural gesture emulation function. A BMI system onboard the vehicle may receive a continuous neural data feed of neural data from the user using the trained BMI device, determine a user intention for a control instruction to control a vehicle system using the continuous neural data feed, and perform an action based on the control instruction. A user may control aspects of automated parking using the BMI device in conjunction with a vehicle controller that governs some aspects of the parking operation.

Abstract: A training system for a neural network system and method of training is disclosed. The method may comprise: receiving, from a sensor, an image frame captured while an operator is controlling a vehicle; using an eye-tracking system associated with the sensor, monitoring the eyes of the operator to determine eyeball gaze data; determining, from the image frame, a plurality of pedestrians; and iteratively training the neural network system to determine, from among the plurality of pedestrians, the one or more target pedestrians using the eyeball gaze data and an answer dataset that is based on the eyeball gaze data, wherein the determined one or more target pedestrians have a relatively-higher probability of collision with the vehicle than a remainder of the plurality of pedestrians.

Abstract: Embodiments describe a vehicle configured with a brain machine interface (BMI) for a vehicle computing system to control vehicle functions using electrical impulses from motor cortex activity in a user's brain. A BMI training system trains the BMI device to interpret neural data generated by a motor cortex of a user and correlate the neural data to a vehicle control command associated with a neural gesture emulation function. A BMI system onboard the vehicle may receive a neural data feed of neural data from the user using the trained BMI device, determine, a user intention for a control instruction to control a vehicle infotainment system using the neural data feed, and perform an action based on the control instruction. The vehicle may further include a headrest configured as a Human Machine Interface (HMI) device that reads the electrical impulses without invasive electrode connectivity.

Abstract: Brain impairment, for example due to stroke, is corrected in order to improve body movement. An fNIRS device is positioned over the motor cortex of non-impaired individuals, and blood oxygen in locations of the brain is analyzed to determine brain activity corresponding to a particular body movement. The movements are statistically analyzed, and are compared with fNIRS data gathered from a movement impaired individual attempting the same movement. A weighted value corresponding to the desired brain activity is generated using the statistical analysis, and is graphically displayed to the movement impaired individual during the attempts. This produces a feedback loop relating to the movement which can be repeated to produce brain plasticity in the impaired individual to facilitate the movement. Additionally, correct brain activity can be used to cause the application of an electrical signal to muscles of the body to produce the desired movement.