Abstract: A method for training a neural network including receiving a plurality of images of a driver's field of view, generating a depth information, a driver's gaze probability and a known object indication for each of the plurality of images, estimating a probability of an unknown object within each of the plurality of images in response to the depth information, the driver's gaze probability and the known object indication, generating a plurality of annotated images in response to annotating each of the plurality of images having the probability of the unknown object exceeding a threshold probability to identify the unknown object, wherein each of the plurality of annotated images is annotated to identify the unknown object, and training the neural network in response to the plurality of annotated images.

Abstract: A system includes a monitoring module configured to determine at least one of a location and a route of a vehicle, and a recommendation module configured to identify a plurality of service stations based on the at least one of the location and the route of the vehicle. The monitoring module is configured to compare a preference of a first user of the vehicle to preference data related to a second user of another vehicle, predict a preferred service station of the plurality of service stations based on the comparing, and present a recommendation to the first user, the recommendation indicating the preferred service station.

Abstract: An assistance system includes: a perception module configured to collect sensor data including data tracking behavior of a vehicle occupant in a host vehicle, and to determine perception information describing a current situation warranting initiation of an interactive dialog with the vehicle occupant; an interactive goal module configured to determine an interactive goal based on the determined perception information; an interaction timing module configured to determine timing for initiating the interactive dialog based on the interactive goal; a dialog module configured to, based on the perception information and the timing, initiate the interactive dialog to provide at least one of a suggestion and an offer to the vehicle occupant; and a vehicle control module configured to control operation of at least one of a device and a system of the host vehicle in response to the vehicle occupant accepting the at least one of the suggestion and the offer.

Abstract: Methods and systems are provided that include one or more first sensors, one or more second sensors, and a processor of a vehicle. The one or more first sensors have a first modality, and are configured to receive a first input from a passenger of the vehicle pertaining to a request. The processor is configured to at least facilitate providing instructions to the passenger for providing an additional input pertaining to the request within a predetermined amount of time. The one or more second sensors have a second modality that is different from the first modality, and are configured to receive a second input from the passenger pertaining to the request. The processor is further configured to at least facilitate interpreting the second input; and performing a vehicle action corresponding to the request based on the interpreting of the second input.

Abstract: A method includes receiving a trip history for an operator of the vehicle with the trip history including trip information regarding previous trips by the operator of the vehicle. A data cluster corresponding to each destination in the trip history is generated by extracting input features from trip information for each trip. The input features characterize a relationship between the operator of the vehicle and the previous trips. A training dataset is generated based on collecting the data cluster corresponding to each of the destinations in the trip history. The training dataset is utilized to develop a gradient boosted trees model. At least one destination for the operator of the vehicle is predicted with the gradient boosted trees model utilizing at least one of an origin location of the operator of the vehicle, a time, or a day as input conditions for the gradient boosted trees model.

Abstract: A method of training a disparity estimation network. The method includes obtaining an eye-gaze dataset having first images with at least one gaze direction associated with each of the first images. A gaze prediction neural network is trained based on the eye-gaze dataset to develop a model trained to provide a gaze prediction for an external image. A depth database is obtained that includes second images having depth information associated with each of the second images. A disparity estimation neural network for object detection is trained based on an output from the gaze prediction neural network and an output from the depth database.

Abstract: A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations. The operations include generating, at an occupancy estimation network, an occupancy probability at one or more voxels, predicting, via a gaze prediction model, a gaze direction of a gaze prediction, and identifying, based on the predicted gaze direction, an object of interest. The operations also include generating, via an occupancy estimation application, a gaze saliency map based on the gaze direction and identified object of interest, and updating, based on the determined gaze direction and the gaze saliency map, the occupancy probability of the one or more voxels.

Abstract: A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations. The operations include generating, at an occupancy estimation network, an occupancy probability at one or more voxels, predicting, via a gaze prediction model, a gaze direction of a gaze prediction, and identifying, based on the predicted gaze direction, an object of interest. The operations also include generating, via an occupancy estimation application, a gaze saliency map based on the gaze direction and identified object of interest, and updating, based on the determined gaze direction and the gaze saliency map, the occupancy probability of the one or more voxels.

Abstract: A method of analyzing includes obtaining a dataset having images of an area surrounding a vehicle and identifying at least one object in each image of the images. Eye-gaze information directed to an operator of the vehicle is obtained from an eye-gaze monitoring system. The eye-gaze information includes an eye-gaze direction of the operator corresponding to each of the images. A subset of images from the images is identified for performing additional data labeling based on a relationship between the eye-gaze direction to the at least one object identified in each image of the images.

Abstract: An assistance system includes: a perception module configured to collect sensor data including data tracking behavior of a vehicle occupant in a host vehicle, and to determine perception information describing a current situation warranting initiation of an interactive dialog with the vehicle occupant; an interactive goal module configured to determine an interactive goal based on the determined perception information; an interaction timing module configured to determine timing for initiating the interactive dialog based on the interactive goal; a dialog module configured to, based on the perception information and the timing, initiate the interactive dialog to provide at least one of a suggestion and an offer to the vehicle occupant; and a vehicle control module configured to control operation of at least one of a device and a system of the host vehicle in response to the vehicle occupant accepting the at least one of the suggestion and the offer.

Abstract: A system and method for estimation of a driver state based on eye gaze includes, capturing and sending, using an outward looking camera situated in a vehicle, a first video stream of surrounding environment to a neural controller. The neural controller, based on the first video stream, generates an expected gaze distribution. Using an inward looking camera situated in the vehicle, the camera captures and sends a second video stream of a face of a driver to an eye tracker controller, where based on the second video stream, the eye tracker controller extracts a plurality of gaze directions. A gaze distribution module generates, based on the plurality of gaze directions, an actual gaze distribution. A distance distribution controller, based on a difference between the expected gaze distribution and the actual gaze distribution, generates a distance measure where a determination is made that the distance measure exceeds a threshold.

Abstract: A method of training a disparity estimation network. The method includes obtaining an eye-gaze dataset having first images with at least one gaze direction associated with each of the first images. A gaze prediction neural network is trained based on the eye-gaze dataset to develop a model trained to provide a gaze prediction for an external image. A depth database is obtained that includes second images having depth information associated with each of the second images. A disparity estimation neural network for object detection is trained based on an output from the gaze prediction neural network and an output from the depth database.

Abstract: Embodiments for generating appropriate data sets for learning to identify user actions. A user uses one or more applications over a suitable period of time. As the user uses the applications, a monitoring device, acting as a “man-in-the-middle,” intermediates the exchange of encrypted communication between the applications and the servers that serve the applications. The monitoring device obtains, for each action performed by the user, two corresponding (bidirectional) flows of communication: an encrypted flow, and an unencrypted flow. Since the unencrypted flow indicates the type of action that was performed by the user, the correspondence between the encrypted flow and the unencrypted flow may be used to automatically label the encrypted flow, without decrypting the encrypted flow. Features of the encrypted communication may then be stored in association with the label to automatically generate appropriately-sized learning set for each application of interest.

Abstract: Methods and systems for analyzing encrypted traffic, such as to identify, or “classify,” the user actions that generated the traffic. Such classification is performed, even without decrypting the traffic, based on features of the traffic. Such features may include statistical properties of (i) the times at which the packets in the traffic were received, (ii) the sizes of the packets, and/or (iii) the directionality of the packets. To classify the user actions, a processor receives the encrypted traffic and ascertains the types (or “classes”) of user actions that generated the traffic. Unsupervised or semi-supervised transfer-learning techniques may be used to perform the classification process. Using transfer-learning techniques facilitates adapting to different runtime environments, and to changes in the patterns of traffic generated in these runtime environments, without requiring the large amount of time and resources involved in conventional supervised-learning techniques.

Abstract: There is provided a method for matching subject data to database patient data based on matching phenotypes and related genetic sequences, comprising: receiving a dataset including at least one phenotype disease description of a subject and a genetic sequence of the subject, the phenotype disease description describing clinically significant manifestations of disease in the subject; calculating a ranking score for each of a dataset of patients, the ranking score indicative of a similarity correlation between the dataset of each respective patient and the dataset of the subject, wherein the related genetic sequences of the dataset of patients are underlying genetic mutations attributable to the at least one phenotypic disease description; matching the dataset of the subject with at least one dataset of patients according to a requirement of the ranking score; and providing data indicative of the matched patients.