Abstract: Systems and methods for managing visual allocation are provided herein that use models to determine states based on visual data and, based thereon, output feedback based on the determined states. Visual data is initially obtained by a visual allocation management system. The visual data includes eye image sequences of a person in a particular state, such as engaging in a task or activity. Visual features can be identified from the visual data, such that glance information including direction and duration can be calculated. The visual data, information derived therefrom, and/or other contextual data is input into the models, which correspond to states, to calculate probabilities that the particular state that the person is engaged in is one of the modeled states. Based on the state identified as having the highest probability, an optimal feedback, such as a warning or instruction, can be output to a connected devices, systems, or objects.

Abstract: A method to generate synthetic data instances. The method includes generating a synthetic data instance for an input variable value of an input variable supplied to the generative model, classifying the synthetic data instance to generate a classification result, determining a loss function value of a loss function, the loss function evaluating the classification result and determining the gradient of the loss function with respect to the input variable. Depending on the absolute value of the gradient, the method includes generating a plurality of modified input variable values, determining, for each modified input variable value, the gradient of the loss function, combining the gradients of the loss function to generate an estimated gradient, and modifying the input variable value in a direction determined by the estimated gradient to generate a further input variable value. The generative model generates a further synthetic data instance for the further input variable value.

Abstract: A method for generating a counterfactual data sample for a neural network based on an input sensor data sample is described. The method includes determining, using the neural network, a class prediction for the input sensor data sample, determining, in addition to the class prediction, an estimate of the uncertainty of the class prediction, generating a candidate counterfactual data sample for which the neural network determines a different class prediction than for the input sensor data sample, determining a loss function, wherein the loss function includes the estimate of the uncertainty of the class prediction by the neural network for the candidate counterfactual data sample, modifying the candidate counterfactual data sample to obtain a counterfactual data sample based on the determined loss function and outputting the counterfactual data sample.

Abstract: Systems and methods for managing visual allocation are provided herein that use models to determine states based on visual data and, based thereon, output feedback based on the determined states. Visual data is initially obtained by a visual allocation management system. The visual data includes eye image sequences of a person in a particular state, such as engaging in a task or activity. Visual features can be identified from the visual data, such that glance information including direction and duration can be calculated. The visual data, information derived therefrom, and/or other contextual data is input into the models, which correspond to states, to calculate probabilities that the particular state that the person is engaged in is one of the modeled states. Based on the state identified as having the highest probability, an optimal feedback, such as a warning or instruction, can be output to a connected devices, systems, or objects.

Abstract: A method for operating a neural network is described comprising determining, for neural network input sensor data, neural network output data using the neural network, selecting a portion of output data points to form a region of interest and determining, for each of at least some output data points outside the region of interest, a contribution value representing a contribution of one or more input data points associated with the output data point for the neural network determining the output data point values assigned to output data points in the region of interest.

Abstract: A system for training a generative model and a discriminative model. The generative model generates synthetic instances from latent feature vectors by generating an intermediate representation from the latent feature vector and generating the synthetic instance from the intermediate representation. The discriminative model determines multiple discriminator scores for multiple parts of an input instance, indicating whether the part is from a synthetic instance or an actual instance. The generative model is trained by backpropagation. During the backpropagation, partial derivatives of the loss with respect to entries of the intermediate representation are updated based on a discriminator score for a part of the synthetic instance, wherein the part of the synthetic instance is generated based at least in part on the entry of the intermediate representation, and wherein the partial derivative is decreased in value if the discriminator score indicates an actual instance.

Abstract: Systems and methods for managing visual allocation are provided herein that use models to determine states based on visual data and, based thereon, output feedback based on the determined states. Visual data is initially obtained by a visual allocation management system. The visual data includes eye image sequences of a person in a particular state, such as engaging in a task or activity. Visual features can be identified from the visual data, such that glance information including direction and duration can be calculated. The visual data, information derived therefrom, and/or other contextual data is input into the models, which correspond to states, to calculate probabilities that the particular state that the person is engaged in is one of the modeled states. Based on the state identified as having the highest probability, an optimal feedback, such as a warning or instruction, can be output to a connected devices, systems, or objects.