Abstract: Provided are systems and methods for generating a score for any model which can be updated online, regardless of model type architecture and parameters, leveraging relations between regret and uncertainty.

Abstract: Systems and methods according to the present disclosure can employ a computer-implemented method for inference using a machine-learned model. The method can be implemented by a computing system having one or more computing devices. The method can include obtaining data descriptive of a neural network including one or more network units and one or more gating paths, wherein each of the gating path(s) includes one or more gating units. The method can include obtaining data descriptive of one or more input features. The method can include determining one or more network unit outputs from the network unit(s) based at least in part on the input feature(s). The method can include determining one or more gating values from the gating path(s). The method can include determining one or more gated network unit outputs based at least in part on a combination of the network unit output(s) and the gating value(s).

Abstract: Systems and methods according to the present disclosure can employ a computer-implemented method for inference using a machine-learned model. The method can be implemented by a computing system having one or more computing devices. The method can include obtaining data descriptive of a neural network including one or more network units and one or more gating paths, wherein each of the gating path(s) includes one or more gating units. The method can include obtaining data descriptive of one or more input features. The method can include determining one or more network unit outputs from the network unit(s) based at least in part on the input feature(s). The method can include determining one or more gating values from the gating path(s). The method can include determining one or more gated network unit outputs based at least in part on a combination of the network unit output(s) and the gating value(s).

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, that obtain a first machine learning model that is configured to output a score. The training examples can each include feature values that represent features of an item, and an outcome label for the item. From the training examples, training pairs of training examples are determined. For each training pair: (i) a score is generated for each training example in the training pair using the first machine learning model; and (ii) for the training pair, a score difference of the scores generated for the training examples in the training pair is determined. Using the training pairs and the score differences, a second machine learning model is trained to produce score differences that, for the same training examples, are within a threshold value of the score differences produced by the first machine learning model.

Abstract: Thus, aspects of the present disclosure address model “blow up” by changing the functionality of the activation, thereby providing “dead” or “dying” neurons with the ability to recover from this situation. As one example, for activation functions that have an input region in which the neuron is turned off by a 0 or close to 0 gradient, a training computing system can keep the neuron turned off when the gradient pushes the unit farther into the region (e.g., by applying an update with zero or reduced magnitude). However, if the gradient for the current training example (or batch) attempts to push the unit towards a region in which the neuron is active again, the system can allow for a non-zero gradient (e.g., by applying an update with standard or increased magnitude).

Abstract: Systems and methods leverage low complexity (e.g., linear overall, fixed per example) analytical approximations to perform machine learning problems such as, for example, the sparse online logistic regression problem. Unlike variational inference and other methods, the proposed systems and methods lead to analytical closed forms, lowering the practical number of computations. Further, unlike techniques used for dense features sets, such as Gaussian Mixtures, the proposed systems and methods allow for sparse problems with huge feature sets without increasing complexity. With the analytical closed forms, there is also no need for applying stochastic gradient methods on surrogate losses, and for tuning and balancing learning and regularization parameters of such methods.

Abstract: Thus, aspects of the present disclosure address model “blow up” by changing the functionality of the activation, thereby providing “dead” or “dying” neurons with the ability to recover from this situation. As one example, for activation functions that have an input region in which the neuron is turned off by a 0 or close to 0 gradient, a training computing system can keep the neuron turned off when the gradient pushes the unit farther into the region (e.g., by applying an update with zero or reduced magnitude). However, if the gradient for the current training example (or batch) attempts to push the unit towards a region in which the neuron is active again, the system can allow for a non-zero gradient (e.g., by applying an update with standard or increased magnitude).

Abstract: Systems and methods according to the present disclosure can employ a computer-implemented method for inference using a machine-learned model. The method can be implemented by a computing system having one or more computing devices. The method can include obtaining data descriptive of a neural network including one or more network units and one or more gating paths, wherein each of the gating path(s) includes one or more gating units. The method can include obtaining data descriptive of one or more input features. The method can include determining one or more network unit outputs from the network unit(s) based at least in part on the input feature(s). The method can include determining one or more gating values from the gating path(s). The method can include determining one or more gated network unit outputs based at least in part on a combination of the network unit output(s) and the gating value(s).

Abstract: Systems and methods can improve the reproducibility of neural networks by distilling from ensembles. In particular, aspects of the present disclosure are directed to a training scheme that utilizes a combination of an ensemble of neural networks and a single, “wide” neural network that is more powerful (e.g., exhibits a greater accuracy) than the ensemble. Specifically, the output of the ensemble can be distilled into the single neural network during training of the single neural network. After training, the single neural network can be deployed to generate inferences. In such fashion, the single neural model can provide a superior prediction accuracy while, during training, the ensemble can serve to influence the single neural network to be more reproducible. In addition, an additional single wide tower can be added to generate another output, that can be distilled to the single neural network, to further improve its accuracy.

Abstract: Aspects of the present disclosure are directed to novel activation functions which enable improved reproducibility and accuracy tradeoffs in neural networks. In particular, the present disclosure provides a family of activation functions that, on one hand, are smooth with continuous gradient and optionally monotonic but, on the other hand, also mimic the mathematical behavior of a Rectified Linear Unit (ReLU). As examples, the activation functions described herein include a smooth rectified linear unit function and also a leaky version of such function. In various implementations, the proposed functions can provide both a complete stop region and a constant positive gradient (e.g., that can be 1) pass region like a ReLU, thereby matching accuracy performance of a ReLU. Additional implementations include a leaky version and/or functions that feature different constant gradients in the pass region.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage medium, for regularizing feature weights maintained by a machine learning model. The method includes actions of obtaining a set of training data that includes multiple training feature vectors, and training the machine learning model on each of the training feature vectors, comprising, for each feature vector and for each of a plurality of the features of the feature vector: determining a first loss for the feature vector with the feature, determining a second loss for the feature vector without the feature, and updating a current benefit score for the feature using the first loss and the second loss, wherein the benefit score for the feature is indicative of the usefulness of the feature in generating accurate predicted outcomes for training feature vectors.

Abstract: The present disclosure relates generally to machine learning. More particularly, the present disclosure relates to systems and methods for improved generalization, reproducibility, and stabilization of neural networks via the application of error control, modulation, and/or lattice code constraints during training.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage medium, for regularizing feature weights maintained by a machine learning model. The method includes actions of obtaining a set of training data that includes multiple training feature vectors, and training the machine learning model on each of the training feature vectors, comprising, for each feature vector and for each of a plurality of the features of the feature vector: determining a first loss for the feature vector with the feature, determining a second loss for the feature vector without the feature, and updating a current benefit score for the feature using the first loss and the second loss, wherein the benefit score for the feature is indicative of the usefulness of the feature in generating accurate predicted outcomes for training feature vectors.

Abstract: A system and method of detecting trees in an image. A system and method may receive a dimension related to the trees in an input image. A two dimensional (2D) high pass filter may be applied to the input image to produce a high pass image. Objects may be marked in the high pass image based on the dimension. A processed image may be produced by associating a set of pixels in the high pass image with a respective set of grayscale values. A density operator may be applied to the processed image to identify locations with high frequency changes. Shapes may be defined to include the locations. Trees may be identified by grouping one or more shapes.

Abstract: A system and method for detecting and representing a directionality of objects in an image. A system and method may process an input image to produce a set of direction-filtered images, calculate a local gradient field based on the of direction-filtered images, calculate a magnitude of a projection of a local gradient on a predefined direction and use the projection to represent a directionality of the set of objects. A system and method may calculate a local orientation angle and associate the local orientation angle with pixels in an input digital image.

Abstract: A system and method of detecting trees in an image. A system and method may receive a dimension related to the trees in an input image. A two dimensional (2D) high pass filter may be applied to the input image to produce a high pass image. Objects may be marked in the high pass image based on the dimension. A processed image may be produced by associating a set of pixels in the high pass image with a respective set of grayscale values. A density operator may be applied to the processed image to identify locations with high frequency changes. Shapes may be defined to include the locations. Trees may be identified by grouping one or more shapes.

Abstract: A system and method for detecting and representing a directionality of objects in an image. A system and method may process an input image to produce a set of direction-filtered images, calculate a local gradient field based on the of direction-filtered images, calculate a magnitude of a projection of a local gradient on a predefined direction and use the projection to represent a directionality of the set of objects. A system and method may calculate a local orientation angle and associate the local orientation angle with pixels in an input digital image.

Abstract: A device and method for identifying plant rows in a field represented by an image is provided. The plant rows may be identified using the frequency domain. The plant rows may further be identified using information regarding plant positions. Additionally, plant rows may be obtained by any appropriate method and analyzed to differentiate between planted and non-planted rows. Further, plant rows may be segmented according to predefined classifications or attributes thereof, wherein the classification/attributes may derived from an image of the area in which the plant rows are found and/or using any other appropriate method.

Abstract: A device and method for identifying plant rows in a field represented by an image is provided. The plant rows may be identified using the frequency domain. The plant rows may further be identified using information regarding plant positions. Additionally, plant rows may be obtained by any appropriate method and analyzed to differentiate between planted and non-planted rows. Further, plant rows may be segmented according to predefined classifications or attributes thereof, wherein the classification/attributes may derived from an image of the area in which the plant rows are found and/or using any other appropriate method.

Abstract: Systems and methods for providing online content include determining the likelihood of an online event occurring regarding the content. A likelihood value may be generated by analyzing history data indicative of one or more online events to identify content presentations and content interactions. Content presentations and content interactions may be grouped by topical category, in some implementations.