Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive effective learning rate while also ensuring that the effective learning rate is non-increasing.

Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive effective learning rate while also ensuring that the effective learning rate is non-increasing.

Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive learning rate while also ensuring that the learning rate is non-increasing.

Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive learning rate while also ensuring that the learning rate is non-increasing.

Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive effective learning rate while also ensuring that the effective learning rate is non-increasing.

Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive effective learning rate while also ensuring that the effective learning rate is non-increasing.

Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive effective learning rate while also ensuring that the effective learning rate is non-increasing.

Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive learning rate while also ensuring that the learning rate is non-increasing.

Abstract: One or more computing devices, systems, and/or methods for message classification are provided. For example, a set of messages is clustered into a set of clusters. A cluster comprises messages with similar features (e.g., similar subject lines, message body content, sender information, recipient information, structure, user action such as reading or deleting, spam vote information, etc.). Cluster features are computed for the clusters based upon features of messages within such clusters. A first table, comprising cluster entries corresponding cluster features of clusters, and a second table, comprising message entries corresponding to clusters to which messages are assigned, are created. Message features of a message are created, using the first table and second table, based upon features of the message and cluster features of clusters to which the message is assigned. A message classifier is used to classify the message (e.g., spam, safe, a threat, etc.) based upon the message features.

Abstract: Systems and methods for evaluating a loss function or a gradient of the loss function. In one example embodiment, a computer-implemented method includes partitioning a weight matrix into a plurality of blocks. The method includes identifying a first set of labels for each of the plurality of blocks with a score greater than a first threshold value. The method includes constructing a sparse approximation of a scoring vector for each of the plurality of blocks based on the first set of labels. The method includes determining a correction value for each sparse approximation of the scoring vector. The method includes determining an approximation of a loss or a gradient of a loss associated with the scoring function based on each sparse approximation of the scoring vector and the correction value associated with the sparse approximation of the scoring vector.

Abstract: Systems and methods for evaluating a loss function or a gradient of the loss function. In one example embodiment, a computer-implemented method includes partitioning a weight matrix into a plurality of blocks. The method includes identifying a first set of labels for each of the plurality of blocks with a score greater than a first threshold value. The method includes constructing a sparse approximation of a scoring vector for each of the plurality of blocks based on the first set of labels. The method includes determining a correction value for each sparse approximation of the scoring vector. The method includes determining an approximation of a loss or a gradient of a loss associated with the scoring function based on each sparse approximation of the scoring vector and the correction value associated with the sparse approximation of the scoring vector.

Abstract: One or more computing devices, systems, and/or methods for message classification are provided. For example, a set of messages is clustered into a set of clusters. A cluster comprises messages with similar features (e.g., similar subject lines, message body content, sender information, recipient information, structure, user action such as reading or deleting, spam vote information, etc.). Cluster features are computed for the clusters based upon features of messages within such clusters. A first table, comprising cluster entries corresponding cluster features of clusters, and a second table, comprising message entries corresponding to clusters to which messages are assigned, are created. Message features of a message are created, using the first table and second table, based upon features of the message and cluster features of clusters to which the message is assigned. A message classifier is used to classify the message (e.g., spam, safe, a threat, etc.) based upon the message features.

Abstract: Described is a paid search advertising technology in which ratings values are computed for advertisements (or other web content items) based upon head-to-head evaluations as to which advertisement or advertisements were selected (clicked) from among a set of advertisements that were shown together. Records of a query log contain data of advertisements that were shown together, and each advertisement of those shown together that was clicked. Pairs of advertisements are selected, with the rating value of an advertisement that was clicked increased, and the rating value of the non-clicked advertisement decreased. Only those pairs in which one advertisement was selected and another was not selected may be used as a pair. Elo ratings formulas may be employed for the increasing and decreasing computations. The rating values may be combined with other prediction results into a combined result used to select and/or rank advertisements for returning with a query response.