Abstract: A machine learning system includes a coach machine learning system that uses machine learning to help a student machine learning system learn its system. By monitoring the student learning system, the coach machine learning system can learn (through machine learning techniques) “hyperparameters” for the student learning system that control the machine learning process for the student learning system. The machine learning coach could also determine structural modifications for the student learning system architecture. The learning coach can also control data flow to the student learning system.

Abstract: Computer-implemented, machine-learning systems and methods relate to a neural network having at least two subnetworks, i.e., a first subnetwork and a second subnetwork. The systems and methods estimate the partial derivative(s) of an objective with respect to (i) an output activation of a node in first subnetwork, (ii) the input to the node, and/or (iii) the connection weights to the node. The estimated partial derivative(s) are stored in a data store and provided as input to the second subnetwork. Because the estimated partial derivative(s) are persisted in a data store, the second subnetwork has access to them even after the second subnetwork has gone through subsequent training iterations. Using this information, subnetwork 160 can compute classifications and regression functions that can help, for example, in the training of the first subnetwork.

Abstract: Computer systems and methods modify a base deep neural network (DNN). The method comprises replacing the target node of the base DNN with a compound node to thereby create a modified base DNN. The compound node comprises at least first and second nodes. The first node is trained to detect target node patterns in inputs to the first node and the second node is trained to detect an absence of the target node patterns in inputs to the second node, and the first and second nodes are trained to be non-complementary.

Abstract: A machine learning system includes a coach machine learning system that uses machine learning to help a student machine learning system learn its system. By monitoring the student learning system, the coach machine learning system can learn (through machine learning techniques) “hyperparameters” for the student learning system that control the machine learning process for the student learning system. The machine learning coach could also determine structural modifications for the student learning system architecture. The learning coach can also control data flow to the student learning system.

Abstract: A machine learning system includes a coach machine learning system that uses machine learning to help a student machine learning system learn its system. By monitoring the student learning system, the coach machine learning system can learn (through machine learning techniques) “hyperparameters” for the student learning system that control the machine learning process for the student learning system. The machine learning coach could also determine structural modifications for the student learning system architecture. The learning coach can also control data flow to the student learning system.

Abstract: Data-dependent node-to-node knowledge sharing to increase the interpretability of the activation pattern of one or more nodes in a neural network, is implemented by a set of knowledge sharing links. Each link may comprise a knowledge providing node or other source P and a knowledge receiving node R. A knowledge sharing link can impose a node-specific regularization on the knowledge receiving node R to help guide the knowledge receiving node R to have an activation pattern that is more easily interpreted. The specification and training of the knowledge sharing links may be controlled by a cooperative human-AI learning supervisor system in which a human and an artificial intelligence system work cooperatively to improve the interpretability and performance of the client system.

Abstract: Methods and computer systems improve a trained base deep neural network by structurally changing the base deep neural network to create an updated deep neural network, such that the updated deep neural network has no degradation in performance relative to the base deep neural network on the training data. The updated deep neural network is subsequently training. Also, an asynchronous agent for use in a machine learning system comprises a second machine learning system ML2 that is to be trained to perform some machine learning task. The asynchronous agent further comprises a learning coach LC and an optional data selector machine learning system DS. The purpose of the data selection machine learning system DS is to make the second stage machine learning system ML2 more efficient in its learning (by selecting a set of training data that is smaller but sufficient) and/or more effective (by selecting a set of training data that is focused on an important task).

Abstract: Computer systems and methods cooperatively train multiple generators and a classifier. Cooperative training includes: training, through machine learning, the multiple generators such that each generator is trained according to a first objective to output examples of a designated classification category; training, through machine learning, the classifier to determine, for each generated by the multiple generators, which of the multiple generators generated the example; and back-propagating partial derivatives of an error cost function from the classifier to the multiple generators.

Abstract: A machine learning system includes a coach machine learning system that uses machine learning to help a student machine learning system learn its system. By monitoring the student learning system, the coach machine learning system can learn (through machine learning techniques) “hyperparameters” for the student learning system that control the machine learning process for the student learning system. The machine learning coach could also determine structural modifications for the student learning system architecture. The learning coach can also control data flow to the student learning system.

Abstract: Machine-learning computer system breaks a neural network into a plurality of modules and tracks the training process module-by-module and datum-by-datum, recording auxiliary information during one iteration of the training process for retrieval during a later iteration. Based on this auxiliary information, the computer system can make decisions that can greatly reduce the amount of computation required by the training process. The auxiliary information allows the computer system to diagnose and fix problems that occur during the training process on a module-by-module and/or datum-by-datum basis.

Abstract: Computer systems and methods generate data examples by training, through machine learning, a data generator with a training objective to produce a data example for a specific value of R, where R is value related to S1(x) and S2(x), where, for a data example, x, generated by the data generator, S1(x) is a likelihood that the data example x is in a first class of a first selected data example and S2(x) is a likelihood that the data example x is in a second class of a second selected data example. S1(x) and S2(x) are determined by a discriminator that is trained through machine learning to discriminate between the first and second classes. After training the data generator, the data generator generates a synthetic data example for each of multiple specific values of R.

Abstract: Systems and methods analyze training of a first machine learning system with a second machine learning system. The first machine learning system comprises a neural network with a first inner layer node. The method includes connecting the first machine learning system to an input of the second machine learning system. The second machine learning system comprises a second objective function for analyzing an internal characteristic of the first machine learning system and which is different from a first objective function for the first machine learning system.

Abstract: Computer systems and methods cooperatively train multiple generators and a classifier. Cooperative training includes: training, through machine learning, the multiple generators such that each generator is trained according to a first objective to output examples of a designated classification category; training, through machine learning, the classifier to determine, for each generated by the multiple generators, which of the multiple generators generated the example; and back-propagating partial derivatives of an error cost function from the classifier to the multiple generators.

Abstract: Computer systems and computer-implemented methods modify a machine learning network, such as a deep neural network, to introduce judgment to the network. A “combining” node is added to the network, to thereby generate a modified network, where activation of the combining node is based, at least in part, on output from a subject node of the network. The computer system then trains the modified network by, for each training data item in a set of training data, performing forward and back propagation computations through the modified network, where the backward propagation computation through the modified network comprises computing estimated partial derivatives of an error function of an objective for the network, except that the combining node selectively blocks back-propagation of estimated partial derivatives to the subject node, even though activation of the combining node is based on the activation of the subject node.

Abstract: Computer systems and methods generate a stochastic categorical autoencoder learning network (SCAN). The SCAN is trained to have an encoder network that outputs, subject to one or more constraints, parameters for parametric probability distributions of sample random variables from input data. The parameters comprise measures of central tendency and measures of dispersion. The one or more constraints comprise a first constraint that constrains a measure of a magnitude of a vector of the measures of central tendency as compared to a measure of a magnitude of a vector of the measures of dispersion. Thereafter, the sample random variables are generated from the parameters and a decoder is trained to output the input data from the sample random variables.

Abstract: A system and method for controlling a nodal network. The method includes estimating an effect on the objective caused by the existence or non-existence of a direct connection between a pair of nodes and changing a structure of the nodal network based at least in part on the estimate of the effect. A nodal network includes a strict partially ordered set, a weighted directed acyclic graph, an artificial neural network, and/or a layered feed-forward neural network.

Abstract: Methods and computer systems improve a trained base deep neural network by structurally changing the base deep neural network to create an updated deep neural network, such that the updated deep neural network has no degradation in performance relative to the base deep neural network on the training data. The updated deep neural network is subsequently training. Also, an asynchronous agent for use in a machine learning system comprises a second machine learning system ML2 that is to be trained to perform some machine learning task. The asynchronous agent further comprises a learning coach LC and an optional data selector machine learning system DS. The purpose of the data selection machine learning system DS is to make the second stage machine learning system ML2 more efficient in its learning (by selecting a set of training data that is smaller but sufficient) and/or more effective (by selecting a set of training data that is focused on an important task).

Abstract: Various systems and methods are described herein for improving the aggressive development of machine learning systems. In machine learning, there is always a trade-off between allowing a machine learning system to learn as much as it can from training data and overfitting on the training data. This trade-off is important because overfitting usually causes performance on new data to be worse. However, various systems and methods can be utilized to separate the process of detailed learning and knowledge acquisition and the process of imposing restrictions and smoothing estimates, thereby allowing machine learning systems to aggressively learn from training data, while mitigating the effects of overfitting on the training data.

Abstract: Methods and computer systems improve a trained base deep neural network by structurally changing the base deep neural network to create an updated deep neural network, such that the updated deep neural network has no degradation in performance relative to the base deep neural network on the training data. The updated deep neural network is subsequently training. Also, an asynchronous agent for use in a machine learning system comprises a second machine learning system ML2 that is to be trained to perform some machine learning task. The asynchronous agent further comprises a learning coach LC and an optional data selector machine learning system DS. The purpose of the data selection machine learning system DS is to make the second stage machine learning system ML2 more efficient in its learning (by selecting a set of training data that is smaller but sufficient) and/or more effective (by selecting a set of training data that is focused on an important task).

Abstract: A machine learning system includes a coach machine learning system that uses machine learning to help a student machine learning system learn its system. By monitoring the student learning system, the coach machine learning system can learn (through machine learning techniques) “hyperparameters” for the student learning system that control the machine learning process for the student learning system. The machine learning coach could also determine structural modifications for the student learning system architecture. The learning coach can also control data flow to the student learning system.