Abstract: A neural network structure is separated into an odd neural network including only the odd layers and an even neural network including only the even layers. In order to allow for parallel execution, for forward propagation a second input is generated from the original input, while for backward propagation a second error gradient is generated. Parallel execution may accelerate the forward and backward propagation operations without significant change in accuracy of the model. Additionally, restructuring a single neural network into two or more parallel neural networks may reduce the total time needed for training.

Abstract: Herein are techniques that train regressor(s) to predict how effective would a machine learning model (MLM) be if trained with new hyperparameters and/or dataset. In an embodiment, for each training dataset, a computer derives, from the dataset, values for dataset metafeatures. The computer performs, for each hyperparameters configuration (HC) of a MLM, including landmark HCs: configuring the MLM based on the HC, training the MLM based on the dataset, and obtaining an empirical quality score that indicates how effective was said training the MLM when configured with the HC. A performance tuple is generated that contains: the HC, the values for the dataset metafeatures, the empirical quality score and, for each landmark configuration, the empirical quality score of the landmark configuration and/or the landmark configuration itself. Based on the performance tuples, a regressor is trained to predict an estimated quality score based on a given dataset and a given HC.

Abstract: A computer-implemented method, a computer program product, and a computer system for incremental and decentralized pruning of a machine learning model in federated learning. A federated learning system determines a serial sequence of participating in model pruning by agents in the federated learning system. A server in the federated learning system sends, to a first agent in the serial sequence, an initial model to trigger a federated pruning process for the machine learning model. The each of agents in the serial sequence prunes the machine learning model. The each of agents in the serial sequence generates an intermediately pruned model for an immediately next agent to prune. A final agent in the serial sequence prunes the machine learning model and generates a finally pruned model. The final agent sends the finally pruned model to the server.

Abstract: Detecting sequences of computer-executed operations, including training a BLSTM to determine forward and backward probabilities of encountering each computer-executed operations within a training set of consecutive computer-executed operations in forward and backward execution directions of the operations, and identifying reference sequences of operations within the training set where for each given one of the sequences the forward probability of encountering a first computer-executed operation in the given sequence is below a predefined lower threshold, the forward probability of encountering a last computer-executed operation in the given sequence is above a predefined upper threshold, the backward probability of encountering the last computer-executed operation in the given sequence is below the predefined lower threshold, and the backward probability of encountering the first computer-executed operation in the given sequence is above the predefined upper threshold, and where the predefined lower threshold

Abstract: The present disclosure provides systems and methods for modification (e.g., pruning, compression, quantization, etc.) of artificial neural networks based on estimations of the utility of network connections (also known as “edges”). In particular, the present disclosure provides novel techniques for estimating the utility of one or more edges of a neural network in a fashion that requires far less expenditure of resources than calculation of the actual utility. Based on these estimated edge utilities, a computing system can make intelligent decisions regarding network pruning, network quantization, or other modifications to a neural network. In particular, these modifications can reduce resource requirements associated with the neural network. By making these decisions with knowledge of and based on the utility of various edges, this reduction in resource requirements can be achieved with only a minimal, if any, degradation of network performance (e.g., prediction accuracy).

Abstract: A method for training an automated learning system includes processing training input with a first neural network and processing the output of the first neural network with a second neural network. The input layer of the second neural network corresponding to the output layer of the first neural network. The output layer of the second neural network corresponding to the input layer of the first neural network. An objective function is determined using the output of the second neural network and a predetermined modification magnitude. The objective function is approximated using random Cauchy projections which are propagated through the second neural network.

Abstract: A system, an apparatus and methods for utilizing software and hardware portions of a neural network to fix, or hardwire, certain portions, while modifying other portions are provided. A first set of weights for layers of the first neural network are established, and selected weights are modified to generate a second set of weights, based on a second dataset. The second set of weights is then used to train a second neural network.

Abstract: Systems and methods for modeling complex probability distributions are described. One embodiment includes a method for training a restricted Boltzmann machine (RBM), wherein the method includes generating, from a first set of visible values, a set of hidden values in a hidden layer of a RBM and generating a second set of visible values in a visible layer of the RBM based on the generated set of hidden values. The method includes computing a set of likelihood gradients based on the first set of visible values and the generated set of visible values, computing a set of adversarial gradients using an adversarial model based on at least one of the set of hidden values and the set of visible values, computing a set of compound gradients based on the set of likelihood gradients and the set of adversarial gradients, and updating the RBM based on the set of compound gradients.

Abstract: A method of training a neural network includes, at a local computing node, receiving remote parameters from a set of one or more remote computing nodes, initiating execution of a forward pass in a local neural network in the local computing node to determine a final output based on the remote parameters, initiating execution of a backward pass in the local neural network to determine updated parameters for the local neural network, and prior to completion of the backward pass, transmitting a subset of the updated parameters to the set of remote computing nodes.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for training machine learning models. One method includes obtaining a machine learning model, wherein the machine learning model comprises one or more model parameters, and the machine learning model is trained using gradient descent techniques to optimize an objective function; determining an update rule for the model parameters using a recurrent neural network (RNN); and applying a determined update rule for a final time step in a sequence of multiple time steps to the model parameters.

Abstract: This document relates to automated generation of machine learning models, such as neural networks. One example system includes a hardware processing unit and a storage resource. The storage resource can store computer-readable instructions cause the hardware processing unit to perform an iterative model-growing process that involves modifying parent models to obtain child models. The iterative model-growing process can also include selecting candidate layers to include in the child models based at least on weights learned in an initialization process of the candidate layers. The system can also output a final model selected from the child models.

Abstract: A hardware neural network conversion method, a computing device, a compiling method and a neural network software and hardware collaboration system for converting a neural network application into a hardware neural network fulfilling a hardware constraint condition are disclosed. The method comprises: obtaining a neural network connection diagram corresponding to the neural network application; splitting the neural network connection diagram into neural network basic units; converting each of the neural network basic units so as to form a network having equivalent functions thereto and formed by connecting basic module virtual entities of neural network hardware; and connecting the obtained basic unit hardware network according to the sequence of splitting so as to create a parameter file for the hardware neural network. The present disclosure provides a novel neural network and a brain-like computing software and hardware system.

Abstract: A neural network classifies an input signal. For example, an accelerometer signal may be classified to detect human activity. In a first convolutional layer, two-valued weights are applied to the input signal. In a first two-valued function layer coupled at input to an output of the first convolutional layer, a two-valued function is applied. In a second convolutional layer coupled at input to an output of the first two-valued functional layer, weights of the second convolutional layer are applied. In a fully-connected layer coupled at input to an output of the second convolutional layer, two-valued weights of the fully connected layer are applied. In a second two-valued function layer coupled at input to an output of the fully connected layer, a two-valued function of the second two-valued function layer is applied. A classifier classifies the input signal based on an output signal of second two-valued function layer.

Abstract: A system for training a deep learning system to detect engine knock with accuracy associated with high fidelity knock detection sensors despite using data from a low fidelity knock detection sensor. The system includes an engine, a high fidelity knock detection sensor, a low fidelity knock detection sensor, and an electronic processor. The electronic processor is configured to receive first data from the high fidelity knock detection sensor. The electronic processor is also configured to receive second data from the low fidelity knock detection sensor. The electronic processor is further configured to map the first data to the second data, train the deep learning system, using training data including the mapped data, to determine a predicted peak pressure using data from the low fidelity knock detection sensor, receive third data from the low fidelity knock detection sensor, and using the third data, determine the predicted peak pressure.

Abstract: Disclosed herein are techniques for scheduling and executing multi-layer neural network computations for multiple contexts. In one embodiment, a method comprises determining a set of computation tasks to be executed, the set of computation tasks including a first computation task and a second computation task, as well as a third computation task and a fourth computation task to provide input data for the first and second computation tasks; determining a first execution batch comprising the first and second computation tasks; determining a second execution batch comprising at least the third computation task to be executed before the first execution batch; determining whether to include the fourth computation task in the second execution batch based on whether the memory device has sufficient capacity to hold input data and output data of both of the third and fourth computation; executing the second execution batch followed by the first execution batch.

Abstract: A computer-implemented method for controlling a device based on an ensemble model can include receiving sensing information associated with a user's biometric state; inputting first sensing information to a first model, determining a first uncertainty of the first model, and generating a first weight value for weighting a first result value; inputting second sensing information into a second model, determining a second uncertainty of the second model, and generating a second weight value for weighting a second result value; generating a final result value based on combining the first result value weighted by the first weight value and the second result value weighted by the second weight value; generating a predicted biometric state of the user based on the final result value; and executing an operation of the device based on the predicted biometric state.

Abstract: A method for de-prejudicing Artificial Intelligence (AI) based anomaly detection is disclosed. The method includes training and testing an AI model based on a labelled training data, determining whether the AI model reveals a bias, based on one or more prejudicing variables, and thereafter re-building the AI model based on iterative process of de-prejudicing the feature set of the AI model and de-prejudicing the training data. A check is made to determine whether the feature set of the AI model feature set includes any proxy variables associated with any of the prejudicing variables and identifies the weight to be assigned to a proxy variable based on the intra-cohort variation in separate machine learning models built for each cohort associated with each value of the prejudicing variable. The feature set of the AI model is de-prejudiced based on the explanatory power of the proxy variables independent of the prejudicing variables.