Abstract: A computer-implemented method includes: initializing model parameters for training a neural network; performing a forward pass and backpropagation for a first minibatch of training data; determining a new weight value for each of a plurality of nodes of the neural network using a gradient descent of the first minibatch; for each determined new weight value, determining whether to update a running mean corresponding to a weight of a particular node; based on a determination to update the running mean, calculating a new mean weight value for the particular node using the determined new weight value; updating the weight parameters for all nodes based on the calculated new mean weight values corresponding to each node; assigning the running mean as the weight for the particular node when training on the first minibatch is completed; and reinitializing running means for all nodes at a start of training a second minibatch.

Abstract: A method of operating a heating ventilation and air conditioning (HVAC) system of a structure, includes collecting first sensor data corresponding to a parameter of the HVAC system, collecting second sensor data that is different than the first sensor data, and generating clustered data by clustering the first sensor data and the second sensor data into a plurality of data clusters with a controller. The method also includes forming a transactional dataset based on at least the first sensor data, the second sensor data, and the clustered data with the controller, performing association rule mining (ARM) on the transactional dataset to generate a plurality of rules for each data cluster of the plurality of data clusters with the controller, and changing an operating characteristic of the HVAC system based on the plurality of rules with the controller to optimize the parameter.

Abstract: A computer-implemented method includes: initializing model parameters for training a neural network; performing a forward pass and backpropagation for a first minibatch of training data; determining a new weight value for each of a plurality of nodes of the neural network using a gradient descent of the first minibatch; for each determined new weight value, determining whether to update a running mean corresponding to a weight of a particular node; based on a determination to update the running mean, calculating a new mean weight value for the particular node using the determined new weight value; updating the weight parameters for all nodes based on the calculated new mean weight values corresponding to each node; assigning the running mean as the weight for the particular node when training on the first minibatch is completed; and reinitializing running means for all nodes at a start of training a second minibatch.

Abstract: A method of operating a heating ventilation and air conditioning (HVAC) system of a structure, includes collecting first sensor data corresponding to a parameter of the HVAC system, collecting second sensor data that is different than the first sensor data, and generating clustered data by clustering the first sensor data and the second sensor data into a plurality of data clusters with a controller. The method also includes forming a transactional dataset based on at least the first sensor data, the second sensor data, and the clustered data with the controller, performing association rule mining (ARM) on the transactional dataset to generate a plurality of rules for each data cluster of the plurality of data clusters with the controller, and changing an operating characteristic of the HVAC system based on the plurality of rules with the controller to optimize the parameter.

Abstract: A method of training a supervised neural network to solve an optimization problem that involves minimizing an error function ƒ(?) where ? is a vector of independent and identically distributed (i.i.d.) samples of a target distribution £t is proposed. The method includes generating an adversarial probabilistic regularizer (APR) ?£t(?) using a discriminator of a generative adversarial network. The discriminator receives samples from ? and samples from a regularizer distribution pr as inputs. The APR ?£t(?) is then added to the error function ƒ(?) for each training iteration of the supervised neural network.

Abstract: A search framework for finding effective architectural building blocks for deep convolutional neural networks is disclosed. The search framework described herein utilizes a building block which incorporates branch and skip connections. At least some operations of the architecture of the building block are undefined and treated as hyperparameters which can be automatically selected and optimized for a particular task. The search framework uses random search over the reduced search space to generate a building block and repeats the building block multiple times to create a deep convolutional neural network.

Abstract: A search framework for finding effective architectural building blocks for deep convolutional neural networks is disclosed. The search framework described herein utilizes a building block which incorporates branch and skip connections. At least some operations of the architecture of the building block are undefined and treated as hyperparameters which can be automatically selected and optimized for a particular task. The search framework uses random search over the reduced search space to generate a building block and repeats the building block multiple times to create a deep convolutional neural network.