Abstract: A method for compressing a deep neural network includes determining a pruning ratio for a channel and a mixed-precision quantization bit-width based on an operational budget of a device implementing the deep neural network. The method further includes quantizing a weight parameter of the deep neural network and/or an activation parameter of the deep neural network based on the quantization bit-width. The method also includes pruning the channel of the deep neural network based on the pruning ratio.

Abstract: Certain aspects of the present disclosure provide techniques for performing federated learning, including receiving a global model from a federated learning server; determining an updated model based on the global model and local data; and sending the updated model to the federated learning server using relative entropy coding.

Abstract: In one embodiment, a method of simulating an operation of an artificial neural network on a binary neural network processor includes receiving a binary input vector for a layer including a probabilistic binary weight matrix and performing vector-matrix multiplication of the input vector with the probabilistic binary weight matrix, wherein the multiplication results are modified by simulated binary-neural-processing hardware noise, to generate a binary output vector, where the simulation is performed in the forward pass of a training algorithm for a neural network model for the binary-neural-processing hardware.

Abstract: Aspects described herein provide techniques for performing federated learning of a machine learning model, comprising: for each respective client of a plurality of clients and for each training round in a plurality of training rounds: generating a subset of model elements for the respective client based on sampling a gate probability distribution for each model element of a set of model elements for a global machine learning model; transmitting to the respective client: the subset of model elements; and a set of gate probabilities based on the sampling, wherein each gate probability of the set of gate probabilities is associated with one model element of the subset of model elements; receiving from each respective client of the plurality of clients a respective set of model updates; and updating the global machine learning model based on the respective set of model updates from each respective client of the plurality of clients.

Abstract: A method of collaboratively training a neural network model, includes receiving a local update from a subset of the multiple users. The local update is related to one or more subsets of a dataset of the neural network model. A local component of the neural network model identifies a subset of the one or more subsets to which a data point belongs. A global update is computed for the neural network model based on the local updates from the subset of the users. The global updates for each portion of the network are aggregated to train the neural network model.

Abstract: Aspects described herein provide a method of processing data, including: receiving a set of global parameters for a plurality of machine learning models; processing data stored locally on an processing device with the plurality of machine learning models according to the set of global parameters to generate a machine learning model output; receiving, at the processing device, user feedback regarding machine learning model output for the plurality of machine learning models; performing an optimization of the plurality of machine learning models based on the machine learning output and the user feedback to generate locally updated machine learning model parameters; sending the locally updated machine learning model parameters to a remote processing device; and receiving a set of globally updated machine learning model parameters for the plurality of machine learning models.

Abstract: A method for quantizing a neural network includes modeling noise of parameters of the neural network. The method also includes assigning grid values to each realization of the parameters according to a concrete distribution that depends on a local fixed-point quantization grid and the modeled noise and. The method further includes computing a fixed-point value representing parameters of a hard fixed-point quantized neural network.

Abstract: A method for compressing a deep neural network includes determining a pruning ratio for a channel and a mixed-precision quantization bit-width based on an operational budget of a device implementing the deep neural network. The method further includes quantizing a weight parameter of the deep neural network and/or an activation parameter of the deep neural network based on the quantization bit-width. The method also includes pruning the channel of the deep neural network based on the pruning ratio.

Abstract: In one embodiment, a method of simulating an operation of an artificial neural network on a binary neural network processor includes receiving a binary input vector for a layer including a probabilistic binary weight matrix and performing vector-matrix multiplication of the input vector with the probabilistic binary weight matrix, wherein the multiplication results are modified by simulated binary-neural-processing hardware noise, to generate a binary output vector, where the simulation is performed in the forward pass of a training algorithm for a neural network model for the binary-neural-processing hardware.

Abstract: A neural network may be configured to receive, during a training phase of the neural network, a first input at an input layer of the neural network. The neural network may determine, during the training phase, a first classification at an output layer of the neural network based on the first input. The neural network may adjust, during the training phase and based on a comparison between the determined first classification and an expected classification of the first input, weights for artificial neurons of the neural network based on a loss function. The neural network may output, during an operational phase of the neural network, a second classification determined based on a second input, the second classification being determined by processing the second input through the artificial neurons using the adjusted weights.

Abstract: A method for quantizing a neural network includes modeling noise of parameters of the neural network. The method also includes assigning grid values to each realization of the parameters according to a concrete distribution that depends on a local fixed-point quantization grid and the modeled noise and. The method further includes computing a fixed-point value representing parameters of a hard fixed-point quantized neural network.