Abstract: Methods and systems are disclosed to reduce the time and memory complexities associated with automatic differentiation of tensor models. The disclosed embodiment consists of a tensor contraction gradient calculator (TCGC) method, a tensor automatic differentiation (TAD) method and a TAD system. The disclosed embodiment eliminates the need to compute partial derivatives or Jacobians for computing tensor gradients of tensor contractions and tensor models. The disclosed embodiment computes tensor gradients of any arbitrary tensor model automatically with both memory and time complexities asymptotically equal to those of the evaluation of tensor models that are theoretically the lowest achievable complexities.

Abstract: Methods and systems are disclosed to reduce the memory requirement of neural networks by encoding the coefficients of a neural network during training stage and decoding them during the inference. The disclosed embodiment consists of a neural network coefficient decoder (NNCD), a genetic algorithm-based encoding system (GAbES), a coefficient encoding method (CEM), and a genetic algorithm-based neural network coefficient encoding/decoding (GANCED) system. The design is consistent with both hardware and firmware and it can be implemented by bitwise operation as a hardware accelerator or easily computed by a traditional processing unit. The disclosed embodiment reduces the memory storage requirement of hardware implementation of neural networks. This reduction speeds up the processing of neural networks and reduces the dynamic power consumption of the circuit.