Abstract: A neural network is trained using a training neural network having the same topology as the original network but having a differential network output and accepting also differential network inputs. This new training method enables deeper neural networks to be successfully trained by avoiding a problem occuring in conventional training methods in which errors vanish as they are propagated in the reverse direction through deep networks. An acceleration in convergence rate is achieved by adjusting the error used in training to compensate for the linkage between multiple training data points.

Abstract: A new technique for adjusting the parameters of a feedback controller. A linear feedback controller includes proportional, integral and derivative (PID) controllers but is not limited to them. The feedback controller may utilize non-linearity in its control and still have its parameters adjusted by this algorithm. The technique involves calculating an error derivative signal, which is the derivative of the processes loops' error with a parameter, for each parameter being adjusted and utilizing these error derivative signals along with error using a modified LMS algorithm. A key aspect of the modified LMS algorithm is the development of the error derivative signals.