Abstract: An iterative learning machine uses, as a direction of changing iterative weight, a conjugate gradient direction in place of the conventional steepest descent direction, thereby saving time. Learning rates are set dynamically. Error calculation for plural learning rates, with respect to a certain weight changing direction, are accomplished by storing a product-sum of the input signals and weights in a hidden layer and a product-sum of the input signals and the weight changing direction in the hidden layer. When the learning falls into a non-effective state where further iteration does not effectively reduce an error, the weights are adjusted in order to restart the learning.