Abstract: Machine learning techniques which allow machine learning to be performed even when a cost function is not a convex function are provided. A machine learning system includes a plurality of node portions which learn mapping that uses one common primal variable by machine learning based on their respective input data while sending and receiving information to and from each other. The machine learning is performed so as to minimize, instead of a cost function of a non-convex function originally corresponding to the machine learning, a proxy convex function serving as an upper bound on the cost function. The proxy convex function is represented by a formula of a first-order gradient of the cost function with respect to the primal variable or by a formula of a first-order gradient and a formula of a second-order gradient of the cost function with respect to the primal variable.

Abstract: Techniques for achieving a scalable DNN and a multi-task DNN, for example, are provided. A neural network system is a neural network system including a plurality of models. Each one of the plurality of models is a DNN including a plurality of layers. Some or all of the plurality of models include at least one layer for which some or all of model variables are equivalent or common (hereinafter referred to as a “shared layer”) and also include at least one layer for which the model variables are not equivalent nor common (hereinafter referred to as an “unshared layer”).