Abstract: An action-value function initializing unit inputs search information including a history of a solution, a constraint equation, and an initial state of a selectable domain of a decision variable, sets a decision variable selected in each step and a value of the decision variable as a policy, and initializes an action-value function including the policy, a selectable domain of a decision variable before policy decision, and a selectable domain of a decision variable after the policy decision as parameters. A search unit receives information of the action-value function initialized by the action-value function initializing unit, obtains a value of a corresponding action-value function from the policy, the domain of the decision variable before the policy decision, and a domain of the action-value function after the policy decision, searches for a policy in which the action-value function is largest, and searches for an optimum solution for the problem information.