Abstract: A learning support device stores, as execution information, a set of an input value, an execution output value as an execution result and an expected output value, for a machine learning program. An arithmetic device generates an output value different from the expected output value as a teacher output value in accordance with a predetermined rule when the execution output value and the expected output value for a predetermined input value match each other, generates a loss function based on a difference between the teacher output value and the execution output value, generates a change value indicating a change in the loss function for each parameter in the neural network, calculates an influence degree of predetermined learning by calculating influence of an update value of the parameter obtained by the learning on the change value, and determines whether or not the update value is adopted based on the influence degree.

Abstract: A validity of a prediction model can be evaluated comprehensively. A machine learning program verification apparatus 100 includes a calculation device 104. The calculation device 104 obtains a decision tree logical expression by logically combining path logical expressions indicating decision tree paths indecision trees for a program created by machine learning, creates a combined logical expression by logically combining a verification property logical expression and an objective variable calculation logical expression with the decision tree logical expression, performs satisfiability determination by inputting the combined logical expression to a satisfiability determiner, and when a result of the determination indicates satisfaction, obtains, from a satisfaction solution of the satisfiability determination, a violation input value that is a value of an explanatory variable that violates a verification property and a violation output value that is a value of an objective variable.

Abstract: The boundary search test support device includes: a storage device that holds a plurality of input vectors; and an arithmetic device that executes a test by sequentially inputting the input vectors to a program generated by a neural network and acquiring output vectors which are test results, respectively generates, in a coordinate system which takes each of a predetermined plurality of elements among elements constituting the output vectors as a coordinate axis, a straight line in which the plurality of elements has a same value and a hyperplane in which a sum of values of the plurality of elements is taken as a predetermined value, and arranges a most antagonistic point and boundary vectors whose values of the elements rank higher than or equal to a predetermined ranking among the output vectors in the coordinate system, and outputs the coordinate system together with input vectors corresponding to the boundary vectors.

Abstract: Software test equipment including a data conversion unit receives test input data to convert the test input data into software input data to be provided to software and model input data, a software execution unit receives the software input data, executes the test target software based on the software input data, and outputs an execution result, a model execution unit receives a reference model for the software to generate a model allowable output value range of the execution result obtained by executing the software, based on the model input data and the reference model, a difference analysis unit generates difference information based on the execution result output by the software execution unit and the model allowable output value range generated by the model execution unit, and an evaluation unit receives evaluation criteria and evaluates the behavior of the software based on the difference information and the evaluation criteria.

Abstract: A coverage test support device includes a memory device that stores a test case and specification content of each of a plurality of coverage indexes, and an arithmetic device that sequentially gives a test input value of each pair in the test case to a program created by a neural network, executes a predetermined number of tests, and acquires a test result of the tests and neuron information at the time of test execution, applies the acquired neuron information to the specification content of each coverage indexes and calculates a value for each coverage index, and identifies, among the coverage indexes, a coverage index in which an elongation rate of the calculated value shows a predetermined tendency, as a preferential coverage index that is to be used preferentially, when either the number of executions of the tests or the number of bugs in the test result exceeds a predetermined standard.

Abstract: A validity of a prediction model can be evaluated comprehensively. A machine learning program verification apparatus 100 includes a calculation device 104. The calculation device 104 obtains a decision tree logical expression by logically combining path logical expressions indicating decision tree paths indecision trees for a program created by machine learning, creates a combined logical expression by logically combining a verification property logical expression and an objective variable calculation logical expression with the decision tree logical expression, performs satisfiability determination by inputting the combined logical expression to a satisfiability determiner, and when a result of the determination indicates satisfaction, obtains, from a satisfaction solution of the satisfiability determination, a violation input value that is a value of an explanatory variable that violates a verification property and a violation output value that is a value of an objective variable.

Abstract: A learning support device stores, as execution information, a set of an input value, an execution output value as an execution result and an expected output value, for a machine learning program. An arithmetic device generates an output value different from the expected output value as a teacher output value in accordance with a predetermined rule when the execution output value and the expected output value for a predetermined input value match each other, generates a loss function based on a difference between the teacher output value and the execution output value, generates a change value indicating a change in the loss function for each parameter in the neural network, calculates an influence degree of predetermined learning by calculating influence of an update value of the parameter obtained by the learning on the change value, and determines whether or not the update value is adopted based on the influence degree.

Abstract: The boundary search test support device includes: a storage device that holds a plurality of input vectors; and an arithmetic device that executes a test by sequentially inputting the input vectors to a program generated by a neural network and acquiring output vectors which are test results, respectively generates, in a coordinate system which takes each of a predetermined plurality of elements among elements constituting the output vectors as a coordinate axis, a straight line in which the plurality of elements has a same value and a hyperplane in which a sum of values of the plurality of elements is taken as a predetermined value, and arranges a most antagonistic point and boundary vectors whose values of the elements rank higher than or equal to a predetermined ranking among the output vectors in the coordinate system, and outputs the coordinate system together with input vectors corresponding to the boundary vectors.

Abstract: A coverage test support device includes a memory device that stores a test case and specification content of each of a plurality of coverage indexes, and an arithmetic device that sequentially gives a test input value of each pair in the test case to a program created by a neural network, executes a predetermined number of tests, and acquires a test result of the tests and neuron information at the time of test execution, applies the acquired neuron information to the specification content of each coverage indexes and calculates a value for each coverage index, and identifies, among the coverage indexes, a coverage index in which an elongation rate of the calculated value shows a predetermined tendency, as a preferential coverage index that is to be used preferentially, when either the number of executions of the tests or the number of bugs in the test result exceeds a predetermined standard.

Abstract: Software test equipment including a data conversion unit receives test input data to convert the test input data into software input data to be provided to software and model input data, a software execution unit receives the software input data, executes the test target software based on the software input data, and outputs an execution result, a model execution unit receives a reference model for the software to generate a model allowable output value range of the execution result obtained by executing the software, based on the model input data and the reference model, a difference analysis unit generates difference information based on the execution result output by the software execution unit and the model allowable output value range generated by the model execution unit, and an evaluation unit receives evaluation criteria and evaluates the behavior of the software based on the difference information and the evaluation criteria.

Abstract: Mismatches between software specifications are detected by dependence relation verification.