Abstract: A method of generating a detection model to be used to detect accuracy deterioration of a trained model, the method including: acquiring training data that has been used in training of the trained model, the trained model being a model that has model applicability domains on a feature amount space and being configured to classify input data into classes; and generating, based on the acquired training data, a first detection model for a first applicability domain of the model applicability domains and a second detection model for a second applicability domain of the model applicability domains, the first detection model being the detection model having a third applicability domain narrower than the first applicability domain, the second detection model being the detection model having a fourth applicability domain narrower than the second applicability domain.

Abstract: A computer-implemented generation method of generating a detection model to be used to detect accuracy deterioration of a trained model, the generation method including: acquiring first training data that has been used in training of a trained model; acquiring second training data including a label not included in the first training data; and generating, on the basis of the acquired first training data and the acquired second training data, the detection model configured to output a prediction result based on the first training data in a case where input data belongs to within an applicability domain of the trained model, and output the label in a case where the input data belongs to outside the applicability domain of the trained model.

Abstract: A creation method that is executed by a computer, the creation method includes acquiring scores representing accuracy of classification of a machine learning model that classifies input data into classes; acquiring a difference in the scores between a first class that has a highest score and a second class that has a next highest score after the first class; and generating a first detection model that determines the classification is undecided when the difference is equal to or less than a first threshold value.

Abstract: A determination method performed by a computer, the determination method includes acquiring a first output result when data generated under second environment different from a first environment that is a training environment is input to a trained model, acquiring a second output result when the data is input to a detection model that detects decrease in a correct answer rate of a trained model when the trained model is converted into the second environment, and determining whether or not to retrain the trained model when the trained model is converted into the second environment based on the first output result and the second output result.

Abstract: A computer-implemented generation method of generating a detection model, the generation method including: acquiring training data that has been used in training of a trained model; training the detection model on the basis of the acquired training data, the detection model being configured to detect accuracy deterioration occurred in the trained model by a change in a trend of data to be processed in a data stream; determining, for the trained detection model, a state of over-training; acquiring a feature amount of the trained detection model when the determined state of over-training satisfies a given condition; and generating the detection model on the basis of the acquired feature amount.

Abstract: A generation method performed by a computer, the generation method includes classifying a plurality of pieces of teacher data into each cycle according to a condition that is preset, and generating each detection model, which corresponds to each cycle, that detects a change in an output result of a machine learning model for each cycle by using teacher data that belongs to each cycle.

Abstract: A deterioration detection method performed by a computer includes acquiring a first output result when input data is input to a trained model, acquiring a second output result when the input data is input to a detection model that detects performance deterioration of the trained model, calculating a first matching result obtained by comparing the first output result and the second output result in a first period, calculating a second matching result obtained by comparing the first output result and the second output result in a second period different from the first period, and outputting a change in accuracy deterioration of the trained model by using the first matching result and the second matching result.

Abstract: A deterioration detection method performed by a computer, the deterioration detection method includes acquiring each detection model, which corresponds to each cycle of data to be input, that detects a change in an output result of a machine learning model, acquiring a first output result when data is input to the machine learning model, acquiring each second output result when data is input to each detection model that corresponds to each cycle, and detecting a change in an output result of the machine learning model based on each of the second output results and the first output result.

Abstract: A computer-implemented detection method including: specifying, from a first image, an area that contributed to calculation of one of scores for a first class among the scores for each class obtained by inputting the first image to a deep learning model; generating a second image in which the area other than the area specified by the specifying is masked in the first image; and acquiring the scores obtained by inputting the second image to the deep learning model.

Abstract: A deterioration detection method implemented by a computer, the deterioration detection method including: acquiring a first output result when data is input to a trained model; acquiring a second output result when data is input to a detection model in which a model applicability domain of the trained model is narrowed; and detecting, on the basis of the first output result and the second output result, accuracy deterioration of the trained model due to time change in a trend of data.

Abstract: A computer-readable recording medium stores an operation control program for causing a computer to execute processing including: specifying a region of an object in a first image obtained by capturing an operating environment of a device at a first timing; generating, by using a first machine learning model, second operation information that represents an operating state of the device at a second timing after the first timing on the basis of first operation information that represents an operating state of the device at the first timing; specifying, by using a second machine learning model, a region of the device in a second image that represents the operating environment of the device on the basis of the second operation information; comparing the region of the device with the region of the object; and executing an avoidance operation of the device on the basis of a result of the processing of comparing.

Abstract: A non-transitory computer-readable recording medium stores an abnormality determination program for causing a computer to execute a process for determining an abnormality including: generating, based on first motion information of a device generated for a first timing by a machine learning model and a plurality of first feature quantities obtained by sensing the device at the first timing, estimated values of second motion information and a plurality of second feature quantities for a second timing that is after the first timing by using the machine learning model; controlling, based on the second motion information, a motion of the device at the second timing; comparing a plurality of second feature quantities obtained by sensing the device at the second timing with the generated estimated values of the plurality of second feature quantities; and determining, based on a result of the comparing, whether there is an abnormality.

Abstract: A non-transitory computer-readable recording medium having stored therein an apparatus control program.

Abstract: A non-transitory computer-readable recording medium stores an operation control program for causing a computer to execute processing including: detecting a position of an object included in an operating environment of a device; specifying an operation path of the device on the basis of an operation position of the device and the position of the object; generating first operation information on the basis of the operation path and reference information that associates position information of a plurality of points included in the operating environment with operation information that represents an operating state of the device when the plurality of points are the operation positions; and controlling the device on the basis of the first operation information.

Abstract: A learning method is performed by a computers The method includes: receiving a first image that includes an object; generating a first rectangle in the first image, the first rectangle including therein a figure, that is set in advance to have a first inclination and that represents a gripping position of an object, and having a side parallel to a first direction; inputting the first image to a model, which outputs, from the input image, a rectangle parallel to the first direction and an inclination, to cause the model to output a second rectangle and a second inclination; and updating the model such that errors of the second rectangle and the second inclination with respect to the first rectangle and the first inclination respectively decrease.

Abstract: A learning method is performed by a computer. The method includes: inputting a first image to a model, which outputs, from an input image, candidates for a specific region and confidences indicating probabilities of the respective candidates being the specific region, to cause the model to output a plurality of candidates for the specific region and confidences for the respective candidates; calculating a first value for each of candidates whose confidences do not satisfy a certain criterion among the candidates output by the model, the first value increasing as the confidence increases; calculating a second value obtained by weighting the first value such that the second value decreases as the confidence increases; and updating the model such that the second value decreases.

Abstract: A non-transitory computer-readable storage medium storing a program that causes a computer to execute a process, the process includes identifying, among combinations of any two pieces of image data included in a plurality of pieces of image data that satisfies a first condition, similarity between two pieces of image data in a combination in which one image data satisfies a second condition in addition to the first condition; identifying, based on the calculated similarity between the two pieces of image data, a score that becomes greater as the similarity increases; and performing, by using training data based on another image data in the combination and the score, machine learning.

Abstract: A non-transitory computer-readable storage medium storing a position detection program which causes a processor to perform processing for object recognition, the processing includes: acquiring a plurality of pieces of three-dimensional data of simple shapes that are not similar to each other; carrying out learning by using the plurality of acquired pieces of data; acquiring an image obtained by imaging by an imaging unit; and detecting a position of an object from the acquired image by using a first learning model generated based on the learning.

Abstract: A non-transitory computer-readable storage medium storing a position detection program which causes a processor to perform processing for object recognition, the processing includes: acquiring a plurality of pieces of three-dimensional data of simple shapes that are not similar to each other; carrying out learning by using the plurality of acquired pieces of data; acquiring an image obtained by imaging by an imaging unit; and detecting a position of an object from the acquired image by using a first learning model generated based on the learning.

Abstract: A learning method is performed by a computer. The method includes: inputting a first image to a model, which outputs, from an input image, candidates for a specific region and confidences indicating probabilities of the respective candidates being the specific region, to cause the model to output a plurality of candidates for the specific region and confidences for the respective candidates; calculating a first value for each of candidates whose confidences do not satisfy a certain criterion among the candidates output by the model, the first value increasing as the confidence increases; calculating a second value obtained by weighting the first value such that the second value decreases as the confidence increases; and updating the model such that the second value decreases.