Abstract: A system includes a device management infrastructure arranged to process a set of training data using a data transformation model to generate first characteristic data indicative of values of one or more characteristics for the training data, and an electronic device communicatively coupled to the device management infrastructure. The device includes memory circuitry arranged to store a machine learning model trained using the set of training data, and a copy of the data transformation model. The device is arranged to process a set of input data using the data transformation model to generate second characteristic data indicative of values of said set of data characteristics for the input data. The device and/or the device management infrastructure is arranged to determine whether the first characteristic data and the second characteristic data satisfy one or more consistency criteria indicative of consistency between the training data and the input data.

Abstract: A method performed by an electronic device for testing machine learning models is described. The electronic device includes a program for executing a first machine learning model and a second machine learning model. The electronic device receives a machine learning model update data package and generates the second machine learning model from the update data package by partially or fully updating the first machine learning model. When the program is executed, both the first and second machine learning model use a common set of input data to perform inference. The program collects outputs from the first machine learning model and the second machine learning model, for further analysis.