Abstract: This application discloses a data processing method and apparatus in the artificial intelligence field, to improve prediction accuracy of a neural predictor. The neural predictor uses a small quantity of training samples. In the data processing method, a hyperparameter combination sampled from a hyperparameter search space corresponding to a user task, a plurality of samples included in a training set, and evaluation metrics of the plurality of samples are used as inputs to the neural predictor, and a prediction metric corresponding to the hyperparameter combination is determined by using the neural predictor. A hyperparameter sample and an evaluation metric of the hyperparameter sample are used to assist in predicting the hyperparameter combination sampled from the hyperparameter search space. Because the hyperparameter combination is predicted based on the evaluation metric and the hyperparameter sample that already has the evaluation metric, accuracy can be improved.

Abstract: An input of an optimized query Query feature transformation module is obtained based on an output feature of at least one previous network layer of the optimized attention layer. An input of an optimized key Key feature transformation module is obtained based on an output feature of at least one previous network layer of the optimized attention layer. An input of an optimized value Value feature transformation module is obtained based on an output feature of at least one previous network layer of the optimized attention layer. An input of at least one feature transformation module in the optimized query Query feature transformation module, the optimized key Key feature transformation module, and the optimized value Value feature transformation module is obtained based on an output feature of at least one non-adjacent previous network layer of the optimized attention layer.

Abstract: This application discloses a data processing method, and a neural network model training method and apparatus in the field of artificial intelligence. The data processing method includes: processing to-be-processed data by using a target neural network quantization model, where the target neural network quantization model includes a plurality of groups of fusion parameters, the target neural network quantization model is obtained by quantizing a target neural network model, an activation function of the target neural network model includes a piecewise linear function (PWL), the PWL includes a plurality of intervals, and there is a correspondence between the plurality of groups of fusion parameters and the plurality of intervals. According to the method in this application, a model that uses the PWL as an activation function can be quantized, thereby improving an inference speed of the model.

Abstract: A neural network model training method, a data processing method, and an apparatus are disclosed. The neural network model training method includes: training a neural network model based on training data, where an activation function of the neural network model includes at least one piecewise function, and the piecewise function includes a plurality of trainable parameters; and updating the plurality of trainable parameters of the at least one piecewise function in a training process. According to the method, the activation function suitable for the neural network model can be obtained. This can improve performance of the neural network model.

Abstract: This application discloses a model training method, which may be applied to the field of artificial intelligence. The method includes: obtaining a first neural network model; replacing a first convolutional layer in the first neural network model with a linear operation to obtain a plurality of second neural network models; and performing model training on a plurality of second neural network models, to obtain a neural network model with a highest model precision in a plurality of trained second neural network models. In this application, a convolutional layer in a to-be-trained neural network is replaced with a linear operation equivalent to a convolutional layer. A manner with highest precision is selected from a plurality of replacement manners, to improve precision of a trained model.

Abstract: This application relates to the field of artificial intelligence technologies, and describes a classification model training method, a hyperparameter search method, and an apparatus. The training method includes obtaining a target hyperparameter of a to-be-trained classification model. The target hyperparameter is used to control a gradient update operation of the to-be-trained classification model. The to-be-trained classification model includes a scaling invariance linear layer. The scaling invariance linear layer enables a predicted classification result output when a weight parameter of the to-be-trained classification model is multiplied by any scaling coefficient to remain unchanged. The method further includes updating the weight parameter of the to-be-trained classification model based on the target hyperparameter and a target training manner, to obtain a trained classification model.