Abstract: A system receives identification of a machine learning model, stored by a vehicle, that is ready for training. The system determines, based on a configuration file associated with the model, whether batch-based or live training is to be used to train the model and collects data for training, defined by a configuration file associated with the model. The system calls a learning as a service vehicle process to load a kernel and configures the kernel based on configuration data defined in the configuration file. The system receives notification from the learning as a service process that the training is complete. Additionally, the system validates a model, responsive to the notification including indication that training was successful, using a vehicle model validation process to test the model with live data before deployment by background execution of the model and saves a copy of the model for deployment responsive to successful validation.

Abstract: A vehicle system receives indication of a newly trained machine learning model designated for validation. The system load a copy of the model into shadow execution hardware, capable of background execution of the model and subscribes to one or more data topics to which input data for the model, gathered by a vehicle data gathering process, is published. The system executes the model in the background as the vehicle travels, using data published to the data topics and benchmarks output from the model to determine whether the model outperforms a prior version of the model, that represents the model prior to the model being newly trained, based on relative performance of both models compared to performance expectations defined in a configuration file stored by the vehicle. Also, the system, responsive to the model outperforming the prior version of the model based on the performance expectations defined by the configuration file, validates the model as suitable for deployment.

Abstract: A system receives identification of data to be gathered, via a request configured based on a configuration file associated with a machine learning model stored by a vehicle. The system receives identification of how the data is to be labeled, defined by the configuration file and create one or more topics for publication of the data, onboard the vehicle, the publication including both the gathered data and any meta-data usable to label the data in accordance with the definitions in the configuration file. Also, the system subscribes to the topics to receive the published data and the metadata and appends labels to the data, using the metadata, to label the data in accordance with the definitions for labeling in the configuration file. The system saves the labeled data in vehicle memory as data associated with the model.

Abstract: A system subscribes to one or more inference topics to which inferences are published on behalf of trainable software models executing in a vehicle computing environment. The system receives inferences from the topics as the inferences are published to the topics and associates the inferences with one or more trainable software models to be monitored. Also, the system identifies instances of unexpected output based on comparison of received inferences, associated with a given model to be monitored, to expected inference values identified in a configuration file, stored in a vehicle memory and associated with the given model, and, responsive to identifying the unexpected output, devises a modification strategy for the model based on characteristics of the unexpected output.

Abstract: A system receives a plurality of data sets relating to differently trained versions of a global machine learning model, from a plurality of vehicles, the data sets including at least a present local loss value experienced by a current version of the global model executing on a given vehicle for which a data set of the plurality of data sets was received. The system determines a loss reduction for each received data set, representing a loss reduction since a previous local loss value included in a previous received data set corresponding to the given vehicle. The system determines whether the loss reduction for each received data set of the plurality of data sets exceeds a predefined threshold cutoff value and trains the global model using federated learning and based on the data sets of the plurality of data sets for which the loss reduction exceeds the predefined cutoff value.