Abstract: A system monitors execution of a machine learning model in an environment, for example, development environment or production environment. The system receives a training dataset and a production dataset. The system initializes a review dataset based on elements of the training dataset. The system samples a subset of elements of the production dataset by identifying elements from the production dataset based on their distance from elements of the review dataset. The system sends elements of the review dataset for presentation via a user interface for receiving user feedback indicating accuracy of the result of execution of the machine learning model. The execution of the machine learning model is monitored to make determination regarding deployment of the model in a production environment for continuous delivery of the model or for evaluation or quality assurance of model executing in an environment.

Abstract: A model management system adaptively refines a training dataset for more effective visual inspection. The system trains a machine learning model using the initial training dataset and sends the trained model to a client for deployment. The deployment process generates outputs that are sent back to the system. The system determines that performance of predictions for noisy data points are inadequate and determines a cause of failure based on a mapping of the noisy data point to a distribution generated for the training dataset across multiple dimensions. The system determines a cause of failure based on an attribute of the noisy datapoint that deviates from the distribution of the training dataset and performs refinement towards the training dataset based on the identified cause of failure. The system retrains the machine learning model with the refined training dataset and sends the retrained machine learning model back to the client for re-deployment.

Abstract: In one embodiment, a cloud client device identifies a configuration event. The cloud client device identifies a configuration associated with the configuration event. The cloud client device stores a first security key associated with the configuration and configures the cloud client device in accordance with the configuration.

Abstract: In one embodiment, a cloud client device identifies a configuration event. The cloud client device identifies a configuration associated with the configuration event. The cloud client device stores a first security key associated with the configuration and configures the cloud client device in accordance with the configuration.