Abstract: A system monitors execution of a machine learning model in an environment, for example, development environment or production environment. The system receives a training dataset and a production dataset. The system initializes a review dataset based on elements of the training dataset. The system samples a subset of elements of the production dataset by identifying elements from the production dataset based on their distance from elements of the review dataset. The system sends elements of the review dataset for presentation via a user interface for receiving user feedback indicating accuracy of the result of execution of the machine learning model. The execution of the machine learning model is monitored to make determination regarding deployment of the model in a production environment for continuous delivery of the model or for evaluation or quality assurance of model executing in an environment.

Abstract: A model management system performs error analysis on results predicted by a machine learning model. The model management system identifies an incorrectly classified image outputted from a machine learning model and identifies using the Neural Template Matching (NTM) algorithm, an additional image correlated to the selected image. The system outputs correlated images based on a given image and a selection by a user through a user interface of a region of interest (ROI) of the given image. The region is defined by a bounding polygon input and the correlated images include features correlated to the features within the ROI. The system prompts a task associated with the additional image. The system receives a response that includes an indication that the additional image is incorrectly labeled and including a replacement label and instruct that the machine learning model be retrained using an updated training dataset that includes the replacement label.

Abstract: A model management system adaptively refines a training dataset for more effective visual inspection. The system trains a machine learning model using the initial training dataset and sends the trained model to a client for deployment. The deployment process generates outputs that are sent back to the system. The system determines that performance of predictions for noisy data points are inadequate and determines a cause of failure based on a mapping of the noisy data point to a distribution generated for the training dataset across multiple dimensions. The system determines a cause of failure based on an attribute of the noisy datapoint that deviates from the distribution of the training dataset and performs refinement towards the training dataset based on the identified cause of failure. The system retrains the machine learning model with the refined training dataset and sends the retrained machine learning model back to the client for re-deployment.

Abstract: A processor receives an image of a syringe. After identifying a background and foreground of the image, where the foreground indicates pixels that may be associated with a defect, the processor subtracts the background to generate an updated image with an accentuated foreground. The processor applies a bounding box to a group of pixels in the foreground and inputs the bounding box into a classifier. The classifier outputs a label indicating whether the syringe is defective.

Abstract: A processor receives an image of a syringe. After identifying a background and foreground of the image, where the foreground indicates pixels that may be associated with a defect, the processor subtracts the background to generate an updated image with an accentuated foreground. The processor applies a bounding box to a group of pixels in the foreground and inputs the bounding box into a classifier. The classifier outputs a label indicating whether the syringe is defective.