Abstract: In response to registration requests, metadata of a plurality of machine learning artifacts are stored. In response to a pipeline definition request, which does not specify resources to be used for the pipeline, a representation of a pipeline comprising nodes corresponding to registered artifacts is stored. A benchmarking operation is conducted to select the types of resources to be used for the nodes, and results of the benchmarking are provided.

Abstract: Computer systems and associated methods are disclosed to implement a model development environment (MDE) that allows a team of users to perform iterative model experiments to develop machine learning (ML) media models. In embodiments, the MDE implements a media data management interface that allows users to annotate and manage training data for models. In embodiments, the MDE implements a model experimentation interface that allows users to configure and run model experiments, which include a training run and a test run of a model. In embodiments, the MDE implements a model diagnosis interface that displays the model's performance metrics and allows users to visually inspect media samples that were used during the model experiment to determine corrective actions to improve model performance for later iterations of experiments. In embodiments, the MDE allows different types of users to collaborate on a series of model experiments to build an optimal media model.