Abstract: An online system identifies an additional feature to evaluate for inclusion in a machine learned model. The additional feature is based on characteristics of one or more dimensions of information maintained by the online system. To generate data for evaluating the additional feature, the online system generates various partitions of stored data, where each partition includes characteristics associated with one or more dimensions on which the additional feature is based. Using values of characteristics in a partition, the online system generates values for the additional feature and includes the values of the additional feature in the partition. Values for the additional feature are generated for various partitions based on the values of characteristics in each partition. The online system combines multiple partitions that include values for the additional feature to generate a training set for evaluating a machine learned model including the additional feature.

Abstract: An online system identifies an additional feature to evaluate for inclusion in a machine learned model. The additional feature is based on characteristics of one or more dimensions of information maintained by the online system. To generate data for evaluating the additional feature, the online system generates various partitions of stored data, where each partition includes characteristics associated with one or more dimensions on which the additional feature is based. Using values of characteristics in a partition, the online system generates values for the additional feature and includes the values of the additional feature in the partition. Values for the additional feature are generated for various partitions based on the values of characteristics in each partition. The online system combines multiple partitions that include values for the additional feature to generate a training set for evaluating a machine learned model including the additional feature.

Abstract: Some embodiments include a workflow authoring tool that accesses a text string representation of a workflow and a text string representation of at least a data processing operator type. The workflow authoring tool enables definition of one or more data processing operator types that can be referenced in defining the machine learning workflow. When scheduling a workflow, the text string representation of the workflow can be parsed and traversed to generate an interdependency graph of one or more data processing operators. The text string representation of the data processing operator type can identify operator attributes associated with the data processing operator type.

Abstract: Some embodiments include an experiment management interface for a machine learning system. The experiment management interface can manage one or more workflow runs related to building or testing machine learning models. The experiment management interface can receive an experiment initialization command to create a new experiment associated with a new workflow. A workflow can be represented by an interdependency graph of one or more data processing operators. The experiment management interface enables definition of the new workflow from scratch or by cloning and modifying an existing workflow. The workflow can define a summary format for its inputs and outputs. In some embodiments, the experiment management interface can automatically generate a comparative visualization at the conclusion of running the new workflow based on an input schema or an output schema of the new workflow.

Abstract: Some embodiments include a method of machine learner workflow processing. For example, a workflow execution engine can receive an interdependency graph of operator instances for a workflow run. The operator instances can be associated with one or more operator types. The workflow execution engine can assign one or more computing environments from a candidate pool to execute the operator instances based on the interdependency graph. The workflow execution engine can generate a schedule plan of one or more execution requests associated with the operator instances. The workflow execution engine can distribute code packages associated the operator instances to the assigned computing environments. The workflow execution engine can maintain a memoization repository to cache one or more outputs of the operator instances upon completion of the execution requests.

Abstract: An online system simplifies modification of features used by machine learned models used by the online system, such as machined learned models with high dimensionality. The online system obtains a superset of features including features used by at least one machine learned model and may include additional features. From the superset of features, the online system generates various groups of features for a machine learned model. The groups of features may be a group including features currently used by the machine learned model, a group including all available features, and one or more intermediate groups. Intermediate groups include various numbers of features from the set selected based on measures of feature impact on the machine learned model associated with various features. A user may select a group of features, test the machine learning model using the selected group, and then launch the tested model based on the results.

Abstract: Some embodiments include a machine learner platform. The machine learner platform can implement a model tracking service to track one or more machine learning models for one or more application services. A model tracker database can record a version history and/or training configurations of the machine learning models. The machine learner platform can implement a platform interface configured to present interactive controls for building, modifying, evaluating, deploying, or compare the machine learning models. A model trainer engine can task out a model training task to one or more computing devices. A model evaluation engine can compute an evaluative metric for a resulting model from the model training task.

Abstract: Some embodiments include an experiment management interface for a machine learning system. The experiment management interface can manage one or more workflow runs related to building or testing machine learning models. The experiment management interface can receive an experiment initialization command to create a new experiment associated with a new workflow. A workflow can be represented by an interdependency graph of one or more data processing operators. The experiment management interface enables definition of the new workflow from scratch or by cloning and modifying an existing workflow. The workflow can define a summary format for its inputs and outputs. In some embodiments, the experiment management interface can automatically generate a comparative visualization at the conclusion of running the new workflow based on an input schema or an output schema of the new workflow.

Abstract: Some embodiments include a method of machine learner workflow processing. For example, a workflow execution engine can receive an interdependency graph of operator instances for a workflow run. The operator instances can be associated with one or more operator types. The workflow execution engine can assign one or more computing environments from a candidate pool to execute the operator instances based on the interdependency graph. The workflow execution engine can generate a schedule plan of one or more execution requests associated with the operator instances. The workflow execution engine can distribute code packages associated the operator instances to the assigned computing environments. The workflow execution engine can maintain a memoization repository to cache one or more outputs of the operator instances upon completion of the execution requests.

Abstract: Some embodiments include a workflow authoring tool that accesses a text string representation of a workflow and a text string representation of at least a data processing operator type. The workflow authoring tool enables definition of one or more data processing operator types that can be referenced in defining the machine learning workflow. When scheduling a workflow, the text string representation of the workflow can be parsed and traversed to generate an interdependency graph of one or more data processing operators. The text string representation of the data processing operator type can identify operator attributes associated with the data processing operator type.

Abstract: Some embodiments include a machine learner platform. The machine learner platform can implement a model tracking service to track one or more machine learning models for one or more application services. A model tracker database can record a version history and/or training configurations of the machine learning models. The machine learner platform can implement a platform interface configured to present interactive controls for building, modifying, evaluating, deploying, or compare the machine learning models. A model trainer engine can task out a model training task to one or more computing devices. A model evaluation engine can compute an evaluative metric for a resulting model from the model training task.

Abstract: An online system identifies an additional feature to evaluate for inclusion in a machine learned model. The additional feature is based on characteristics of one or more dimensions of information maintained by the online system. To generate data for evaluating the additional feature, the online system generates various partitions of stored data, where each partition includes characteristics associated with one or more dimensions on which the additional feature is based. Using values of characteristics in a partition, the online system generates values for the additional feature and includes the values of the additional feature in the partition. Values for the additional feature are generated for various partitions based on the values of characteristics in each partition. The online system combines multiple partitions that include values for the additional feature to generate a training set for evaluating a machine learned model including the additional feature.

Abstract: An online system simplifies modification of features used by machine learned models used by the online system, such as machined learned models with high dimensionality. The online system obtains a superset of features including features used by at least one machine learned model and may include additional features. From the superset of features, the online system generates various groups of features for a machine learned model. The groups of features may be a group including features currently used by the machine learned model, a group including all available features, and one or more intermediate groups. Intermediate groups include various numbers of features from the set selected based on measures of feature impact on the machine learned model associated with various features. A user may select a group of features, test the machine learning model using the selected group, and then launch the tested model based on the results.