Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a feature configuration for a feature. Next, the system obtains, from the feature configuration, an anchor containing metadata for accessing the feature in an environment. The system then uses one or more attributes of the anchor to retrieve one or more feature values of the feature from the environment. Finally, the system provides the one or more feature values for use with one or more machine-learning models.

Abstract: One embodiment of the present invention sets forth a technique for analyzing a transcript of a structured interaction. The technique includes determining a first portion of the transcript that corresponds to a first dialogue act. The technique also includes matching the first portion of the transcript to a first component of a script for the structured interaction based on a first set of embeddings for the first portion of the transcript and a second set of embeddings associated with the first component of the script. The technique further includes causing a first mapping between the first portion of the transcript and the first component to be outputted.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a function call for a function that calculates an attribute associated with a machine learning model. For each argument of the function call, the system identifies a parameter type of the argument, wherein the parameter type represents a type of data used with the machine learning model. The system also obtains a value accessor for retrieving features specific to the parameter type and obtains a value represented by the argument using the value accessor. The system then calculates the attribute by applying the function to the value and uses the attribute to execute the machine learning model.

Abstract: The disclosed embodiments provide a system for performing online hyperparameter tuning in distributed machine learning. During operation, the system uses input data for a first set of versions of a statistical model for a set of entities to calculate a batch of performance metrics for the first set of versions. Next, the system applies an optimization technique to the batch to produce updates to a set of hyperparameters for the statistical model. The system then uses the updates to modulate the execution of a second set of versions of the statistical model for the set of entities. When a new entity is added to the set of entities, the system updates the set of hyperparameters with a new dimension for the new entity.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a hierarchical representation containing a set of namespaces of a set of features shared by a set of statistical models. Next, the system uses the hierarchical representation to obtain, from one or more execution environments, a subset of the features for use in calculating the derived feature. The system then applies a formula from the hierarchical representation to the subset of the features to produce the derived feature. Finally, the system provides the derived feature for use by one or more of the statistical models.

Abstract: The disclosed embodiments provide a method and system for performing regularized model adaptation for in-session recommendations. During operation, the system obtains, from a server, a first global version of a statistical model. During a first user session with a user, the system improves a performance of the statistical model by using the first global version to output one or more recommendations to the user and using the first global version and user feedback from the user to create a first personalized version of the statistical model. At an end of the first user session, the system transmits an update containing a difference between the first personalized version and the first global version to the server for use in producing a second global version of the statistical model by the server.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a feature configuration for a feature. Next, the system obtains, from the feature configuration, an anchor containing metadata for accessing the feature in an environment. The system then uses one or more attributes of the anchor to retrieve one or more feature values of the feature from the environment. Finally, the system provides the one or more feature values for use with one or more machine-learning models.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a function call for a function that calculates an attribute associated with a machine learning model. For each argument of the function call, the system identifies a parameter type of the argument, wherein the parameter type represents a type of data used with the machine learning model. The system also obtains a value accessor for retrieving features specific to the parameter type and obtains a value represented by the argument using the value accessor. The system then calculates the attribute by applying the function to the value and uses the attribute to execute the machine learning model.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains feature configurations for a set of features. Next, the system obtains, from the feature configurations, an anchor containing metadata for accessing a first feature in an environment and a feature derivation for generating a second feature from the first feature. The system then uses the anchor to retrieve feature values of the first feature from the environment and uses the feature derivation to generate additional feature values of the second feature from the feature values of the first feature. Finally, the system provides the additional feature values for use with one or more machine learning models.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a feature dependency graph of features for a machine learning model and an operator dependency graph comprising operators to be applied to the features. Next, the system generates feature values of the features according to an evaluation order associated with the operator dependency graph and feature dependencies from the feature dependency graph. During evaluation of an operator in the evaluation order, the system updates a list of calculated features with one or more features that have been calculated for use with the operator. During evaluation of a subsequent operator in the evaluation order, the system uses the list of calculated features to omit recalculation of the feature(s) for use with the subsequent operator.

Abstract: A system and method for managing asynchronously receiving updates and merging updates into global versions of a statistical model using version control are disclosed. During operation, the system transmits a first global version of a statistical model to a set of client computer systems. Next, the system obtains, from a first subset of the client computer systems, a first set of updates to the first global version. The system then merges the first set of updates into a second global version of the statistical model. Finally, the system transmits the second global version to the client computer systems asynchronously from receiving a second set of updates to the first and/or second global versions from a second subset of the client computer systems.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a model definition and a training configuration for a machine-learning model, wherein the training configuration includes a set of required features, a training technique, and a scoring function. Next, the system uses the model definition and the training configuration to load the machine-learning model and the set of required features into a training pipeline without requiring a user to manually identify the set of required features. The system then uses the training pipeline and the training configuration to update a set of parameters for the machine-learning model. Finally, the system stores mappings containing the updated set of parameters and the set of required features in a representation of the machine-learning model.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a set of values and a set of reference values for one or more features used with one or more statistical models. Next, the system applies a hypothesis test to the set of values and the set of reference values to assess a distribution-level consistency in the one or more features. The system then outputs the distribution-level consistency for use in monitoring the distribution of the one or more features. Finally, the system includes, with the outputted distribution-level consistency, one or more factors that contribute to the distribution-level consistency.

Abstract: The disclosed embodiments provide a system for performing online hyperparameter tuning in distributed machine learning. During operation, the system uses input data for a first set of versions of a statistical model for a set of entities to calculate a batch of performance metrics for the first set of versions. Next, the system applies an optimization technique to the batch to produce updates to a set of hyperparameters for the statistical model. The system then uses the updates to modulate the execution of a second set of versions of the statistical model for the set of entities. When a new entity is added to the set of entities, the system updates the set of hyperparameters with a new dimension for the new entity.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system generates, from a set of traces of an asynchronous workflow, a graph-based representation of the asynchronous workflow. Next, the system uses a set of causal relationships in the asynchronous workflow to update the graph-based representation. The system then analyzes the updated graph-based representation to identify a set of high-latency paths in the asynchronous workflow. Finally, the system uses the set of high-latency paths to output an execution profile for the asynchronous workflow, wherein the execution profile includes a subset of tasks associated with the high-latency paths in the asynchronous workflow.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a set of latencies for a set of requests in a multi-phase parallel task. Next, the system includes the latencies in a graph-based representation of the multi-phase parallel task. The system then analyzes the graph-based representation to identify a set of high-latency paths in the multi-phase parallel task. Finally, the system uses the set of high-latency paths to output an execution profile for the multi-phase parallel task, wherein the execution profile includes a subset of the requests associated with the high-latency paths.

Abstract: In order to address annotation bias in batch annotations, obtained via crowdsourcing, on a set of comments on user posts in a social network, a system determines an annotation probability distribution based on a factor-graph model of the batch annotations. In particular, during operation the system computes the factor-graph model that represents relationships between feature vectors that represent the comments and the annotations for the comments. Note that, for a given batch of k comments, the factor-graph model may include a statistically dependent combination of statistically independent models of the interrelationships between the feature vectors and the annotations for the k comments. Then, the system calculates the annotation probability distribution based on model parameters associated with the factor-graph model, a mapping function that maps from the feature vectors to the annotations, and an indicator function that represents the annotations for the comments in the batches.

Abstract: In order to leverage annotation bias in batch annotations, obtained via crowdsourcing, on a set of comments on user posts in a social network, a system may select a subset of the comments for annotation based on how informative expected annotations for the comments in the subset are for the one or more classifiers and probabilities of occurrence of the expected annotations based on a predetermined annotation probability distribution. Note that the classifier may predict how likely the expected annotations are accurate for the comments in a given subset. Moreover, the predetermined annotation probability distribution may specify the annotation bias. In this way, the system may use the annotation bias to select the subset that is likely to receive expected annotations and, thus, are that are easier to use in training the classifier.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a hierarchical representation containing a set of namespaces of a set of features shared by a set of statistical models. Next, the system uses the hierarchical representation to obtain, from one or more execution environments, a subset of the features for use in calculating the derived feature. The system then applies a formula from the hierarchical representation to the subset of the features to produce the derived feature. Finally, the system provides the derived feature for use by one or more of the statistical models.

Abstract: The disclosed embodiments provide a system and method for performing version control for asynchronous distributed machine learning. During operation, the system transmits a first global version of a statistical model to a set of client computer systems. Next, the system obtains, from a first subset of the client computer systems, a first set of updates to the first global version. The system then merges the first set of updates into a second global version of the statistical model. Finally, the system transmits the second global version to the client computer systems asynchronously from receiving a second set of updates to the first and/or second global versions from a second subset of the client computer systems.