Abstract: One example method includes constructing a machine learning model which, when completed, is operable to screen candidates, from a group of candidates, to define a candidate pool that has specified characteristics. The constructing includes: broadcasting, from a central node to edges of a federation, an indication that construction of a random forest, of the machine learning model, has started; performing a federated feature categorization, by the central node based on information received from the edges, of a feature to be included in respective decision trees of the edges; based on the categorizing, broadcasting a feature category to the edges; performing, by the central node using respective purity information received from the edges, a federated purity calculation; and based on the federated purity calculation, broadcasting, by the central node to the edges, a winning feature split for the feature.

Abstract: One example method includes initiating an audit of a machine learning model, providing input data to the machine learning model as part of the audit, while the audit is running, receiving information regarding operation of the machine learning model, wherein the information comprises a computational resource footprint, analyzing the computational resource footprint, and determining, based on the analyzing, that the computational resource footprint is characteristic of an adversarial attack on the machine learning model.

Abstract: One example method includes running a workload through a trained open-set classification model, recovering, as a result of the running, a class and an open-setness score corresponding to the workload, determining, based on the class and the open-setness score, whether the workload is new, and when the workload is determined to be new, starting a new cluster that includes the workload. A response time predictor model may be used to predict a response time associated with the new workload.

Abstract: One example method includes receiving a random forest classifier model that comprises a group of decision trees, wherein the random forest classifier model is created using a vertical federated framework, providing new observations, not included in a set of training observations, to a trained random forest classifier model, wherein the random forest classifier model is trained in the vertical federated framework, and wherein the training is performed using the set of training observations as input to the random forest classifier model, and generating, by the trained random forest classifier model, one or more diversity scores pertaining to the new observations.

Abstract: Training examples are created from telemetry data, in which each training example engineered features derived from the telemetry data, storage system characteristics about the storage system that processed the workload associated with the telemetry data, and the response time of the storage system while processing the workload. The training examples are provided to an unsupervised learning process which assigns the training examples to clusters. Training examples of each cluster are used to train/test a separate supervised learning process for the cluster, to cause each supervised learning process to learn a regression between independent variables (system characteristics and workload features) and a dependent variable (storage system response time). To determine a response time of a proposed storage system, the proposed workload is used to select one of the clusters, and then the trained learning process for the selected cluster is used to determine the response time of the proposed storage system.

Abstract: One example method includes accessing input data elements from logs that identify user problems with computing system components, the data elements each associated with a respective original class label that identifies a class of computing system components to which the data element relates, the respective original class labels forming a group of class labels, and a first of the original class labels is overrepresented in the group, and reducing overrepresentation of the first original class label in the group by creating an arbitrary aggregation of some of the class labels that includes the first original class label. The method includes creating, based on a hierarchical modeling structure, prepared data in which an original class label is replaced by the aggregation. Next a hierarchical model and benchmark model are trained, and each model generates respective predictions for comparison. An inferencing process is performed to determine which predicted label will be used.

Abstract: One example method includes receiving, at a decision tree trained with a group of training observations, a group of new observations, traversing the decision tree with the new observations, calculating, for one or more nodes of the decision tree, a respective local diversity score, and aggregating the local diversity scores to create an aggregate diversity score, and the aggregate diversity score indicates an extent to which one or more of the new observations are similar, in one or more respects, to the group of training observations.

Abstract: An outlier detection mechanism is disclosed that improves transparency and explainability in machine learning models. The outlier detection mechanism can quantify, at prediction time, how a new observation differs from training observations. The outlier detection mechanism can also provide a way to aggregate outputs from decision trees by weighting the outputs of the decision trees based on their explainability.

Abstract: A global prediction manager for generating predictions using data from data zones includes storage for storing a model repository comprising a global model set and a prediction manager. The prediction manager obtains a local model set from a data zone of the data zones indicating that the global model set is unacceptable; makes a determination that the local model set is acceptable; in response to the determination: distributes the local model set to at least one second data zone of the data zones; obtains compressed telemetry data, that was compressed using the local model set, from the data zone and the at least one second data zone; and generates a global prediction regarding a future operating condition of the data zones using: the compressed local telemetry data and the local model set.

Abstract: Techniques are provided for allocation of shared computing resources using source code feature extraction and machine learning techniques. An exemplary method comprises obtaining source code for execution in a shared computing environment; extracting a plurality of discriminative features from the source code; obtaining a trained machine learning model; and generating a prediction of an allocation of one or more resources of the shared computing environment needed to satisfy one or more service level agreement requirements for the source code. The generated prediction is optionally adjusted using a statistical analysis of an error curve, based on one or more error boundaries obtained by the trained machine learning model.

Abstract: Techniques described herein relate a method for explainability for Random Forest (RF) classifiers. The method may include generating a plurality of class labels for a target variable; training a RF classifier using the plurality of class labels and a historical dataset to obtain a trained RF classifier; building a transaction database using the trained RF classifier; identifying a plurality of class association rules using the transaction database; identifying a portion of the plurality of class association rules that have minimum confidence values greater than a minimum confidence value threshold; and presenting the portion of the plurality of class association rules to an interested entity as explainability results.

Abstract: An autoregressor that compresses input data for a specific purpose. Input data is compressed using a compression/decompression framework and by accounting for a purpose of a prediction model. The compression aspect of the framework is distributed and the decompression aspect of the framework may be centralized. The compression/decompression framework and a machine learning prediction model can be centrally trained. The compressor is distributed to nodes such that the input data can be compressed and transmitted to a central node. The model and the compression/decompression framework are continually trained on new data. This allows for lossy compression and higher compression rates while maintaining low prediction error rates.

Abstract: A prediction manager for providing responsiveness predictions for deployments includes persistent storage and a predictor. The persistent storage stores training data and conditioned training data. The predictor is programmed to obtain training data based on: a configuration of at least one deployment of the deployments, and a measured responsiveness of the at least one deployment, perform a peak extraction analysis on the measured responsiveness to obtain conditioned training data, obtain a prediction model using: the training data, and a first untrained prediction model, obtain a confidence prediction model using: the conditioned training data, and a second untrained prediction model, obtain a combined prediction using: the prediction model, and the confidence prediction model, and perform, based on the combined prediction, an action set to prevent a responsiveness failure.

Abstract: Techniques are provided for provenance-based software script reuse. One method comprises extracting provenance data from source code including, for example, source code fragments, wherein the extracted provenance data indicates a control flow and a data flow of the source code; encapsulating source code fragments from the source code that satisfy one or more similarity criteria as a reusable source code fragment; and providing a repository of encapsulated reusable source code fragments for reuse during a development of new software scripts. The repository of encapsulated reusable source code fragments optionally comprises a searchable database further including, for example, the provenance data, data annotations, input parameters and generated results for the corresponding source code fragment.

Abstract: A deployment manager includes storage for storing a prediction model based on telemetry data from the deployments and a prediction manager. The prediction manager generates, using the prediction model and second telemetry data obtained from a deployment of the deployments: a prediction, and a prediction error estimate; in response to a determination that the prediction indicates a negative impact on the deployment: generates a confidence estimation for the prediction based on a variability of the second telemetry data from the telemetry data; in response to a second determination that the confidence estimation indicates that the prediction error estimate is inaccurate: remediates the prediction based on the variability to obtain an updated prediction; and performs an action set, based on the updated prediction, to reduce an impact of the negative impact on the deployment.

Abstract: Techniques are provided for allocating shared computing resources using source code feature extraction and cluster-based training of machine learning models. An exemplary method comprises: obtaining a source code corpus with source code segments for execution in a shared computing environment; extracting discriminative features from the source code segments in the source code corpus; obtaining a trained machine learning model, wherein the trained machine learning model is trained using samples of source code segments from clusters derived from clustering the source code corpus based on (i) a term frequency metric, and/or (ii) observed values of execution metrics; and generating, using the trained machine learning model, a prediction of an allocation of resources of the shared computing environment needed to satisfy service level agreement requirements for source code to be executed in the shared computing environment.

Abstract: A system and method for implementing design cycles for developing a hardware component including receiving sets of experimental data, each of set experimental data resulting from an application of a set of variables to the hardware component during a common or a different design cycle of the hardware component, where each variable represents an aspect of the hardware component, determining discretized classes of the experimental data based on one or more quality metrics, and obtaining statistical measurements of the variables to determine correlations between the discretized classes of the quality metrics and the statistical measurements of variables for determining a pattern of the quality metrics to reduce the number of design cycles implemented on the hardware component during the developing of the hardware component.

Abstract: One or more aspects of the present disclosure relate to providing storage system configuration recommendations. System configurations of one or more storage devices can be determined based on their respective collected telemetry information. Performance of storage devices having different system configurations can be predicted based on one or more of: the collected telemetry information and each of the different system configurations. In response to receiving one or more requested performance characteristics and workload conditions, one or more recommended storage device configurations can be provided for each request based on the predicted performance characteristics, the requested performance characteristics, and the workload conditions.

Abstract: Decoding random forest problem solving through node labeling and subtree distributions. Random forests, like any other type of machine learning algorithm, are designed and configured to solve classification, regression, and/or prediction problems. Solutions (or outputs) provided by random forests, given inputs in the form of values for a set of features, may sometimes be inaccurate, unexpected, or undesirable. Understanding or decoding how a random forest solves a given problem may be a way to correct or improve the random forest. The disclosed method, accordingly, proposes decoding random forest problem solving through the identification of subtrees (by way of node labeling) amongst a random forest, as well as the frequencies that these subtrees appear (or distributions thereof) throughout the random forest.

Abstract: A distribution of response times of a storage system can be estimated for a proposed workload using a trained learning process. Collections of information about operational characteristics of multiple storage systems are obtained, in which each collection includes parameters describing the configuration of the storage system that was used to create the collection, workload characteristics describing features of the workload that the storage system processed, and storage system response times. For each collection, workload characteristics are aggregated, and the storage system response information is used to train a probabilistic mixture model. The aggregated workload information, storage system characteristics, and probabilistic mixture model parameters of the collections form training examples that are used to train the learning process.