Abstract: Disclosed embodiments provide techniques for estimating imputation algorithm performance. Multiple imputer algorithms are selected, and an evaluation of how well each of the imputer algorithms can estimate the missing data is performed. Disclosed embodiments obtain an imputer candidate dataset (ICD). The imputer candidate dataset is compared to the incomplete data range, and a similarity metric is determined between the data range and the ICD. When the similarity metric exceeds a predetermined threshold, an imputer evaluation dataset (IED) is created from the ICD by removing one or more data points from the ICD. Each imputer algorithm is evaluated by applying the IED to it, and computing an imputer evaluation metric based on its performance. The multiple imputer algorithms are ranked based on the imputer evaluation metric. The best ranked imputer algorithm can then be selected for use on the incomplete data range within the measurement dataset.

Abstract: Automated techniques are disclosed for minimizing communication between nodes in a system comprising multiple nodes for executing requests in which a request type is associated with a particular node. For example, a technique comprises the following steps. Information is maintained about frequencies of compound requests received and individual requests comprising the compound requests. For a plurality of request types which frequently occur in a compound request, the plurality of request types is associated to a same node. As another example, a technique for minimizing communication between nodes, in a system comprising multiple nodes for executing a plurality of applications, comprises the steps of maintaining information about an amount of communication between said applications, and using said information to place said applications on said nodes to minimize communication among said nodes.

Abstract: An information processing system, computer readable storage medium, and a method for managing computer memory. The method includes deploying at least one memory allocator on a computing system which includes a multi-user computing system or a cloud computing system; dynamically allocating, by the at least one memory allocator, an amount of dynamically allocated memory for use by an executing program at runtime; and determining a cost to charge a user based on the amount of memory that has been allocated. Multiple billing methods can be used including: restricting the amount of dynamically allocated memory available to the user based on how much the user is paying; charging the user a fee which grows with an amount of dynamically allocated memory consumed; charging the user higher fees during higher periods of dynamically allocated memory usage; and charging the user a low fee for maintaining dynamically allocated memory usage below a threshold.

Abstract: Methods and systems to provide a form of probabilistic labeling to associate an outage with a disturbance, which could itself be either known based on the available data or unknown. In the latter case, labeling is especially challenging, as it necessitates the discovery of the disturbance. One approach incorporates a statistical change-point analysis to time-series events that correspond to service tickets in the relevant geographic sub-regions. The method is calibrated to separate the regular periods from the environmental disturbance periods, under the assumption that disturbances significantly increase the rate of loss-causing events. To obtain the probability that a given loss-causing event is related to an environmental disturbance, the method leverages the difference between the rate of events expected in the absence of any disturbances (baseline) and the rate of actually observed events. In the analysis, the local disturbances are identified and estimators of their duration and magnitude are provided.

Abstract: Improved techniques are disclosed for processing transactions or requests in a data processing system. For example, a method for processing requests in a system including a plurality of nodes includes the following steps. At least two nodes of the plurality of nodes receive a plurality of requests. The two nodes exchange information to determine an order for processing requests. The two nodes process the requests in accordance with the order. The order may include a total order or a partial order.

Abstract: Embodiments of the invention are directed a computer-implemented method for efficiently assessing data quality metrics. A non-limiting example of the computer-implemented method includes receiving, using a processor, a plurality of updates to data points in a data stream. The processor is further used to provide a plurality of data quality metrics (DQMs), and to maintain information on how much the plurality of DQMs are changing over time. The processor also maintains information on computational overhead for the plurality of DQMs, and also updates data quality information based on the maintained information.

Abstract: Embodiments of the invention are directed a computer-implemented method for efficiently assessing data quality metrics. A non-limiting example of the computer-implemented method includes receiving, using a processor, a plurality of updates to data points in a data stream. The processor is further used to provide a plurality of data quality metrics (DQMs), and to maintain information on how much the plurality of DQMs are changing over time. The processor also maintains information on computational overhead for the plurality of DQMs, and also updates data quality information based on the maintained information.

Abstract: Techniques for generating model ensembles are provided. A plurality of models trained to generate predictions at each of a plurality of intervals is received. A respective prediction accuracy of each respective model of the plurality of models is determined for a first interval of the plurality of intervals by processing labeled evaluation data using the respective model. Additionally, a model ensemble specifying one or more of the plurality of models for each of the plurality of intervals is generated, comprising selecting, for the first interval, a first model of the plurality of models based on (i) the respective prediction accuracies and (ii) at least one non-error metric.

Abstract: Techniques for model evaluation and selection are provided. A plurality of models trained to generate predictions at each of a plurality of intervals is received, and a plurality of model ensembles, each specifying one or more of the plurality of models for each of the plurality of intervals, is generated. A test data set is received, where the test data set includes values for at least a first interval of the plurality of intervals and does not include values for at least a second interval of the plurality of intervals. A first model ensemble, of the plurality of model ensembles, is selected based on processing the test data set using each of the plurality of model ensembles.

Abstract: Embodiments of the invention are directed a computer-implemented method for assessing data quality. A non-limiting example of the computer-implemented method includes using a processor to receive a plurality of updates to data points in a data stream. The processor is further used to compute instances of a data quality metric (DQM) from the data points in the data stream. The instances of the DQM are configured to differentiate the data points in the data stream by time and assign a higher weight to the instances of the DQM computed from more recent data points in the data stream. The instances of the DQM are continuously updated as more of the data points are received by the processor while limiting cycles of the processor consumed by updating the instances of the DQM to a threshold.

Abstract: A processing system, a computer program product, and a method for determining a best imputation algorithm from a plurality of imputation algorithms A method includes: providing a plurality of imputation algorithms; defining a data analytics task in which at least one step of the data analytics task includes determining at least one missing data value by imputation; executing the data analytics task multiple times wherein each execution of the data analytics task uses a data imputation algorithm of the plurality of data imputation algorithms to determine at least one missing data value; determining an error for each execution of the data analytics task; and selecting an imputation algorithm which results in a least error for the data analytics task.

Abstract: A processing system, a computer program product, and a method for efficiently determining a best imputation algorithm from a plurality of imputation algorithms A method includes: providing a plurality of imputation algorithms; providing a time parameter tmax to limit an amount of time spent determining a best imputation algorithm; maintaining past information i on accuracy and execution time for at least one of the imputation algorithms; using said information i to compute a utility score for each of the at least one the imputation algorithms; and testing imputation algorithms and associated parameters in an order based on said utility scores.

Abstract: A system, computer program product, and method are provided to de-duplicate one or more memory pages in parallel. Multiple de-duplication processes operate in parallel, with each de-duplication process operatively coupled to at least two data structures, and further leveraging transactional memory to mitigate access conflicts.

Abstract: An information processing system, computer readable storage medium, and a method for managing computer memory. The method includes deploying at least one memory allocator on a computing system which includes a multi-user computing system or a cloud computing system; dynamically allocating, by the at least one memory allocator, an amount of dynamically allocated memory for use by an executing program at runtime; and determining a cost to charge a user based on the amount of memory that has been allocated. Multiple billing methods can be used including: restricting the amount of dynamically allocated memory available to the user based on how much the user is paying; charging the user a fee which grows with an amount of dynamically allocated memory consumed; charging the user higher fees during higher periods of dynamically allocated memory usage; and charging the user a low fee for maintaining dynamically allocated memory usage below a threshold.

Abstract: Automated techniques are disclosed for minimizing communication between nodes in a system comprising multiple nodes for executing requests in which a request type is associated with a particular node. For example, a technique comprises the following steps. Information is maintained about frequencies of compound requests received and individual requests comprising the compound requests. For a plurality of request types which frequently occur in a compound request, the plurality of request types is associated to a same node. As another example, a technique for minimizing communication between nodes, in a system comprising multiple nodes for executing a plurality of applications, comprises the steps of maintaining information about an amount of communication between said applications, and using said information to place said applications on said nodes to minimize communication among said nodes.

Abstract: Automated techniques are disclosed for minimizing communication between nodes in a system comprising multiple nodes for executing requests in which a request type is associated with a particular node. For example, a technique comprises the following steps. Information is maintained about frequencies of compound requests received and individual requests comprising the compound requests. For a plurality of request types which frequently occur in a compound request, the plurality of request types is associated to a same node. As another example, a technique for minimizing communication between nodes, in a system comprising multiple nodes for executing a plurality of applications, comprises the steps of maintaining information about an amount of communication between said applications, and using said information to place said applications on said nodes to minimize communication among said nodes.

Abstract: Automated techniques are disclosed for minimizing communication between nodes in a system comprising multiple nodes for executing requests in which a request type is associated with a particular node. For example, a technique comprises the following steps. Information is maintained about frequencies of compound requests received and individual requests comprising the compound requests. For a plurality of request types which frequently occur in a compound request, the plurality of request types is associated to a same node. As another example, a technique for minimizing communication between nodes, in a system comprising multiple nodes for executing a plurality of applications, comprises the steps of maintaining information about an amount of communication between said applications, and using said information to place said applications on said nodes to minimize communication among said nodes.

Abstract: Automated techniques are disclosed for minimizing communication between nodes in a system comprising multiple nodes for executing requests in which a request type is associated with a particular node. For example, a technique comprises the following steps. Information is maintained about frequencies of compound requests received and individual requests comprising the compound requests. For a plurality of request types which frequently occur in a compound request, the plurality of request types is associated to a same node. As another example, a technique for minimizing communication between nodes, in a system comprising multiple nodes for executing a plurality of applications, comprises the steps of maintaining information about an amount of communication between said applications, and using said information to place said applications on said nodes to minimize communication among said nodes.

Abstract: Improved techniques are disclosed for processing transactions or requests in a data processing system. For example, a method for processing requests in a system including a plurality of nodes includes the following steps. At least two nodes of the plurality of nodes receive a plurality of requests. The two nodes exchange information to determine an order for processing requests. The two nodes process the requests in accordance with the order. The order may include a total order or a partial order.

Abstract: Improved techniques are disclosed for processing transactions or requests in a data processing system. For example, a method for processing requests in a system including a plurality of nodes includes the following steps. At least two nodes of the plurality of nodes receive a plurality of requests. The two nodes exchange information to determine an order for processing requests. The two nodes process the requests in accordance with the order. The order may include a total order or a partial order.