Abstract: Provided are a computer program product, system, and method for profiling workloads in host systems allocated to a cluster to determine adjustments to allocation of host systems to the cluster. A determination is made of workloads processing and resource usage in a computing system. An aggregate completion duration for determined workloads that have completed processing is updated. An aggregate consumed resources, comprising an aggregate of resources consumed by workloads, by resources consumed by the determined workloads is updated. The aggregate completion duration and the aggregate consumed resources are used to determine resources to provision for workloads.

Abstract: Provided are a computer program product, system, and method for estimating resource requests for workloads to offload to host systems in a computing environment. A calculation is made required resources of computational resources required to complete processing a plurality of unfinished workloads that have not completed. A determination is made of allocated resources that are not yet provisioned to workloads. The required resources are reduced by the allocated resources not yet provisioned to determine resources to provision. The resources to provision for the unfinished workloads are requested.

Abstract: Embodiments for combining input data matches in data deduplication of input data by a processor. Matches of input data are calculated using a plurality of independent deduplication processes referencing a plurality of repository data segments for the input data. A combined list of output data matches is calculated by removing those of the input data matches that are fully enclosed within other input data matches; and removing those of the input data matches determined to be smaller than a predetermined threshold for citing. A deduplication operation is performed on the combined list of output data matches. Each pair of the input data matches having an overlap section is processed in an ascending order of a position.

Abstract: Data matches are calculated in a data deduplication system by matching input and repository digests using a digest based data matching process where the reference digests corresponding to a repository interval of data identified as similar to an input interval of data are loaded into two data structures. The dual data structures include a sequential buffer containing a plurality of digest entries in a sequence corresponding to a placement order of calculated values of the reference digests, the placement order of the calculated values of the reference digests correlative to an order in which input digest values were calculated such that the plurality of digests are stored in a linear form independent of a deduplicated form by which data the plurality of digests describe is stored.

Abstract: Embodiments for workload management with data access awareness by ordering hosts for scheduling workloads in a computing cluster. In response to receiving an input workload for scheduling by a workload manager, a set of inputs is retrieved from a storage system by a data requirements evaluator module. The data requirements evaluator module generates a list of cluster hosts ranked for performing the input workload according to data access considerations.

Abstract: Embodiments for aggregated information calculation and injection for application containers by one or more processors. Prior to commencing execution of an application inside a working container, a temporary container having an equivalent application template or container template as the working container is started. A first instance of the application is instantiated and executed from inside the temporary container. Relevant information, obtained during the execution of the first application instance from inside the temporary container, and relevant information from a host associated with the application, is extracted. The relevant information from the host and the temporary container is aggregated. A second instance of the application is executed and the aggregated information is injected into the working container.

Abstract: A computer-implemented method, according to one embodiment includes, for each repository data chunk in repository data that comprises a plurality of the repository data chunks, generating a corresponding set of repository distinguishing characteristics (RDCs). Each set of RDCs is generated by: applying a hash function to the respective input data chunk or repository data chunk to generate a plurality of hashes, each hash comprising a hash value and a hash position within the data chunk, applying a first function to the plurality of generated hashes to identify a first subset of hashes distributed across the data chunk, applying a second function to the hash positions of the hashes of the first subset to identify a second subset of the plurality of generated hashes, and defining the second subset of hashes as the set of RDCs.

Abstract: Embodiments for managing resources in a tiered data storage system, by a processor device. In response to differing business priorities for executing workloads by a workload manager, a resource plan is generated by an administrator of the workload manager. The resource plan defines what resources of the storage system are allocated to a particular group of tenants and at what time the workloads of the particular group are executed, thereby optimizing utilization of the resources of the storage system. A time period of which a particular one of a plurality of storage tiers of the tiered data storage system is allocated for use by the respective one of the tenants is defined by an administrator, where a peak demand summed over all of tenants is allowed to exceed a total capacity of each individual one of the plurality of storage tiers.

Abstract: Embodiments for computing and implementing automatic scaling operations in a distributed computing environment. A current allocation of a specific resource used by a specific application instance of an application. A high watermark is configured as a low bound for a high tier of consumption of the specific resource used by the specific application instance, and a low watermark is configured as a high bound for a low tier of consumption of the specific resource used by the specific application instance. An increase in allocation of the specific resource for the specific application instance is computed and implemented when sustained consumption of the specific resource at the high tier is identified, and a decrease in allocation of the specific resource for the specific application instance is computed and implemented when the sustained consumption of the specific resource at the low tier is identified.

Abstract: Input backup data is deduplicated with data of a synthetic backup previously constructed by a deduplication storage system. A synthetic backup is constructed by processing metadata instructions provided by a backup application. Deduplication digests are calculated based on the data of the synthetic backup and the deduplication digests are stored in a digests index. When new backup data is processed, deduplication digests of the new data are calculated and searched in the digests index. Matching digests of previously constructed synthetic backups are located in the digests index. Each of the located matching digest references stored data are included in the synthetic backup, and the stored data is similar to the input backup data. Data matches are found in the input backup data and data in the synthetic backup.

Abstract: Embodiments for aggregated information calculation and injection for application containers by one or more processors. Prior to commencing execution of an application inside a working container, a temporary container having an equivalent application template or container template as the working container is started. Relevant information from inside the temporary container and relevant information from a host associated with the application is extracted. Aggregated information is calculated based on the extracted relevant information from inside the temporary container and from the host.

Abstract: A DSM component is organized as a matrix of page. The data structure of a set of data structures occupies a column in the matrix of pages. A recovery file is maintained in a persistent storage. The recovery file consists of entries and each one of the entries corresponds to a column in the matrix of pages by a location of each one of the entries. The set of data structures is stored in the DSM component and in the persistent storage. Incorporated into each one of the plurality of entries in the recovery file is an indication if an associated column in the matrix of pages is assigned with the data structure of the set of data structures; and additionally incorporated into each one of the plurality of entries in the recovery file are identifying key properties of the data structure of the set of data structures.

Abstract: Embodiments for managing data stored in a tiered data storage system, by a processor device. Upon submission of a workload to a queue of workloads of a workload manager, a set of storage demands included in metadata associated with the workload is received by an optimization policy generator. A current state of each one of a plurality of storage tiers within the storage system is combined with information comprising the set of storage demands to create data scheduling and placement policies such that the optimization policy generator controls which storage tier in the tiered data storage system incoming data is written to while a scheduling logic determines at what time to dispatch at least a portion of the workload.

Abstract: Input backup data is deduplicated with data of a synthetic backup previously constructed by a deduplication storage system. A synthetic backup is constructed by processing metadata instructions provided by a backup application. Deduplication digests are calculated based on the data of the synthetic backup and the deduplication digests are stored in a digests index. When new backup data is processed, deduplication digests of the new data are calculated and searched in the digests index. Matching digests of previously constructed synthetic backups are located in the digests index. Each of the located matching digest references stored data are included in the synthetic backup, and the stored data is similar to the input backup data. Data matches are found in the input backup data and data in the synthetic backup.

Abstract: Embodiments for workload management with data access awareness by ordering hosts for scheduling workloads in a computing cluster. In response to receiving an input workload for scheduling by a workload manager, a set of inputs is retrieved from a storage system by a data requirements evaluator module. The data requirements evaluator module generates a list of cluster hosts ranked for performing the input workload according to data access considerations.

Abstract: A system for improving workload performance using data locality and workload placement may include a schedule module that schedules workloads for execution on a computing device in one or more computing devices. Also, the computing devices access a shared data storage. Further, the system may include a tracking module that maintains a data store that associates paths in the shared data storage with dispatched workloads and associates the paths with at least one computing device, wherein a dispatched workload and the at least one computing device accessed a path; and a preferred device identification module that identifies preferred computing devices in the one or more computing devices for executing undispatched workloads based on information in the data store, wherein the schedule module schedules the execution of the undispatched workloads in response to the identified preferred computing devices.

Abstract: Embodiments for workload management by aggregating locality information for a set of files in a cluster of hosts, from a file level to a level of the set of files in a cluster of hosts. To facilitate workload scheduling in the cluster, a subset of the set of files is selected. A set of storage size counters, each assigned to a host in the cluster, is reset. An overall storage size counter is reset, and the files in the subset of the set of files are scanned. For each scanned file, locality information of the file is retrieved and added to the storage size counters of the hosts, and a total size of the file is added to the overall storage size counter. An output proportion of the storage size counter of each host is then computed from the overall storage size counter.

Abstract: Embodiments for workload management with data access awareness in a computing cluster. In response to receiving an input workload for scheduling by a workload manager, a set of inputs is retrieved from a storage system by a data requirements evaluator module. The data requirements evaluator module generates a list of cluster hosts ranked for performing the input workload according to data access considerations and provides the ranked list of cluster hosts to a scheduler module. The scheduler module generates a scheduling of the input workload to certain hosts within the computing cluster where the generated scheduling is optimized with the data access considerations.

Abstract: Embodiments for workload management with data access awareness by ordering hosts for scheduling workloads in a computing cluster. In response to receiving an input workload for scheduling by a workload manager, a set of inputs is retrieved from a storage system by a data requirements evaluator module. The data requirements evaluator module generates a list of cluster hosts ranked for performing the input workload according to data access considerations.

Abstract: Embodiments for managing resources in a tiered data storage system, by a processor device. In response to differing business priorities for executing workloads by a workload manager, a resource plan is generated by an administrator of the workload manager. The resource plan defines what resources of the storage system are allocated to a particular group of tenants and at what time the workloads of the particular group are executed, thereby optimizing utilization of the resources of the storage system. A time period of which a particular one of a plurality of storage tiers of the tiered data storage system is allocated for use by the respective one of the tenants is defined by an administrator, where a peak demand summed over all of tenants is allowed to exceed a total capacity of each individual one of the plurality of storage tiers.