Abstract: A computer program product for controlling dispatching work tasks in a multi-tier storage environment. A computer system receives storage demands of work tasks. The computer system determines placement and migration policies for data in storage tiers in a storage system. The computer system prepares the storage tiers for meeting the storage demands of work tasks, based on the placement and migration policies. The computer system determines a state of preparation of the storage tiers for meeting the storage demands of work tasks. The computer system determines a list including work tasks that can proceed and work tasks that cannot proceed, based on the state of the preparation. The computer system modifies a schedule of the work tasks, based on the list.

Abstract: A computer system for controlling dispatching work tasks in a multi-tier storage environment. The computer system receives storage demands of work tasks. The computer system determines placement and migration policies for data in storage tiers in a storage system. The computer system prepares the storage tiers for meeting the storage demands of work tasks, based on the placement and migration policies. The computer system determines a state of preparation of the storage tiers for meeting the storage demands of work tasks. The computer system determines a list including work tasks that can proceed and work tasks that cannot proceed, based on the state of the preparation. The computer system modifies a schedule of the work tasks, based on the list.

Abstract: Repository data intervals are determined as similar to an input data interval. Repository digests corresponding to the similar repository data interval are loaded into a sequential representation and into a search structure. Matches of input digests and the repository digests are found using the search structure. Each one of the found matches of the input digests and repository digests are extended using the sequential representation. Data matches are determined between the input data and the repository data using extended matches of digests. A compact index pointing to a position in the sequential representation of digests is incorporated into entries of the search structure.

Abstract: Embodiments for automatic diagonal scaling of workloads in a distributed computing environment. For each of a plurality of resources of each of a plurality of application instances, a determination as to whether a change in allocation of at least one of the plurality of resources is required. Operations requirements are computed for each of the plurality of application instances, the computed requirements including vertical increase and decrease operations, and horizontal split and collapse operations. The vertical decrease and horizontal collapse operations are first processed, the vertical increase and horizontal split operations are ordered, and the vertical increase and horizontal split operations are subsequently processed based on the ordering, thereby optimizing application efficiency and utilization of the plurality of resources in the distributed computing environment.

Abstract: Various embodiments for reducing resource allocations and application instances in diagonal scaling in a distributed computing environment. Resource consumption of each one of a plurality of application instances is tracked, and the tracked resource consumption is compared to a resource allocation of each one of the plurality of application instances. A plurality of resource reduction operations for allocation of resources assigned to each one of the plurality of application instances is computed and applied. Idle application instances of the plurality of application instances are identified, and one or more of the identified idle application instances are terminated, thereby optimizing application efficiency and resource utilization in the distributed computing environment.

Abstract: Embodiments for computing and implementing automatic scaling operations in a distributed computing environment. A current allocation of a specific resource used by a specific application instance of an application. A high watermark is configured as a low bound for a high tier of consumption of the specific resource used by the specific application instance, and a low watermark is configured as a high bound for a low tier of consumption of the specific resource used by the specific application instance. An increase in allocation of the specific resource for the specific application instance is computed and implemented when sustained consumption of the specific resource at the high tier is identified, and a decrease in allocation of the specific resource for the specific application instance is computed and implemented when the sustained consumption of the specific resource at the low tier is identified.

Abstract: Embodiments for prioritizing applications for diagonal scaling operations in a distributed computing environment. A significance value of an application of a plurality of applications is defined, the significance value representing an importance of the application or the functionality performed by the application, and dependencies between the plurality of applications are configured. A significance value of a dependency of a dependent application on an antecedent one of the plurality of applications is defined, and priorities for each of the plurality of applications are computed based on the significance values of each of the plurality of applications and respective dependencies therebetween for performing the diagonal scaling operations of resources allocated to each of the applications in the distributed computing environment.

Abstract: Embodiments for increasing resource allocations and application instances in diagonal scaling in a distributed computing environment. Resource consumption of each one of a plurality of application instances is tracked and compared to resource allocations of each one of the plurality of application instances. For each one of a plurality of resource increase operations, a resource increase operation to an associated application instance of the plurality of application instances is computed and applied, the resource increase operation being applied when a resource increase can be accommodated on a host running the associated application instance and an increase limit associated with the associated application instance for the resource has not been reached, thereby optimizing application efficiency and resource utilization in the distributed computing environment.

Abstract: Various embodiments for adaptively computing and implementing changes in resource allocations using historical data in a distributed computing environment. Historical data on resource consumption levels of an application is maintained. Either of an increase operation or reduction operation is triggered for a resource required by an application instance of the application, where a predicted consumption level for the resource required by the application instance is computed. The increase or reduction operation is computed and implemented for the resource required by the application instance to accommodate the predicted consumption level, thereby optimizing application efficiency and resource utilization in the distributed computing environment.

Abstract: Embodiments for managing resources in a tiered data storage system, by a processor device. In response to differing business priorities for executing workloads by a workload manager, a resource plan is generated by an administrator of the workload manager. The resource plan defines what resources of the storage system are allocated to a particular group of tenants and at what time the workloads of the particular group are executed, thereby optimizing utilization of the resources of the storage system.

Abstract: A method, a computer program product, and a computer system for controlling dispatching work tasks in a multi-tier storage environment. A computer system receives storage demands of work tasks. The computer system determines placement and migration policies for data in storage tiers in a storage system. The computer system prepares the storage tiers for meeting the storage demands of work tasks, based on the placement and migration policies. The computer system determines a state of preparation of the storage tiers for meeting the storage demands of work tasks. The computer system determines a list including work tasks that can proceed and work tasks that cannot proceed, based on the state of the preparation. The computer system modifies a schedule of the work tasks, based on the list.

Abstract: Data matches are calculated between input data and repository data via a digest based matching algorithm where the reference digests corresponding to a repository interval of data identified as similar to an input interval of data are loaded into a sequential array and into a search structure. Each of the matching digests found using the search structure are extended using the sequential array of reference digests. Repository data intervals are determined as similar to an input data interval. Reference digests corresponding to the similar repository data interval are loaded into a sequential representation and into a search structure. Matches of input digests and the reference digests are found using the search structure. Each one of the found matches of the input digests and repository digests are extended using the sequential representation. Data matches are determined between the input data and the repository data using extended matches of digests.

Abstract: For read ahead of digests in similarity based data deduplication in a data deduplication system using a processor device in a computing environment, input data is partitioned into data chunks and digest values are calculated for each of the data chunks. The positions and sizes of similar data intervals in a repository of data are found for each of the data chunks. The positions and the sizes of read ahead intervals are calculated based on the similar data intervals. The read ahead digests of the read ahead intervals are located and loaded into memory in a background read ahead process.

Abstract: A computer program product for searching a repository of binary uninterpretted data, according to one embodiment, includes a computer readable storage medium having program instructions executable by a computer to cause the computer to perform a method comprising: analyzing, by the computer, segments of each of the repository and input data to determine a repository segment that is similar to an input segment, the analyzing including searching an index of representation values of the repository data for matching representation values of the input in a time independent of a size of the repository and linear in a size of the input data; and analyzing, by the computer, the similar repository segment with respect to the input segment to determine their common data sections while utilizing at least some of the matching representation values for data alignment, in a time linear in a size of the input segment.

Abstract: Embodiments for rebalancing distributed deduplication metadata by a processor. An input similarity unit of data and a corresponding input representative value are received on an ingesting processor. A corresponding deduplication metadata slice and owning processor are calculated for the input similarity unit. The input representative value and input digest values are sent to the owning processor. The owning processor is used to search for the input representative value in a corresponding deduplication metadata slice, and to forward the input representative value and input digest values to an additional processor, if the input representative value is not found by the owning processor and a rebalancing status of the owning processor is in-process. The additional processor is used to send a reply message to the owning processor that facilitates migration of the input representative value and corresponding input digest values to the owning processor, if the input representative value is found.

Abstract: Embodiments for distributed data deduplication in a grid of processors. Input data is received on a processor. The input data is partitioned into a plurality of similarity units. A corresponding deduplication metadata slice and owning processor for one of the similarity units is calculated. A representative value and corresponding digest values of the similarity unit are sent to the owning processor. The owning processor is used to search for the representative value in the deduplication metadata slice, and to send a specification and owning processors of calculated identical data sections to the processor. The processor is used to send nominal information of the calculated identical data sections to the owning processors of the data referenced by the calculated identical data sections.

Abstract: Various embodiments for managing data in a data storage having data deduplication. For a back reference data structure incorporating reference information for at least one user data segment to a storage block, a user data segment identification (ID) representative of the at least one user data segment is removed from the back reference data structure.

Abstract: Embodiments for storing data deduplication metadata in a grid of processors. Each of a plurality of slices of deduplication metadata is assigned to be stored by a corresponding processor in a grid of processors. Each slice of the plurality of slices includes at least one of a slice of a similarity index and groups of digests corresponding to those of a plurality of representative values in the slice of the similarity index. A hashing method is use to map between a plurality of input representative values and the plurality of slices of deduplication metadata.

Abstract: Data matches are calculated between input data and repository data via a digest based matching algorithm where in a first step digest matches, anchored at already verified matching positions in the input data and in the repository data, are extended to produce data matches. In a second step the remaining unmatched input digests are matched with repository digests and extended to produce further data matches.

Abstract: For producing digest block segmentations based on reference segmentations in a data deduplication system using a processor device in a computing environment, digests are calculated for an input data chunk. Data matches and data mismatches are produced based on matching input digests with reference digests. Secondary digest block segmentations are obtained from similar reference intervals for each of the data mismatches and applied to the input data.