Abstract: Embodiments relate to a new application program interface (API) and supporting tools to introduce efficiency associated with a transaction. An in-memory translation table maintains accurate address locations of key-value pair locations. The new API employs the translation table to obtain both the requested data and any updated address associated with the requested data. Any subsequent API may be communicated with the updated address.

Abstract: A storage controller receives hints provided by one or more applications over a period of time, wherein the hints are used by the storage controller for organizing data in storage managed by the storage controller. Data on conflicts caused by the provided hints are collected over the period of time. Based on the collected data on the conflicts, one or more conflict avoidance rules are executed to reduce possibility of future conflicts.

Abstract: A method to efficiently checkpoint and reconstruct an in-memory index associated with a log-structured object store includes enabling asynchronous write operations to occur to a log-structured object store. The log-structured object store utilizes an in-memory index to access objects therein. The method further enables checkpoint operations to occur to the log-structured object store without pausing the asynchronous write operations. When initiating checkpoint operations, the method establishes a “begin checkpoint” marker on the log-structured object store. This “begin checkpoint” marker is configured to point to an earliest address in the log-structured object store that is uncommitted to the in-memory index. In the event the in-memory index is lost, the method reconstructs the in-memory index by analyzing the log-structured object store starting from the earliest address uncommitted to the in-memory index. A corresponding system and computer program product are also disclosed.

Abstract: A method to efficiently checkpoint and reconstruct an in-memory index associated with a log-structured object store includes enabling asynchronous write operations to occur to a log-structured object store. The log-structured object store utilizes an in-memory index to access objects therein. The method further enables checkpoint operations to occur to the log-structured object store without pausing the asynchronous write operations. When initiating checkpoint operations, the method establishes a “begin checkpoint” marker on the log-structured object store. This “begin checkpoint” marker is configured to point to an oldest known log location recorded in the in-memory index. In the event the in-memory index is lost, the method reconstructs the in-memory index by analyzing the log-structured object store starting from the oldest known log location. A corresponding system and computer program product are also disclosed and claimed herein.

Abstract: Provided are a method, system, and computer program product for processing read and write requests in a storage controller. A host adaptor in the storage controller receives a write request from a host system for a storage address in a storage device. The host adaptor sends write information indicating the storage address updated by the write request to a device adaptor in the storage controller. The host adaptor writes the write data to a cache in the storage controller. The device adaptor indicates the storage address indicated in the write information to a modified storage address list stored in the device adaptor, wherein the modified storage address list indicates modified data in the cache for storage addresses in the storage device.

Abstract: Provided are a computer program product, system, and method for determining storage tiers for placement of data sets during execution of tasks in a workflow. A representation of a workflow execution pattern of tasks for a job indicates a dependency of the tasks and data sets operated on by the tasks. A determination is made of an assignment of the data sets for the tasks to a plurality of the storage tiers based on the dependency of the tasks indicated in the workflow execution pattern. A moving is scheduled of a subject data set of the data sets operated on by a subject task of the tasks that is subject to an event to an assigned storage tier indicated in the assignment for the subject task subject. The moving of the data set is scheduled to be performed in response to the event with respect to the subject task.

Abstract: Provided are a computer program product, system, and method for providing access information to a storage controller to determine a storage tier for storing data. Access information is maintained for each data record in a data store, wherein the access information indicates a level of access to each data record in the data store. A write request directed to a data record in the data store is received. A command is generated identifying the data record and including the access information for the data record. The command is transmitted to the storage controller, wherein the storage controller uses the access information to determine one of the plurality of storage tiers on which to store the data record.

Abstract: Provided are a computer program product, system, and method for providing access information to a storage controller to determine a storage tier for storing data. Access information is maintained for each data record in a data store, wherein the access information indicates a level of access to each data record in the data store. A write request directed to a data record in the data store is received. A command is generated identifying the data record and including the access information for the data record. The command is transmitted to the storage controller, wherein the storage controller uses the access information to determine one of the plurality of storage tiers on which to store the data record.

Abstract: A method for organizing data at levels of granularity larger or smaller than an extent is disclosed. The method initially closes a dataset on a host system. The host system includes a mapping table storing a logical address of the dataset. When the dataset is closed, the host system generates a message indicating that the dataset has been closed and sends the message to a storage system storing the dataset. In response, the storage system allocates a new logical address and copies the dataset from the existing logical address to the new logical address to improve data placement. The storage system then returns the new logical address to the host system so that the host system can update its mapping table. A corresponding system and computer program product are also disclosed.

Abstract: A storage controller receives hints provided by one or more applications over a period of time, wherein the hints are used by the storage controller for organizing data in storage managed by the storage controller. Data on conflicts caused by the provided hints are collected over the period of time. Based on the collected data on the conflicts, one or more conflict avoidance rules are executed to reduce possibility of future conflicts.

Abstract: Provided are a method, system, and computer program product for processing read and write requests in a storage controller. A host adaptor in the storage controller receives a write request from a host system for a storage address in a storage device. The host adaptor sends write information indicating the storage address updated by the write request to a device adaptor in the storage controller. The host adaptor writes the write data to a cache in the storage controller. The device adaptor indicates the storage address indicated in the write information to a modified storage address list stored in the device adaptor, wherein the modified storage address list indicates modified data in the cache for storage addresses in the storage device.

Abstract: Methods and arrangements for selecting a migration decision window for hotspots in a multi-tier enterprise storage system. Aspects include collecting usage statistics for data stored in the multi-tier enterprise storage system, identifying hotspots from data stored in the multi-tier enterprise storage system based on the usage statistics, and determining one or more characteristics of the identified hotspots. Aspects further include calculating an average lifespan of the identified hotspots based on the one or more characteristics of the identified hotspots and selecting the migration decision window based on the average lifespan of the identified hotspots and the one or more characteristics of the identified hotspots.

Abstract: Provided are a computer program product, system, and method for determining storage tiers for placement of data sets during execution of tasks in a workflow. A representation of a workflow execution pattern of tasks for a job indicates a dependency of the tasks and data sets operated on by the tasks. A determination is made of an assignment of the data sets for the tasks to a plurality of the storage tiers based on the dependency of the tasks indicated in the workflow execution pattern. A moving is scheduled of a subject data set of the data sets operated on by a subject task of the tasks that is subject to an event to an assigned storage tier indicated in the assignment for the subject task subject. The moving of the data set is scheduled to be performed in response to the event with respect to the subject task.

Abstract: Provided are a computer program product, system, and method for handling failure of a command to add a record to a log. The log has records at log addresses. Each record in the log includes a key and value and the log addresses map to physical addresses in the storage. A first command for a key and value pair for a record at a specified log address in the log to add to a selected log address in the log is sent. A second command for the key and value pair for the record at the specified log address with a resend parameter set is sent in response to not receiving the selected log address. In response to receiving the second command with the resend parameter set, the log is scanned to determine a log address having the key and value pair. The determined log address is returned.

Abstract: Provided are a computer program product, system, and method for managing data in a storage. Each index entry in and index identifies a log address in the log of one of the records. Each record in the log includes a key and value, and the log addresses map to physical addresses in the storage. A plurality of processed log addresses in the log are processed by determining whether the index indicates that the processed log address includes a current version of the record in the log; adding the record at the processed log address to a selected log address in the log in response to determining that the index indicates that the processed log address comprises the current version of the record; and deleting a record at the physical address in the storage to which the processed log address maps.

Abstract: Provided are a computer program product, system, and method for providing access information to a storage controller to determine a storage tier for storing data. Access information is maintained for each data record in a data store, wherein the access information indicates a level of access to each data record in the data store. A write request directed to a data record in the data store is received. A command is generated identifying the data record and including the access information for the data record. The command is transmitted to the storage controller, wherein the storage controller uses the access information to determine one of the plurality of storage tiers on which to store the data record.

Abstract: A storage controller receives hints provided by one or more applications over a period of time, wherein the hints are used by the storage controller for organizing data in storage managed by the storage controller. Data on conflicts caused by the provided hints are collected over the period of time. Based on the collected data on the conflicts, one or more conflict avoidance rules are executed to reduce possibility of future conflicts.

Abstract: For flash-optimized data placement in multi-tiered, log-structured storage systems, based on at least one key-level data heat metric, sets of key-value pairs determined to exhibit similar data heat characteristics are grouped. Those key-value pairs exhibiting a hotter data heat characteristic are placed into a separate flash memory log-structured data container than those key-value pairs exhibiting a colder data heat characteristic.

Abstract: A computer program product, system, and method for managing metadata for caching devices during shutdown and restart procedures. Fragment metadata for each fragment of data from the storage server stored in the cache device is generated. The fragment metadata is written to at least one chunk of storage in the cache device in a metadata directory in the cache device. For each of the at least one chunk in the cache device to which the fragment metadata is written, chunk metadata is generated for the chunk and writing the generated chunk metadata to the metadata directory in the cache device. Header metadata having information on access of the storage server is written to the metadata directory in the cache device. The written header metadata, chunk metadata, and fragment metadata are used to validate the metadata directory and the fragment data in the cache device during a restart operation.

Abstract: An application server can be configured to access data stored on a networked storage server that is accessible over a network and that includes a cache device configured to store data received from the networked storage server. The application server can include a cache management module that is designed to monitor a data access requests transmitted over the network, the data access requests specifying a first page of data. In response to an indication that the requested data includes data stored in the cache device as an existing page of data, the first page of data can be mapped to a location corresponding to the existing page.