Abstract: An example operation may include one or more of identifying one or more data volume management policies from a plurality of different parties, storing the one or more data volume management policies in a smart contract stored on a blockchain, identifying one or more data access thresholds being exceeded on a network, modifying an operating status of the network based on the one or more policies and the one or more data access thresholds being exceeded, and storing the modified operating status in the blockchain.

Abstract: One embodiment provides a method for retrospective snapshot creation including creating, by a processor, a first snapshot that captures logical state of a data store at a first time in a time range. Creation of the first snapshot is based on determining existence of a second snapshot that captures logical state of the data store and recording a retrospective snapshot at a last valid log address offset prior to the first time upon a determination that the second snapshot exists based on determining at least one of: whether log address offsets from a first log entry of a log to a log entry of the log at the first time are contiguous and whether log address offsets from the second snapshot to the first time are contiguous.

Abstract: Provided are a computer program product, system, and method for providing access information to a storage controller to determine a storage tier for storing data. Access information is maintained for each data record in a data store, wherein the access information indicates a level of access to each data record in the data store. A write request directed to a data record in the data store is received. A command is generated identifying the data record and including the access information for the data record. The command is transmitted to the storage controller, wherein the storage controller uses the access information to determine one of the plurality of storage tiers on which to store the data record.

Abstract: A computer-implemented method for distributing data among memory devices may include performing successive spatial partitionings of a graph of data based upon vertices and associated edges to generate spatial partitions with each spatial partition having a respective number of edges below a threshold. The method may also include ordering the spatial partitions and distributing the spatial partitions among the memory devices based upon the ordering.

Abstract: A computer system may include a storage device and nodes cooperating therewith to define a key value database architecture. A first node of the nodes may be configured to generate a plurality of checkpoints. The computer system may also include a second node of the nodes configured to operate a log feed cache manager based upon the checkpoints generated by the first node.

Abstract: A method to efficiently checkpoint and reconstruct an in-memory index associated with a log-structured object store includes enabling asynchronous write operations to occur to a log-structured object store. The log-structured object store utilizes an in-memory index to access objects therein. The method further enables checkpoint operations to occur to the log-structured object store without pausing the asynchronous write operations. When initiating checkpoint operations, the method establishes a “begin checkpoint” marker on the log-structured object store. This “begin checkpoint” marker is configured to point to an earliest address in the log-structured object store that is uncommitted to the in-memory index. In the event the in-memory index is lost, the method reconstructs the in-memory index by analyzing the log-structured object store starting from the earliest address uncommitted to the in-memory index. A corresponding system and computer program product are also disclosed.

Abstract: A method to efficiently checkpoint and reconstruct an in-memory index associated with a log-structured object store includes enabling asynchronous write operations to occur to a log-structured object store. The log-structured object store utilizes an in-memory index to access objects therein. The method further enables checkpoint operations to occur to the log-structured object store without pausing the asynchronous write operations. When initiating checkpoint operations, the method establishes a “begin checkpoint” marker on the log-structured object store. This “begin checkpoint” marker is configured to point to an oldest known log location recorded in the in-memory index. In the event the in-memory index is lost, the method reconstructs the in-memory index by analyzing the log-structured object store starting from the oldest known log location. A corresponding system and computer program product are also disclosed and claimed herein.

Abstract: Provided are a computer program product, system, and method for handling failure of a command to add a record to a log. The log has records at log addresses. A command is received to add a record to the log indicating a current log address at which the record is stored in the log. A determination is made as to whether the command has a resend parameter set, wherein the resend parameter indicates whether the command is being resent to add the record. A determination is made as to whether there is a log address in the log having the record following the current log address in response to determining that the resend parameter indicates that the command is being resent. The determined log address in the log is returned.

Abstract: Provided are a method, system, and computer program product for processing read and write requests in a storage controller. A host adaptor in the storage controller receives a write request from a host system for a storage address in a storage device. The host adaptor sends write information indicating the storage address updated by the write request to a device adaptor in the storage controller. The host adaptor writes the write data to a cache in the storage controller. The device adaptor indicates the storage address indicated in the write information to a modified storage address list stored in the device adaptor, wherein the modified storage address list indicates modified data in the cache for storage addresses in the storage device.

Abstract: Provided are a computer program product, system, and method for handling failure of a command to add a record to a log. The log has records at log addresses. A command is received to add a record to the log indicating a current log address at which the record is stored in the log. A determination is made as to whether the command has a resend parameter set, wherein the resend parameter indicates whether the command is being resent to add the record. A determination is made as to whether there is a log address in the log having the record following the current log address in response to determining that the resend parameter indicates that the command is being resent. The determined log address in the log is returned.

Abstract: In accordance with one aspect of the present description, in response to a detection by a storage controller, of an operation by a host relating to migration of input/output operations from one host to another, a cache server of a storage controller, transmits to a target cache client of the target host, a cache map of the source cache of the source host wherein the cache map identifies locations of a portion of the storage cached in the source cache. In response, the cache client of the target host, may populate the target cache of the target host with data from the locations of the portion of the storage, as identified by the cache map transmitted by the cache server, which may reduce cache warming time. Other features or advantages may be realized in addition to or instead of those described herein, depending upon the particular application.

Abstract: Provided are a computer program product, system, and method for determining storage tiers for placement of data sets during execution of tasks in a workflow. A representation of a workflow execution pattern of tasks for a job indicates a dependency of the tasks and data sets operated on by the tasks. A determination is made of an assignment of the data sets for the tasks to a plurality of the storage tiers based on the dependency of the tasks indicated in the workflow execution pattern. A moving is scheduled of a subject data set of the data sets operated on by a subject task of the tasks that is subject to an event to an assigned storage tier indicated in the assignment for the subject task subject. The moving of the data set is scheduled to be performed in response to the event with respect to the subject task.

Abstract: One embodiment provides a method for retrospective snapshot creation including creating, by a processor, a first snapshot that captures logical state of a data store at a first key. Creation of the first snapshot is based on determining a log offset corresponding to the first key, determining existence of a second snapshot that captures logical state of the data store and recording a retrospective snapshot at a last valid log address offset prior to the first key upon a determination that the second snapshot exists based on determining at least one of: whether log address offsets from a first log entry of a log to a log entry of the log at the first key are contiguous and whether log address offsets from the second snapshot to the first key are contiguous.

Abstract: One embodiment provides a method for retrospective snapshot creation including creating, by a processor, a first snapshot that captures logical state of a data store at a first time in a time range. Creation of the first snapshot is based on determining existence of a second snapshot that captures logical state of the data store and recording a retrospective snapshot at a last valid log address offset prior to the first time upon a determination that the second snapshot exists based on determining at least one of: whether log address offsets from a first log entry of a log to a log entry of the log at the first time are contiguous and whether log address offsets from the second snapshot to the first time are contiguous.

Abstract: In accordance with one aspect of the present description, in response to a detection by a storage controller, of an operation by a host relating to migration of input/output operations from one host to another, a cache server of a storage controller, transmits to a target cache client of the target host, a cache map of the source cache of the source host wherein the cache map identifies locations of a portion of the storage cached in the source cache. In response, the cache client of the target host, may populate the target cache of the target host with data from the locations of the portion of the storage, as identified by the cache map transmitted by the cache server, which may reduce cache warming time. Other features or advantages may be realized in addition to or instead of those described herein, depending upon the particular application.

Abstract: Provided are a computer program product, system, and method for determining storage tiers for placement of data sets during execution of tasks in a workflow. A representation of a workflow execution pattern of tasks for a job indicates a dependency of the tasks and data sets operated on by the tasks. A determination is made of an assignment of the data sets for the tasks to a plurality of the storage tiers based on the dependency of the tasks indicated in the workflow execution pattern. A moving is scheduled of a subject data set of the data sets operated on by a subject task of the tasks that is subject to an event to an assigned storage tier indicated in the assignment for the subject task subject. The moving of the data set is scheduled to be performed in response to the event with respect to the subject task.

Abstract: Provided are a computer program product, system, and method for handling failure of a command to add a record to a log. The log has records at log addresses. A command is received to add a record to the log indicating a current log address at which the record is stored in the log. A determination is made as to whether the command has a resend parameter set, wherein the resend parameter indicates whether the command is being resent to add the record. A determination is made as to whether there is a log address in the log having the record following the current log address in response to determining that the resend parameter indicates that the command is being resent. The determined log address in the log is returned.

Abstract: Embodiments are provided for organizing data elements in a log-structured storage system. A profile is created based on a population of data elements in cache memory related to a log structured storage system. A layout for storage of the data elements on persistent storage related to the log-structured storage system is determined, which includes identifying two or more related data elements based on the profile. A garbage collection process is performed on the persistent storage based on the determined layout, which includes placing the related data elements within substantially close proximity on the persistent storage.

Abstract: Provided are a computer program product, system, and method for handling failure of a command to add a record to a log. The log has records at log addresses. Each record in the log includes a key and value and the log addresses map to physical addresses in the storage. A first command for a key and value pair for a record at a specified log address in the log to add to a selected log address in the log is sent. A second command for the key and value pair for the record at the specified log address with a resend parameter set is sent in response to not receiving the selected log address. In response to receiving the second command with the resend parameter set, the log is scanned to determine a log address having the key and value pair. The determined log address is returned.

Abstract: For flash-optimized data placement in multi-tiered, log-structured storage systems, based on at least one key-level data heat metric, sets of key-value pairs determined to exhibit similar data heat characteristics are grouped. Those key-value pairs exhibiting a hotter data heat characteristic are placed into a separate flash memory log-structured data container than those key-value pairs exhibiting a colder data heat characteristic.