Abstract: Embodiments for optimizing database backups to achieve a Recovery Time Object (RTO). A user-defined RTO configured for one or more databases is received. A backup frequency for initiating backups of the one or more databases is determined based on a continuously predicted recovery time associated with a plurality of factors. The backups of the one or more databases are executed at the determined backup frequency to ensure the user-defined RTO is achieved for the backups of the one or more databases. In some embodiments, a recovery window of the one or more databases may be increased using an RTO-aware tiered or remote storage caching operation for portions of the database, and an RTO-aware re-sharding operation on sharded databases may be performed when the backup frequency exceeds a predetermined threshold such that each shard may be restored within the user-defined RTO.

Abstract: A computer system may include a storage device and nodes cooperating therewith to define a key value database architecture. A first node of the nodes may be configured to generate a plurality of checkpoints. The computer system may also include a second node of the nodes configured to operate a log feed cache manager based upon the checkpoints generated by the first node.

Abstract: One embodiment provides a method for recovery after failure using a checkpoint in a chronological log-structured key-value store in a system including writing tombstone entries in a log structure for dirty checkpoint records to point to data records in an aborted target slot.

Abstract: One embodiment provides a method for recovery from failures during garbage collection processing in a system including recording, by a processor, a specific offset within a garbage collection target slot of a log structure associated with a garbage collection transaction. Each transaction record of the garbage collection transaction includes a garbage collection target slot, a victim slot and a beginning offset in the garbage collection target slot.

Abstract: One embodiment provides a method for recovery from failures during garbage collection processing in a system including performing, by a processor, chronological garbage collection transactionally. A specific offset within a target slot of a log structure associated with the garbage collection transaction is recorded. Records are skipped from an initial offset in the target slot of the log structure recorded by the garbage collection transaction until an end of the target slot.

Abstract: One embodiment provides a method for recovery after failure using a checkpoint in a chronological log-structured key-value store in a system including recording, by a processor, a system state prior to an aborted garbage collection operation. The processor writes tombstone entries in a log structure for dirty checkpoint records to point to data records in an aborted target slot. New checkpoint records are inserted in the log structure for the dirty checkpoint records.

Abstract: Embodiments are provided for organizing data elements in a log-structured storage system. A profile is created based on a population of data elements in cache memory related to a log structured storage system. A layout for storage of the data elements on persistent storage related to the log-structured storage system is determined, which includes identifying two or more related data elements based on the profile. A garbage collection process is performed on the persistent storage based on the determined layout, which includes placing the related data elements within substantially close proximity on the persistent storage.

Abstract: An example operation may include one or more of identifying one or more data volume management policies from a plurality of different parties, storing the one or more data volume management policies in a smart contract stored on a blockchain, identifying one or more data access thresholds being exceeded on a network, modifying an operating status of the network based on the one or more policies and the one or more data access thresholds being exceeded, and storing the modified operating status in the blockchain.

Abstract: In accordance with one aspect of the present description, in response to a detection by a storage controller, of an operation by a host relating to migration of input/output operations from one host to another, a cache server of a storage controller, transmits to a target cache client of the target host, a cache map of the source cache of the source host wherein the cache map identifies locations of a portion of the storage cached in the source cache. In response, the cache client of the target host, may populate the target cache of the target host with data from the locations of the portion of the storage, as identified by the cache map transmitted by the cache server, which may reduce cache warming time. Other features or advantages may be realized in addition to or instead of those described herein, depending upon the particular application.

Abstract: One embodiment provides a method for retrospective snapshot creation including creating, by a processor, a first snapshot that captures logical state of a data store at a first key. Creation of the first snapshot is based on determining a log offset corresponding to the first key, determining existence of a second snapshot that captures logical state of the data store and recording a retrospective snapshot at a last valid log address offset prior to the first key upon a determination that the second snapshot exists based on determining at least one of: whether log address offsets from a first log entry of a log to a log entry of the log at the first key are contiguous and whether log address offsets from the second snapshot to the first key are contiguous.

Abstract: In accordance with one aspect of the present description, in response to a detection by a storage controller, of an operation by a host relating to migration of input/output operations from one host to another, a cache server of a storage controller, transmits to a target cache client of the target host, a cache map of the source cache of the source host wherein the cache map identifies locations of a portion of the storage cached in the source cache. In response, the cache client of the target host, may populate the target cache of the target host with data from the locations of the portion of the storage, as identified by the cache map transmitted by the cache server, which may reduce cache warming time. Other features or advantages may be realized in addition to or instead of those described herein, depending upon the particular application.

Abstract: Provided are a computer program product, system, and method for determining storage tiers for placement of data sets during execution of tasks in a workflow. A representation of a workflow execution pattern of tasks for a job indicates a dependency of the tasks and data sets operated on by the tasks. A determination is made of an assignment of the data sets for the tasks to a plurality of the storage tiers based on the dependency of the tasks indicated in the workflow execution pattern. A moving is scheduled of a subject data set of the data sets operated on by a subject task of the tasks that is subject to an event to an assigned storage tier indicated in the assignment for the subject task subject. The moving of the data set is scheduled to be performed in response to the event with respect to the subject task.

Abstract: A computer-implemented method for distributing data among memory devices may include performing successive spatial partitionings of a graph of data based upon vertices and associated edges to generate spatial partitions with each spatial partition having a respective number of edges below a threshold. The method may also include ordering the spatial partitions and distributing the spatial partitions among the memory devices based upon the ordering.

Abstract: One embodiment provides a method including storing a key-value store in a memory. Out-of-place operations are performed to maintain chronological ordering of the operations by enforcing ordering of the operations on a storage layer in presence of a garbage collection transaction without an explicit secondary index.

Abstract: One embodiment provides a method for recovery from failures during garbage collection processing in a system including performing, by a processor, chronological garbage collection transactionally. A specific offset within a target slot of a log structure associated with the garbage collection transaction is recorded. Records are skipped from an initial offset in the target slot of the log structure recorded by the garbage collection transaction until an end of the target slot.

Abstract: One embodiment provides a method for recovery after failure using a checkpoint in a chronological log-structured key-value store in a system including recording, by a processor, a system state prior to an aborted garbage collection operation. The processor writes tombstone entries in a log structure for dirty checkpoint records to point to data records in an aborted target slot. New checkpoint records are inserted in the log structure for the dirty checkpoint records.

Abstract: Provided are a computer program product, system, and method for managing data in a storage. Each index entry in and index identifies a log address in the log of one of the records. Each record in the log includes a key and value, and the log addresses map to physical addresses in the storage. A plurality of processed log addresses in the log are processed by determining whether the index indicates that the processed log address includes a current version of the record in the log; adding the record at the processed log address to a selected log address in the log in response to determining that the index indicates that the processed log address comprises the current version of the record; and deleting a record at the physical address in the storage to which the processed log address maps.

Abstract: Provided are a computer program product, system, and method for handling failure of a command to add a record to a log. The log has records at log addresses. A command is received to add a record to the log indicating a current log address at which the record is stored in the log. A determination is made as to whether the command has a resend parameter set, wherein the resend parameter indicates whether the command is being resent to add the record. A determination is made as to whether there is a log address in the log having the record following the current log address in response to determining that the resend parameter indicates that the command is being resent. The determined log address in the log is returned.

Abstract: Provided are a computer program product, system, and method for managing data in a storage. Each index entry in and index identifies a log address in the log of one of the records. Each record in the log includes a key and value, and the log addresses map to physical addresses in the storage. A plurality of processed log addresses in the log are processed by determining whether the index indicates that the processed log address includes a current version of the record in the log; adding the record at the processed log address to a selected log address in the log in response to determining that the index indicates that the processed log address comprises the current version of the record; and deleting a record at the physical address in the storage to which the processed log address maps.

Abstract: A method, computer system, and a computer program product for organizing a plurality of column families based on data content is provided. The present invention may include analyzing a plurality of data. The present invention may also include generating a plurality of individual columns based on the analyzed plurality of data. The present invention may then include identifying a plurality of temporal access patterns associated with the generated plurality of individual columns based on the content of the analyzed plurality of data. The present invention may further include forming the plurality of column families based on the identified plurality of temporal access patterns. The present invention may also include storing the formed plurality of column families in a key-value store.