Abstract: A system uses a hybrid key-value storage engine that uses log-structured merge tree and a segmented log-structured object store. The system performs garbage collection of stale document versions avoiding index lookup during log segment compaction. The system separates index and document data to minimize write amplification. The system maintains a delete list using a log-structured merge-tree to store stale document sequence numbers and corresponding sizes per log segment. For each log segment from the plurality of log segments, the system determines a measure of fragmentation of the log segment based on sizes of deleted documents of the log segment from the second log-structured merge-tree. If the fragmentation exceeds a threshold, the system initiates a compaction operation for the log segment.

Abstract: A system uses a hybrid key-value storage engine that uses log-structured merge tree and a segmented log-structured object store. The system performs garbage collection of stale document versions avoiding index lookup during log segment compaction. The system separates index and document data to minimize write amplification. The system maintains a delete list using a log-structured merge-tree to store stale document sequence numbers and corresponding sizes per log segment. For each log segment from the plurality of log segments, the system determines a measure of fragmentation of the log segment based on sizes of deleted documents of the log segment from the second log-structured merge-tree. If the fragmentation exceeds a threshold, the system initiates a compaction operation for the log segment.

Abstract: A method achieves data consistency in a shared storage accessible by a first and second machine. The method includes, in response to receiving state information of the first machine, configuring the second machine to a mirrored operating state corresponding to an operating state of the first machine, receiving a notification that the first machine will overwrite existing data stored in the shared storage, and, in response to the notification, reading the existing data, storing a copy of existing data in a local storage of the second machine, and sending an acknowledgment to the first machine that the copy has been stored in the local storage, to enable the first machine to overwrite the existing data with newly written data. The method also includes, in response to receiving a failure notification, retrieving the copy of the existing data, overwriting the newly written data with the copy of the existing data.

Abstract: A method achieves data consistency in a shared storage accessible by a first and second machine. The method includes, in response to receiving state information of the first machine, configuring the second machine to a mirrored operating state corresponding to an operating state of the first machine, receiving a notification that the first machine will overwrite existing data stored in the shared storage, and, in response to the notification, reading the existing data, storing a copy of existing data in a local storage of the second machine, and sending an acknowledgment to the first machine that the copy has been stored in the local storage, to enable the first machine to overwrite the existing data with newly written data. The method also includes, in response to receiving a failure notification, retrieving the copy of the existing data, overwriting the newly written data with the copy of the existing data.

Abstract: A method of achieving data consistency in a shared storage accessible by a first and second machine. The method includes, in response to receiving state information of the first machine, configuring the second machine to a mirrored operating state corresponding to an operating state of the first machine, receiving a notification that the first machine will overwrite existing data stored in the shared storage, and, in response to the notification, reading the existing data, storing a copy of existing data in a local storage of the second machine, and sending an acknowledgment to the first machine that the copy has been stored in the local storage, to enable the first machine to overwrite the existing data with newly written data. The method also includes, in response to receiving a failure notification, retrieving the copy of the existing data, overwriting the newly written data with the copy of the existing data.