Abstract: Use of a log service computing system to apply a log to storage segments in the context of there being a primary compute system that operates upon data, and secondary compute system(s) that are at the ready to be promoted to the new primary compute system should the old primary compute system fail. Rather than have each of the primary and secondary compute systems maintain their own replicated of the entire data locally, the data is separated in storage segments that are collectively distributed amongst multiple storage segment servers.

Abstract: The mounting a drive to two or more computing systems. For instance, the drive may be mounted to a first computing system so as to be writable (and potentially readable) by the first computing system. But also, the drive is also mounted to one or more other computing systems so as to be only readable by those one or more computing systems. This allows for multiple computing systems to have access to the drive without risk that the data thereon will become corrupt. In one embodiment, the only user data stored on that drive is a single file of fixed size. Thus, even when user data is written into the fixed-size file, the management data stored (that keeps track of the files) on the drive does not change.

Abstract: Assembly of a state of user data as of a particular point in time in the context of the data being spread across multiple storage segment servers. Furthermore, the assembly occurs despite the storage segment servers not needing to coordinate a timing of any snapshots. Rather, the storage segment servers perform a snapshot of the portion of the data that it individually stores without having to coordinate with any other storage segment server. The particular point of time may be any arbitrary time, such as a time selected by a user.

Abstract: A versioned records management system that uses a restart era in order to promote rapid recovery. A persistent store includes a plurality of items that are each marked with a restart era. Any item that was created in the current restart era is marked with the current restart era. The persistent store also includes a first subset of the items comprising a plurality of records. At least one record includes multiple corresponding versions. The persistent store also includes a second subset of the items comprising a collection of objects that conform to an object model that is used to operate upon the plurality of records via the object collection. The versioned records management system recovers from a crash by promoting an item in the persistent store to the current restart era, in parallel with normal operation of the versioned records management system.

Abstract: A log manager causing events to be written to the primary and the secondary computing systems for high availability, and returning threads quickly. Rather than cause every event write to be immediately written to the secondary computing system, the log management component is triggered to write to the secondary computing system upon detecting event write requests of event writes of a particular event type. In response to detecting such an event write, the log management component gathers a group of event writes corresponding to that particular event write, and dispatches the event writes as a group. The particular thread that initiated the particular event write of the particular event type is returned after dispatching the event write group, but before or without confirmation of the particular event write group has been written to the secondary computing system.

Abstract: The recovery of one or more transactions in a computing system in which one or more persistent systems are coupled via a bus to one or more processors. As an example, the persistent system may serve as at least part of the main memory of the computing system. The transaction might implement multi-versioning in which a record is not updated in place. Rather, each record is represented as a sequence of one or more record versions, each version having a valid interval during which the record version is considered to properly represent the record.

Abstract: Caching storage segments (e.g., pages) loaded from a remote storage such that, during recovery, the cached loaded storage segments may be at least partially recovered without reloading the storage segments from the remote storage. During normal operation of a computing system, storage segments are loaded from remote storage into local memory of a computing system. At some point, either due to eviction of the storage segment due to aging out of the storage segment, or due to writing of the storage segment, it is determined to write at least some of the loaded storage segments into local persistent storage. In conjunction with this, the corresponding storage segment is written to a respective storage address of the local persistent storage. Also, a correlation between an identifier of the storage segment and the respective address in the persistent storage is recorded in a persistent data structure.

Abstract: A storage segment server environment that includes storage segment server(s) that cover a contiguous address space of data. The storage segment server has storage segments that are contiguous and that matches the address space of a respective of storage segments in the cloud store service. Thus, a read of the contiguous address spaces of the data from the cloud store service may instead be handled as a single read of the contiguous address space of the storage segment server. Each storage segment server(s) may be assigned to a range within the total contiguous address space of the data. Each storage segment server(s) includes a respective storage segment write component that is configured to perform write operations to storage segments in the respective storage segment server. Each storage segment server(s) also includes a respective cloud storage write component that initiates corresponding write operations to the cloud store service.

Abstract: Adaptive adjusting of the growth of a persistent log. The persistent log has a log record generator that adds log records to the persistent log. In addition, there are multiple log consumers that consume records from the persistent log. The log consumers publish log processing parameters with respect to the persistent log. The log processing parameters are then used to determine an appropriate adjustment in the growth of the log, which adjustments may then be executed. As an example, the log processing parameter may be a log consumption progress, in which case the log generator may be caused to slow down the generation of log records, thereby slowing the growth of the log.

Abstract: Use of a log service computing system to apply a log to storage segments in the context of there being a primary compute system that operates upon data, and secondary compute system(s) that are at the ready to be promoted to the new primary compute system should the old primary compute system fail. Rather than have each of the primary and secondary compute systems maintain their own replicated of the entire data locally, the data is separated in storage segments that are collectively distributed amongst multiple storage segment servers.

Abstract: The bifurcation of a log into a fixed-size log portion and a growable log portion. Log records are first written to the fixed-size log portion. At some point, the older log records within the fixed-size log portion are destaged to the growable log portion. Destaging may involve copying the log records from the fixed-size log portion and tacking the log record onto the end of the growable log portion. This destaging might occur one sub-portion (e.g., one virtual log file) at a time. The more recent log records are within the fixed-size log portion. Accordingly, new writes to the log occur to the fixed-size log portion. Furthermore, a large portion of the reads are of the more recent log records within the fixed-size log portion. Thus, optimizing performance within the fixed-size log portion results in significant improvement in the performance of the overall log.

Abstract: Brokering log records so as to prevent log records that are not yet persisted in a persistent log from being disseminated. The log records may be generated as a primary compute system performs operations. Upon receiving a request for a log record, the broker component determines whether the requested log record has been persisted in a persistent log. If the broker component determines that the log record has been persisted in the persistent log, the broker component responds to the request by causing the requested log record to be provided to the requesting entity (e.g., a secondary compute system). On the other hand, if the log record cannot yet determine that the log record has been persisted in the persistent log, the broker component prevents the log record from being provided to the requesting entity. This prevents data from being inconsistent during recovery.

Abstract: Assembly of a state of user data as of a particular point in time in the context of the data being spread across multiple storage segment servers. Furthermore, the assembly occurs despite the storage segment servers not needing to coordinate a timing of any snapshots. Rather, the storage segment servers perform a snapshot of the portion of the data that it individually stores without having to coordinate with any other storage segment server. The particular point of time may be any arbitrary time, such as a time selected by a user.

Abstract: The use of log marking (otherwise known as “coloring”) of sub-portions of a log that records actions (e.g., data operations) performed by a computing system. The log is composed of multiple sub-portions, such as virtual log files, which are successively added to the log as the log grows. For instance, the sub-portions may be virtual log files of the log. The principles described herein change the use of log marking depending on which sub-portion of the log is being marked. If the computing system fails, and recovery is needed, the recovery process can thus deterministically identify where the last written log record is.

Abstract: The mounting a drive to two or more computing systems. For instance, the drive may be mounted to a first computing system so as to be writable (and potentially readable) by the first computing system. But also, the drive is also mounted to one or more other computing systems so as to be only readable by those one or more computing systems. This allows for multiple computing systems to have access to the drive without risk that the data thereon will become corrupt. In one embodiment, the only user data stored on that drive is a single file of fixed size. Thus, even when user data is written into the fixed-size file, the management data stored (that keeps track of the files) on the drive does not change.

Abstract: The insertion into a data stream of collection records associated with multiple collections of data items in the data stream, for purposes of aiding in subsequent processing of the data stream by collections. Each collection record comprises a collection definition that does not overlap with the collection definition in any of the sequence of collection records including the collection definition of any neighboring collection record in the sequence of collection records. Each collection record also includes a data stream address range extending from the first data item to the last data item in the collection.

Abstract: The subject disclosure relates to a database recovery technique that implements various aspects of transaction logging to optimize database recovery performance. Transactions are logged logically with no reference to physical location, which enables logging to be performed via multiple independent log streams in parallel. A set of log streams can further be adjusted to conform to a local configuration of a mirror or secondary node in order to increase mirroring flexibility. Additionally, individual transactions or groups of transactions are recorded using a single log record, which contains timestamp information to enable database recovery without reference to physical checkpoint files. Further, techniques are provided herein for processing transactions without Write Ahead Logging or hardening of undo information.

Abstract: The use of a skip element when redoing transactions, so as to avoid tracking dependencies between transactions assigned to different threads for parallel processing. When the second thread comes to a second task in the course of redoing a second transaction, if a first task that is mooted by the second task is not already performed, the second thread inserts a skip element associated with the object to be operated upon by the particular task, instead of actually performing the particular task upon the object. When the first thread later comes to the first task in the course of redoing a first transaction, the first thread encounters the skip element associated with the object. Accordingly, instead of performing the dependee task, the first thread skips the dependee task and perhaps removes the skip element. The result is the same regardless of whether the first or second task is redone first.

Abstract: A system that uses a persistent main memory to preserve events that await logging in a persistent store. Each event is written into the persistent main memory so as to be loggable in case of recovery. For instance, the event may be written into a log cache structure, along with other state which identifies that the event is in the particular log cache structure, the location of the event within the particular log cache structure, and the order of the event. To recover, the log in the persistent store is evaluated to identify the end of the stored log. The tail of the log is identified in the persistent main memory by identifying any log cache structures that are after the end of the stored log and which are validly recoverable. The log cache structure contents are then serialized one log cache at a time, earliest first.

Abstract: A versioned records management computing system that uses a restart era in order to promote rapid recovery. A persistent store includes a multi-versioned record collection. The records are also associated with a restart era that corresponds to the era of operation of the computing system after a restart. Upon a recovery, the current restart era changes. An object collection media has an object collection that conforms to an object model such that the object model is used to operate upon the records. The object collection media is operable such that the object collection is durable so as to survive restarts of the system to thereby allow for accelerated recovery.