Abstract: Managing database recovery time. A method includes receiving user input specifying a target recovery time for a database. The method further includes determining an amount of time to read a data page of the database from persistent storage. The method further includes determining an amount of time to process a log record of the database to apply changes specified in the log record to a data page. The method further includes determining a number of dirty pages that presently would be read in recovery if a database failure occurred. The method further includes determining a number of log records that would be processed in recovery if a database failure occurred. The method further includes adjusting at least one of the number of dirty pages that presently would be read in recovery or the number of log records that would be processed in recovery to meet the specified target recovery time.

Abstract: Embodiments are directed to implementing a transitional redo phase to allow pre-redo phase data access and to determining which data pages including unprocessed transactions are to be processed during a transitional redo phase. In an embodiment, a computer system initiates an analysis phase of a database recovery that scans a database transaction log and builds a dirty page table that includes pages corresponding to unprocessed data transactions. The computer system determines from the dirty page table which pages are to be processed during the transitional redo phase, before the final redo phase. The computer system processes the determined pages during the transitional redo phase, before the final redo phase is reached and, upon completion of the system transaction undo phase, makes available those pages that were processed during the transitional redo phase, as well as those database pages that are not scheduled for processing during the final redo phase.

Abstract: Consolidating a set of tenant log streams from separate user databases into a consolidated log stream. The method includes receiving a plurality of tenant log streams from separate user data bases. The method further includes recording the plurality of tenant log streams as a consolidated log stream. The method further includes maintaining metadata about the consolidated log stream to map log records from the plurality of tenant log streams to their location in the consolidated log stream.

Abstract: Embodiments are directed to implementing a transitional redo phase to allow pre-redo phase data access and to determining which data pages including unprocessed transactions are to be processed during a transitional redo phase. In an embodiment, a computer system initiates an analysis phase of a database recovery that scans a database transaction log and builds a dirty page table that includes pages corresponding to unprocessed data transactions. The computer system determines from the dirty page table which pages are to be processed during the transitional redo phase, before the final redo phase. The computer system processes the determined pages during the transitional redo phase, before the final redo phase is reached and, upon completion of the system transaction undo phase, makes available those pages that were processed during the transitional redo phase, as well as those database pages that are not scheduled for processing during the final redo phase.

Abstract: Embodiments are directed to implementing a transitional redo phase to allow pre-redo phase data access and to determining which data pages including unprocessed transactions are to be processed during a transitional redo phase. In an embodiment, a computer system initiates an analysis phase of a database recovery that scans a database transaction log and builds a dirty page table that includes pages corresponding to unprocessed data transactions. The computer system determines from the dirty page table which pages are to be processed during the transitional redo phase, before the final redo phase. The computer system processes the determined pages during the transitional redo phase, before the final redo phase is reached and, upon completion of the system transaction undo phase, makes available those pages that were processed during the transitional redo phase, as well as those database pages that are not scheduled for processing during the final redo phase.

Abstract: Consolidating a set of tenant log streams from separate user databases into a consolidated log stream. The method includes receiving a plurality of tenant log streams from separate user data bases. The method further includes recording the plurality of tenant log streams as a consolidated log stream. The method further includes maintaining metadata about the consolidated log stream to map log records from the plurality of tenant log streams to their location in the consolidated log stream.

Abstract: A distributed database transaction coordination manager's outcome store may be unavailable to resolve in-doubt transactions locally, e.g., after an unclean detach, during an outcomes store restore, or during a logical database restore. To facilitate transactional consistency, AGEs and values based on them are used. Some code avoids reformatting any closed physical portion for reuse based on the segment's most recently received minimum of persisted AGEs. Some code commits any in-doubt transactions that appear on an aggregate list and aborts any not on the aggregate list. Some code writes each segment's transaction commit records to persistent storage until the segment's persisted-AGE is not less than a common-restore-AGE, and some code resolves any in-doubt transactions based on a harvested outcomes list. Outcomes may reside in a log or a persisted transactions table. A restore checkpoint may include harvested outcomes. Segments may reside on cloud nodes and/or in a shared nothing database system.

Abstract: A database is queried as of any wall-clock time within a retention period, via undo that uses database snapshots and a list of page level modifications. The snapshot is user-identified, automatically generated, or extracted from a backup. The list is maintained in a transaction log by persisting page content before a page is re-used, persisting deleted rows before they are moved, persisting compensation log record undo information, and/or logging a full page. To rewind an entire database, the undo scans the transaction log in reverse LSN order and undoes all page modifications. Undo reverses reallocated pages, table truncation, and/or table deletion, as well as page-level modifications of a schema, metadata values, and/or system tables. An as-of query is handled using as-of page(s) from a sparse page file. If the sparse page file does not already contain the responsive page(s), they are created and added to it.

Abstract: A database is queried as of any wall-clock time within a retention period, via undo that uses database snapshots and a list of page level modifications. The snapshot is user-identified, automatically generated, or extracted from a backup. The list is maintained in a transaction log by persisting page content before a page is re-used, persisting deleted rows before they are moved, persisting compensation log record undo information, and/or logging a full page. To rewind an entire database, the undo scans the transaction log in reverse LSN order and undoes all page modifications. Undo reverses reallocated pages, table truncation, and/or table deletion, as well as page-level modifications of a schema, metadata values, and/or system tables. An as-of query is handled using as-of page(s) from a sparse page file. If the sparse page file does not already contain the responsive page(s), they are created and added to it.

Abstract: A distributed database transaction coordination manager's outcome store may be unavailable to resolve in-doubt transactions locally, e.g., after an unclean detach, during an outcomes store restore, or during a logical database restore. To facilitate transactional consistency, AGEs and values based on them are used. Some code avoids reformatting any closed physical portion for reuse based on the segment's most recently received minimum of persisted AGEs. Some code commits any in-doubt transactions that appear on an aggregate list and aborts any not on the aggregate list. Some code writes each segment's transaction commit records to persistent storage until the segment's persisted-AGE is not less than a common-restore-AGE, and some code resolves any in-doubt transactions based on a harvested outcomes list. Outcomes may reside in a log or a persisted transactions table. A restore checkpoint may include harvested outcomes. Segments may reside on cloud nodes and/or in a shared nothing database system.

Abstract: Managing database recovery time. A method includes receiving user input specifying a target recovery time for a database. The method further includes determining an amount of time to read a data page of the database from persistent storage. The method further includes determining an amount of time to process a log record of the database to apply changes specified in the log record to a data page. The method further includes determining a number of dirty pages that presently would be read in recovery if a database failure occurred. The method further includes determining a number of log records that would be processed in recovery if a database failure occurred. The method further includes adjusting at least one of the number of dirty pages that presently would be read in recovery or the number of log records that would be processed in recovery to meet the specified target recovery time.

Abstract: Embodiments are directed to implementing a transitional redo phase to allow pre-redo phase data access and to determining which data pages including unprocessed transactions are to be processed during a transitional redo phase. In an embodiment, a computer system initiates an analysis phase of a database recovery that scans a database transaction log and builds a dirty page table that includes pages corresponding to unprocessed data transactions. The computer system determines from the dirty page table which pages are to be processed during the transitional redo phase, before the final redo phase. The computer system processes the determined pages during the transitional redo phase, before the final redo phase is reached and, upon completion of the system transaction undo phase, makes available those pages that were processed during the transitional redo phase, as well as those database pages that are not scheduled for processing during the final redo phase.

Abstract: Managing database recovery time. A method includes receiving user input specifying a target recovery time for a database. The method further includes determining an amount of time to read a data page of the database from persistent storage. The method further includes determining an amount of time to process a log record of the database to apply changes specified in the log record to a data page. The method further includes determining a number of dirty pages that presently would be read in recovery if a database failure occurred. The method further includes determining a number of log records that would be processed in recovery if a database failure occurred. The method further includes adjusting at least one of the number of dirty pages that presently would be read in recovery or the number of log records that would be processed in recovery to meet the specified target recovery time.

Abstract: To ensure that logs representative of data changes are durably written, localized storage media cache is flushed to force a write to stable storage. Log sequence number tracking is leveraged to determine if log data in a localized storage media cache is likely to have not been written to durable storage. When this condition is likely, a FLUSH_CACHE command is issued to a storage media controller to force a durable write. This allows recovery of data changes to a transactional system even when its associated storage media does not provide write-ordering guarantees. Moreover, flushing of the storage media cache can be accomplished at a frequency that provides an upper bound on a maximum time between a transaction commit and data being made durable.

Abstract: A system and method for serialization and/or de-serialization of file system item(s) and associated entity(ies)is provided. A file system “item” comprises a core class which can include property(ies). Through serialization, a consistent copy of the item and associated entity(ies), such as fragment(s), link(s) with other item(s) and/or extension(s),if any, can be captured. The serialization system includes an identification component that identities entity(ies) associated with an item and a serialization component that serializes the item and associated entity(ies). The serialization component can further serialize a header that includes information associated with the item and associated entity(ies). The header can facilitate random access to the item and associated entity(ies). The serialization system can expose application program interface(s) (API's) that facilitate the copying, moving and/or transfer of an item and its associated entity(ies) from one location to another location.

Abstract: The subject invention relates to systems and/or methods that perform an authoritative and/or a non-authoritative restore of items in a data store. Backups are provided that include data associated with items and metadata related to the items that can include a history of operations and previous synchronizations to enable replications to converge after restoration. The operations that can be performed on the items include, for example, create, delete, update, move, etc. The items in the backups can be restored along with the metadata, which can be utilized to enable synchronization partners to converge based on a desired type of restoration.

Abstract: The subject invention relates to systems and methods that enable pausing and/or resuming a backup of file system items. The items are backed up in an order based on a unique property associated with each of the items. When the backup is paused, the unique property associated with a last backed up item prior to the pause is stored. Additionally, a timestamp associated with a snapshot from which the backup was taken can be stored with the unique property. Upon being resumed, the backup can continue from the paused location via utilizing the stored unique property. The backup can be resumed either employing an original snapshot associated with a time prior to the pause or a second snapshot related to a subsequent time. The subject invention can perform an initial backup of file system items and/or an incremental backup of such items. Additionally, the subject invention can support backing up a file system at disparate levels of granularity such as, for example, an item level.

Abstract: To ensure that logs representative of data changes are durably written, localized storage media cache is flushed to force a write to stable storage. Log sequence number tracking is leveraged to determine if log data in a localized storage media cache is likely to have not been written to durable storage. When this condition is likely, a FLUSH_CACHE command is issued to a storage media controller to force a durable write. This allows recovery of data changes to a transactional system even when its associated storage media does not provide write-ordering guarantees. Moreover, flushing of the storage media cache can be accomplished at a frequency that provides an upper bound on a maximum time between a transaction commit and data being made durable.