Abstract: Methods, systems and computer program products are provided for transaction log validation in a database transaction log service. A transaction log service may perform egress validation to confirm transaction logs match previously served transaction logs and are free of corruption. A transaction log service may determine whether a transaction log (e.g., in the form of a log block) was previously served based on transaction log (e.g., log block) identity indicating a position in a logical stream of transaction logs (e.g., log blocks). Egress validation may improve log service reliability by providing an added layer of defense against corruption of a transaction log payload.

Abstract: Methods, systems, and computer program products are provided for consolidating transaction log requests and transaction logs in a database transaction log service. A scalable log service may manage log requests and logs to reduce resource consumption, such as memory and I/O. Log requests may be managed by consolidating (e.g., organizing, merging and/or de-duplicating) the log requests. Transaction log requests may be mapped to read requests for transaction log storage devices in less than a one-to-one ratio. Transaction logs may be managed by using the consolidated log requests to consolidate (e.g., and prefetch) transaction logs from multiple cache and/or storage tiers to a log pool cache. Log requests may be served from the log pool cache.

Abstract: Methods, systems and computer program products are provided for asynchronous processing of transaction log requests in a database transaction log service. A scalable log service may continuously process log requests. Log request processing may be paused and resumed without reducing log service processing capabilities. Log service threads executing a request that pauses may proceed to process other tasks in the same request or tasks for other new or partially processed requests. Any of multiple log service threads may resume processing of a paused request. Requests may be paused by preserving an execution state and deferring the request from a work queue to a deferral queue. Resumed requests may shift from a deferral queue to a work queue following a wait point.

Abstract: Methods, systems, and computer program products are provided for consolidating transaction log requests and transaction logs in a database transaction log service. A scalable log service may manage log requests and logs to reduce resource consumption, such as memory and I/O. Log requests may be managed by consolidating (e.g., organizing, merging and/or de-duplicating) the log requests. Transaction log requests may be mapped to read requests for transaction log storage devices in less than a one-to-one ratio. Transaction logs may be managed by using the consolidated log requests to consolidate (e.g., and prefetch) transaction logs from multiple cache and/or storage tiers to a log pool cache. Log requests may be served from the log pool cache.

Abstract: Methods, systems and computer program products are provided for transaction log validation in a database transaction log service. A transaction log service may perform egress validation to confirm transaction logs match previously served transaction logs and are free of corruption. A transaction log service may determine whether a transaction log (e.g., in the form of a log block) was previously served based on transaction log (e.g., log block) identity indicating a position in a logical stream of transaction logs (e.g., log blocks). Egress validation may improve log service reliability by providing an added layer of defense against corruption of a transaction log payload.

Abstract: A distributed database system providing data and space management methodology. In one embodiment a system for allocating resources amongst a plurality of database servers sharing resources comprises: an input module for receiving user input for allocating resources shared amongst a plurality of database servers and establishing thresholds for utilization of resources at each database server; a monitoring module at each database server for monitoring local resource utilization at each database server and sending a request for balancing resources to a second database server when local resource utilization reaches an established threshold at a first database server; and a load balancing module for receiving the request for balancing resources and determining whether to adjust resources allocated to the first database server and the second database server in response to the request.

Abstract: A database system providing methodology for execution of functions in queries requesting data from markup language documents is described. In one embodiment, for example, a method is described for executing a function in a query requesting data from a markup language document stored in a database, the markup language document organized hierarchically into nodes, the method comprises steps of: receiving the query requesting data from the markup language document, the query including a function to be executed during execution of the query; determining nodes of the markup language document satisfying the query; deriving values of the nodes of the markup language document; and executing the function with the derived values and the nodes of the markup language document.

Abstract: A distributed database system providing data and space management methodology. In a distributed database system having an allocation page for tracking allocation and deallocation of data pages including a deallocation bit for tracking data page deallocation, a method for determining when to clear the deallocation bit during recovery comprises: during runtime operation, generating log records recording allocation and deallocation of data pages, setting the deallocation bit during deallocation of a data page by a transaction, and clearing the deallocation bit after the transaction has committed; during recovery following a failure, identifying a particular log record corresponding to the deallocation bit set during deallocation of a given data page; determining whether the deallocation of the given data page recorded in the particular log record committed prior to the failure; and if the deallocation of the given data page committed prior to the failure, clearing the deallocation bit.

Abstract: A distributed database system providing data and space management methodology. In a distributed system consisting a plurality of servers sharing access to data pages, a method for setting a timestamp indicating when modifications to data pages are made in a non-regressing fashion includes steps of: maintaining a local sequence at each server and an allocation page timestamp on an allocation page; upon allocation of a data page, applying a timestamp to the data page which is greater than the allocation page timestamp; upon modification of a given data page at a particular server, setting the timestamp on the given data page to the greater of the local sequence at the particular server and an existing timestamp already on the given data page; and when data pages are deallocated, adjusting the allocation page timestamp to a value greater than the timestamp value on any deallocated data page.

Abstract: A distributed database system providing data and space management methodology. In one embodiment, for example, a method for transferring a data structure in cache at a first database server to a second database server in a distributed database system comprises steps of: determining a first database server having a data structure in cache in response to a request for the data structure from a second database server, the request including a request for a lock on the data structure; providing the request for the data structure to the first database server; in response, sending the data structure from the first database server to the second database server; and receiving and using the data structure at the second database server without waiting for the lock request to be explicitly granted.

Abstract: A database system providing self-tuned parallel database recovery is described. In one embodiment, for example, in a database system, a method is described for performing recovery operations using an optimal number of recovery threads, the method comprises steps of: (a) spawning an initial recovery thread to perform recovery operations; (b) measuring I/O (input/output) performance with the initial recovery thread; (c) spawning a subsequent recovery thread to perform recovery operations; (d) measuring I/O performance with the subsequent recovery thread; and (e) as long as I/O performance does not degrade beyond a preselected percentage, repeating steps (c) and (d) for spawning a desired number of additional recovery threads. In another embodiment, the database system auto-tunes the cache during performance of database recovery operations to optimize the performance of recovery operations.

Abstract: A distributed database system providing data and space management methodology. In a distributed database system having an allocation page for tracking allocation and deallocation of data pages including a deallocation bit for tracking data page deallocation, a method for determining when to clear the deallocation bit during recovery comprises: during runtime operation, generating log records recording allocation and deallocation of data pages, setting the deallocation bit during deallocation of a data page by a transaction, and clearing the deallocation bit after the transaction has committed; during recovery following a failure, identifying a particular log record corresponding to the deallocation bit set during deallocation of a given data page; determining whether the deallocation of the given data page recorded in the particular log record committed prior to the failure; and if the deallocation of the given data page committed prior to the failure, clearing the deallocation bit.

Abstract: A distributed database system providing data and space management methodology. In a distributed system comprising a plurality of servers sharing access to data pages, a method for setting a timestamp indicating when modifications to data pages are made in a non-regressing fashion comprises steps of: maintaining a local sequence at each server and an allocation page timestamp on an allocation page; upon allocation of a data page, applying a timestamp to the data page which is greater than the allocation page timestamp; upon modification of a given data page at a particular server, setting the timestamp on the given data page to the greater of the local sequence at the particular server and an existing timestamp already on the given data page; and when data pages are deallocated, adjusting the allocation page timestamp to a value greater than the timestamp value on any deallocated data page.

Abstract: A distributed database system providing data and space management methodology is described. In one embodiment, for example, a method is described for detection of deadlocks in a distributed database system comprising a plurality of servers sharing access to data, the method comprises steps of: in response to a task at a first server waiting to obtain a lock regulating access to data, sending a search request including lock information to a second server holding the lock; at the second server, examining dependencies of the task holding the lock based on the lock information received with the search request and local lock information; if the task holding the lock is waiting to obtain a lock held by an additional server, forwarding the search request and local lock information to the additional server; repeating the examining and forwarding steps until all dependencies are examined; and responding to the search request indicating whether a deadlock was detected and providing lock information found.

Abstract: A distributed database system providing data and space management methodology. In one embodiment a system for allocating resources amongst a plurality of database servers sharing resources comprises: an input module for receiving user input for allocating resources shared amongst a plurality of database servers and establishing thresholds for utilization of resources at each database server; a monitoring module at each database server for monitoring local resource utilization at each database server and sending a request for balancing resources to a second database server when local resource utilization reaches an established threshold at a first database server; and a load balancing module for receiving the request for balancing resources and determining whether to adjust resources allocated to the first database server and the second database server in response to the request.

Abstract: A distributed database system providing data and space management methodology is described. In one embodiment, for example, a method is described for detection of deadlocks in a distributed database system comprising a plurality of servers sharing access to data, the method comprises steps of: in response to a task at a first server waiting to obtain a lock regulating access to data, sending a search request including lock information to a second server holding the lock; at the second server, examining dependencies of the task holding the lock based on the lock information received with the search request and local lock information; if the task holding the lock is waiting to obtain a lock held by an additional server, forwarding the search request and local lock information to the additional server; repeating the examining and forwarding steps until all dependencies are examined; and responding to the search request indicating whether a deadlock was detected and providing lock information found.

Abstract: A database system providing methodology for execution of functions in queries requesting data from markup language documents is described. In one embodiment, for example, a method is described for executing a function in a query requesting data from a markup language document stored in a database, the markup language document organized hierarchically into nodes, the method comprises steps of: receiving the query requesting data from the markup language document, the query including a function to be executed during execution of the query; determining nodes of the markup language document satisfying the query; deriving values of the nodes of the markup language document; and executing the function with the derived values and the nodes of the markup language document.

Abstract: A database system providing self-tuned parallel database recovery is described. In one embodiment, for example, in a database system, a method is described for performing recovery operations using an optimal number of recovery threads, the method comprises steps of: (a) spawning an initial recovery thread to perform recovery operations; (b) measuring I/O (input/output) performance with the initial recovery thread; (c) spawning a subsequent recovery thread to perform recovery operations; (d) measuring I/O performance with the subsequent recovery thread; and (e) as long as I/O performance does not degrade beyond a preselected percentage, repeating steps (c) and (d) for spawning a desired number of additional recovery threads. In another embodiment, the database system auto-tunes the cache during performance of database recovery operations to optimize the performance of recovery operations.

Abstract: A system providing methods enabling data in Extensible Markup Language (“XML”) format to be extracted, transformed and stored in a database, file system or main memory is described. The extraction and transformation process is generalized and can be used on various types of XML data, enabling XML data to be stored and queried using standard database query methodologies. The system includes parse-time functionality to transform XML documents into a structure having an interface that enables efficient access to the underlying data. The system also includes query execution-time functionality providing greater efficiency by bringing only the relevant portions of transformed XML data into memory in response to a query. The system parses and translates queries into a structure that can be executed without the need to write custom application-specific navigation code to search XML data. The system also enables original XML documents (or portions thereof) to be recomposed when required.

Abstract: A system providing methods enabling data in Extensible Markup Language (“XML”) format to be extracted, transformed and stored in a database, file system or main memory is described. The extraction and transformation process is generalized and can be used on various types of XML data, enabling XML data to be stored and queried using standard database query methodologies. The system includes parse-time functionality to transform XML documents into a structure having an interface that enables efficient access to the underlying data. The system also includes query execution-time functionality providing greater efficiency by bringing only the relevant portions of transformed XML data into memory in response to a query. The system parses and translates queries into a structure that can be executed without the need to write custom application-specific navigation code to search XML data. The system also enables original XML documents (or portions thereof) to be recomposed when required.