Abstract: According to one aspect of the invention, a read-write instance of a multiple-instance database system receives a database statement for execution. The multiple-instance database system comprises a database shared by a plurality read-mostly instances that request only read access to a plurality of user database objects in the database and a plurality of read-write instances that request both read access and write access to the plurality of user database objects. The read-write instance selects one or more read-mostly instances, in the plurality of read-mostly instances, to perform one or more operations that require no write access to the plurality of user data objects.

Abstract: A method and apparatus for managing shared resources in a clustered database management system is provided. In an embodiment, multiple master nodes exist in a database management system. A master node receives a lock request from a second node. The lock request is a request for a lock on a shared resource. The master node grants the lock request to the second node. While the second node holds the lock, the second node causes the master node to modify the shared resource.

Abstract: A method and apparatus for recovery a cluster database is provided. Database recovery is divided among a plurality of surviving database server instances in the cluster. A surviving instance is responsible for recovering data blocks to which the surviving instance is assigned. One form of block-to-instance assignment may be based on mastership of the blocks. If a data block is locked by a surviving instance at the time of failure, then no recovery of that data block may be necessary. Else, if a copy of a data block that is to be recovered is stored on a surviving node in the cluster, then one or more redo records are applied to that copy (if necessary). A redo record that corresponds to that data block might not need to be applied to the copy if the redo record reflects changes (to the data block) that are already reflected in the copy.

Abstract: Processes, computer-readable media, and machines are disclosed for reducing a likelihood that active functional components fail in a computing system. An active monitoring component receives metrics associated with different active functional components of a computing system. The different active functional components contribute to different functionalities of the system. Based at least in part on the metrics associated with a particular active functional component, the active monitoring component determines that the particular active functional component has reached a likelihood of failure but has not failed. In response to determining that the particular active functional component has reached the likelihood of failure but has not failed, the active monitoring component causes a set of actions that are predicted to reduce the likelihood of failure.

Abstract: A conflict resolution mechanism collects statistical data regarding how much time certain common actions or waits take. For example, the mechanism may collect statistics on disk I/O for each disk device. Statistics may include the average access time, for example. Such statistics may be collected over a sliding window of time. With the statistical data that the mechanism collects, the mechanism can make a more intelligent judgment regarding whether a process is in a “hanging” condition. For example, if the average I/O to a disk is 10 seconds for the past hour, and if a process is doing disk I/O to that disk for 5 seconds, then the mechanism will not yet determine that the process is hanging. In order to determine whether the process is hanging, the mechanism looks at the average time and the longest time for the particular actions that the process is performing.

Abstract: Processes in a cluster maintain a historical record of problem events as those events occur in the cluster. The record may describe (a) attributes of each event and (b) the resolution action that was performed to resolve each event. Whenever a new event occurs, a process determines whether any entries in the record reflect occurrences of an event with attributes like those of the new event. If a “matching” entry exists, then the process increments that entry's counter. If the historical record indicates that similar events have previously occurred more than a specified number of times, then the process may select and perform a resolution action that differs from the resolution action that is indicated in the entry. Additionally, patterns in the attributes of a recurring problem may be used to predict when the problem is likely to recur. Preventative actions may be taken to avoid recurrence of the problem.

Abstract: A method and apparatus for managing locks in a database system is provided. A master node grants a lock on a first resource and a group of resources that includes the first resource to a first requester node. The requester node receives a mapping corresponding to the group of resources that may indicate that a lock already exists for a second resource in the group. If the requester node desires a lock on a resource located in the group, the requester node grants itself the lock without notifying the master node. A second requester node requests a lock for a particular resource in the group of resources. The first requester node grants the lock on the particular resource and updates the mapping to indicate that a different node holds a lock for the particular resource.

Abstract: Techniques are provided for managing caches in a system with multiple caches that may contain different copies of the same data item. Specifically, techniques are provided for coordinating the write-to-disk operations performed on such data items to ensure that older versions of the data item are not written over newer versions, and to reduce the amount of processing required to recover after a failure. Various approaches are provided in which a master is used to coordinate with the multiple caches to cause a data item to be written to persistent storage. Techniques are also provided for transferring data items and locks associated with the data items from one node to another.

Abstract: Techniques are provided for managing caches in a system with multiple caches that may contain different copies of the same data item. Specifically, techniques are provided for coordinating the write-to-disk operations performed on such data items to ensure that older versions of the data item are not written over newer versions, and to reduce the amount of processing required to recover after a failure. Various approaches are provided in which a master is used to coordinate with the multiple caches to cause a data item to be written to persistent storage. Techniques are also provided for managing checkpoints associated with the caches, where the checkpoints are used to determine the position at which to begin processing recovery logs in the event of a failure.

Abstract: Techniques are provided for managing caches in a system with multiple caches that may contain different copies of the same data item. Specifically, techniques are provided for coordinating the write-to-disk operations performed on such data items to ensure that older versions of the data item are not written over newer versions, and to reduce the amount of processing required to recover after a failure. Various approaches are provided in which a master is used to coordinate with the multiple caches to cause a data item to be written to persistent storage. Techniques are also provided for managing checkpoints associated with the caches, where the checkpoints are used to determine the position at which to begin processing recovery logs in the event of a failure.

Abstract: A method and apparatus for managing shared resources in a clustered database management system is provided. In an embodiment, multiple master nodes exist in a database management system. A master node receives a lock request from a second node. The lock request is a request for a lock on a shared resource. The master node grants the lock request to the second node. While the second node holds the lock, the second node causes the master node to modify the shared resource.

Abstract: A conflict resolution mechanism collects statistical data regarding how much time certain common actions or waits take. For example, the mechanism may collect statistics on disk I/O for each disk device. Statistics may include the average access time, for example. Such statistics may be collected over a sliding window of time. With the statistical data that the mechanism collects, the mechanism can make a more intelligent judgment regarding whether a process is in a “hanging” condition. For example, if the average I/O to a disk is 10 seconds for the past hour, and if a process is doing disk I/O to that disk for 5 seconds, then the mechanism will not yet determine that the process is hanging. In order to determine whether the process is hanging, the mechanism looks at the average time and the longest time for the particular actions that the process is performing.

Abstract: Techniques are provided for managing caches in a system with multiple caches that may contain different copies of the same data item. Specifically, techniques are provided for coordinating the write-to-disk operations performed on such data items to ensure that older versions of the data item are not written over newer versions, and to reduce the amount of processing required to recover after a failure. Various approaches are provided in which a master is used to coordinate with the multiple caches to cause a data item to be written to persistent storage. Techniques are also provided for managing checkpoints associated with the caches, where the checkpoints are used to determine the position at which to begin processing recovery logs in the event of a failure.

Abstract: Processes in a cluster maintain a historical record of problem events as those events occur in the cluster. The record may describe (a) attributes of each event and (b) the resolution action that was performed to resolve each event. Whenever a new event occurs, a process determines whether any entries in the record reflect occurrences of an event with attributes like those of the new event. If a “matching” entry exists, then the process increments that entry's counter. If the historical record indicates that similar events have previously occurred more than a specified number of times, then the process may select and perform a resolution action that differs from the resolution action that is indicated in the entry. Additionally, patterns in the attributes of a recurring problem may be used to predict when the problem is likely to recur. Preventative actions may be taken to avoid recurrence of the problem.

Abstract: A method and apparatus for recovery a cluster database is provided. Database recovery is divided among a plurality of surviving database server instances in the cluster. A surviving instance is responsible for recovering data blocks to which the surviving instance is assigned. One form of block-to-instance assignment may be based on mastership of the blocks. If a data block is locked by a surviving instance at the time of failure, then no recovery of that data block may be necessary. Else, if a copy of a data block that is to be recovered is stored on a surviving node in the cluster, then one or more redo records are applied to that copy (if necessary). A redo record that corresponds to that data block might not need to be applied to the copy if the redo record reflects changes (to the data block) that are already reflected in the copy.

Abstract: Techniques are provided for responding to the termination of a node by selecting another node, and assigning to the selected node the affinity relationships that existed between the terminated node and one or more objects. The resources that belong to the objects involved in the affinity relationships are remastered to the selected node. The selected node then performs recovery of the resources that had been opened by the terminated node and/or serves as a failover node to execute the transactions that had been executing on the terminated node.

Abstract: In a multi-node database system, a list of data blocks to perform recovery operations on in the event of failure of a given node is dynamically maintained on another node. List is available when the given node fails, and allows the database to be made available more quickly.

Abstract: Techniques are provided for managing caches in a system with multiple caches that may contain different copies of the same data item. Specifically, techniques are provided for coordinating the write-to-disk operations performed on such data items to ensure that older versions of the data item are not written over newer versions, and to reduce the amount of processing required to recover after a failure. Various approaches are provided in which a master is used to coordinate with the multiple caches to cause a data item to be written to persistent storage. Techniques are also provided for managing checkpoints associated with the caches, where the checkpoints are used to determine the position at which to begin processing recovery logs in the event of a failure.

Abstract: Techniques are provided for responding to the termination of a node executing one or more transactions by selecting another node to complete the transactions, and assigning to the selected node the affinity relationships that existed between the terminated node and the objects being accessed by said transactions.

Abstract: Techniques for monitoring resources of a computer system are provided. A monitoring process collects and reports utilization data for one or more resources of a computer system, such as CPU, memory, disk I/O, and network I/O. Instead of reporting just an average of the collected data over a period of time (e.g., 10 seconds), the monitoring process at least reports individually collected resource utilization values. If one or more of the utilization values exceed specified thresholds for the respective resources, then an alert may be generated. In one approach, the monitoring process is made a real-time priority process in the computer system to ensure that the memory used by the monitoring process is not swapped out of memory. Also, being a real-time priority process ensures that the monitoring process obtains a CPU in order collect resource utilization data even when the computer system is in a starvation mode.