Abstract: A lock manager system is provided that includes multiple distinct mechanisms for assigning resources to masters. Specifically, the lock manager system includes (1) one or more assignment mechanisms that do not take into account the affinity between nodes and objects (“affinity-independent assignment mechanisms”), and (3) one or more assignment mechanisms that do take into account the affinity between nodes and objects (“affinity-based assignment mechanisms”). At any given time, the assignment of any given resource to a master is governed by only one of the multiple assignment mechanisms employed by the system. Various techniques are described for determining which assignment mechanism to use with which resources.

Abstract: Various techniques are described for improving the performance of a shared-nothing database system in which at least two of the nodes that are running the shared-nothing database system have shared access to a disk. Specifically, techniques are provided for changing the ownership of data in a shared-nothing database without changing the location of the data on persistent storage. Because the persistent storage location for the data is not changed during a transfer of ownership of the data, ownership can be transferred more freely and with less of a performance penalty than would otherwise be incurred by a physical relocation of the data. Various techniques are also described for providing fast run-time reassignment of ownership. Because the reassignment can be performed during run-time, the shared-nothing system does not have to be taken offline to perform the reassignment.

Abstract: Techniques are described for storing portions of files on different storage devices based on the devices' respective storage class. For example, different extents from a large file within a system, application, or database may be stored on different storage devices. The storage devices may be part of a single storage system that includes some storage devices that have different performance capabilities. The decision of where, within the storage classes, to store data may be based on the access patterns of the data. Access patterns regarding file extents are tracked using respective access bits, which are set when a memory block associated with an extent is accessed via an I/O operation. In one embodiment, an access bitmap is modified to account for cached blocks of file extents.

Abstract: Various techniques are described for improving the performance of a multiple node system by allocating, in two or more nodes of the system, partitions of a shared cache. A mapping is established between the data items managed by the system, and the various partitions of the shared cache. When a node requires a data item, the node first determines which partition of the shared cache corresponds to the required data item. If the data item does not currently reside in the corresponding partition, the data item is loaded into the corresponding partition even if the partition does not reside on the same node that requires the data item. The node then reads the data item from the corresponding partition of the shared cache.

Abstract: Described herein are techniques for synchronizing access to resources in a multi node system by establishing a resource as a read-mostly resource based on one or more criteria, and allowing a node to acquire a shared lock, not granted by the master, on the read-mostly resource where no anti-lock exists for the resource.

Abstract: A method and mechanism is disclosed for implementing storage and compression in a computer system. Each granular portion of a file can be individually stored in either a compressed storage unit or an uncompressed storage unit. The storage units can be allocated apriori or on an as-needed basis.

Abstract: A method and apparatus are provided for improving the performance associated with transferring a data item and obtaining a lock in a multi-node system by interpreting the block transfer message as a lock grant message. Typically when a Holder of a data item transfers a data item (e.g. block) to a Requestor of that data item, the Holder will down-convert its lock for that data item and send a message to the Master of this data item indicating that a down-convert has been performed. Subsequently, the Master sends a lock grant message to the Requestor of the data item to inform the Requestor that it has been granted a lock for the data item. By interpreting the block transfer message as a lock grant message, the down-convert message and the lock grant message can be eliminated, which results in improved performance.

Abstract: Techniques are provided for managing caches in a system with multiple caches that may contain different copies of the same data item. Specifically, techniques are provided for coordinating the write-to-disk operations performed on such data items to ensure that older versions of the data item are not written over newer versions, and to reduce the amount of processing required to recover after a failure. Various approaches are provided in which a master is used to coordinate with the multiple caches to cause a data item to be written to persistent storage. Techniques are also provided for managing checkpoints associated with the caches, where the checkpoints are used to determine the position at which to begin processing recovery logs in the event of a failure.

Abstract: Disclosed are improved methods, systems, and mediums for handling reporting transactions in database systems. In some embodiments, database snapshots are used to carry out reporting transactions on a failover node concurrently with execution of non-reporting transactions on a primary node.

Abstract: A system and method for detecting a failure of a first process is disclosed. According to one aspect, a first connection is established between a first process and a second process. A monitoring mechanism monitors the status of the first process to detect whether the first process has failed. In response to determining that the first process has failed, the monitoring mechanism causes an out-of-band message to be sent to the second process.

Abstract: Techniques are provided for managing caches in a system with multiple caches that may contain different copies of the same data item. Specifically, techniques are provided for coordinating the write-to-disk operations performed on such data items to ensure that older versions of the data item are not written over newer versions, and to reduce the amount of processing required to recover after a failure. Various approaches are provided in which a master is used to coordinate with the multiple caches to cause a data item to be written to persistent storage. Techniques are also provided for managing checkpoints associated with the caches, where the checkpoints are used to determine the position at which to begin processing recovery logs in the event of a failure.

Abstract: A method and mechanism is disclosed for implementing storage and compression in a computer system. Each granular portion of a file can be individually stored in either a compressed storage unit or an uncompressed storage unit. The storage units can be allocated apriori or on an as-needed basis.

Abstract: A programmatic interface to allow external functions to be registered and called in parallel from within a database management system is described for coordinating a computation at multiple nodes. In one embodiment, each node executes a process configured for starting a program to perform the computation in response to a command received from a database system. In response to receiving a query at the database system, multiple commands are transmitted to the processes for concurrently performing the computation at each said corresponding process. Results are received from each of the processes and execution of the statement is completed based on the results received.

Abstract: Techniques are provided for automatically tracking errors encountered by a software system. A database server is entrusted with the tasks of automatically recording errors and managing error trace information in a separate database storage space. Techniques are also provided for automatically determining, by a software system, whether any diagnostic features should be turned on in response to errors occurring in the system by inspecting logged information about errors that have occurred in the system. After turning on the selected diagnostic features, the software system can automatically log additional error information upon subsequent occurrences of the errors. Techniques are also provided for automatically circumventing errors encountered by a software system. The software system automatically determines that the code path executed to perform a particular operation contains a software error.

Abstract: Techniques for optimistically obtaining a resource that does not currently reside in a cache associated with a node include sending a request for permission to access the resource. Before receiving a response to the request, an operation is initiated to retrieve the resource; then the response to the request is received. After receiving the response, it is determined whether the operation results in a correct version of the resource. A returned resource is received in response to the operation. If it is determined that the operation results in the correct version, then, after the returned resource is received, the returned resource is placed in the cache associated with the first node. The techniques of the current invention not only perform an optimistic read but also determine whether the results of the optimistic read are valid. When optimistic read results are valid, latency is reduced in retrieving resources.

Abstract: Groups of services are created based on one or both of the following: (a) what is accessed by each service, and (b) a likelihood of conflicts between accesses (e.g., a database object read operation or a database object write operation) performed by each service and accesses performed by each other service. Groups of services, which may be the groups created or subsets of those groups, are assigned to run on particular servers based on the processing power requirements of each group and the processing power available to each server. In the grouping process, accesses can be biased by giving more weight to object write operations than to object read operations, and to access operations on an index or dictionary than to access operations on a table.

Abstract: Techniques for performing window-by-window remastering are described, in which the mechanism used by nodes to determine the resource-to-window mapping is separated from the mechanism used by the nodes to access resource information. For example, all nodes of a multiple-node system may use the same hash function to determine which resources are involved in a remastering window. However, the resource indexes employed by the nodes to access resource information need not use that same hash function. For example, in nodes where the resource indexes are hash indexes, different nodes may use different hash functions. Further, some or all of the nodes may use mechanisms other than hash indexes to access the resource information.

Abstract: Remastering of shared resources among nodes in a cluster, while minimizing the time that such resources are unavailable for access, comprises dividing the shared resources being remastered into multiple “windows” and freezing and remastering one window at a time. The remastering windows are defined over one or more buckets associated with a hash table of resources. In clusters with unequally sized hash tables, an enhancement to the fundamental windowing approach utilizes a functionally specific hashing algorithm and a hash table size constraint to ensure that the set of all resources that hash to a given bucket in a hash table of a given size will hash precisely to the same bucket, plus a set of higher buckets which are well defined, in a node that has a larger hash table.

Abstract: A method and apparatus for mirroring data between a plurality of sites is provided. At a first site of the plurality of sites, a record is maintained that identifies which changes made to one or more data blocks that are stored at the first site have had associated redo information replicated to the other sites of the plurality of sites. A priority value associated with a transaction that is to be performed at the first site is determined. The transaction specifies a modification to a data block. Thereafter, if the priority value indicates that the transaction should not be lost if the first site becomes inoperable, then the transaction is committed only after the record indicates that all other changes that have updated the data block at the first site have had their respective redo information replicated to the other sites of the plurality of sites.

Abstract: A programmatic interface to allow external functions to be registered and called in parallel from within a database management system is described for coordinating a computation at multiple nodes. In one embodiment, each node executes a process configured for starting a program to perform the computation in response to a command received from a database system. In response to receiving a query at the database system, multiple commands are transmitted to the processes for concurrently performing the computation at each said corresponding process. Results are received from each of the processes and execution of the statement is completed based on the results received.