Abstract: Methods of memory management are described which can accommodate non-maskable failures in pages of physical memory. In an embodiment, when an impending non-maskable failure in a page of memory is identified, a pristine page of physical memory is used to replace the page containing the impending failure and memory mappings are updated to remap virtual pages from the failed page to the pristine page. When a new page of virtual memory is then allocated by a process, the failed page may be reused if the process identifies that it can accommodate failures and the process is provided with location information for impending failures. In another embodiment, a process may expose information on failure-tolerant regions of virtual address space such that a physical page of memory containing failures only in failure-tolerant regions may be used to store the data instead of using a pristine page.

Abstract: Methods of querying a large number of endsystems are described in which metadata is replicated between endsystems. When a query is injected, an available endsystem receives a message relating to the query which identifies a range of endsystems for which that available endsystem is responsible. The available endsystem then generates completeness data for the range of endsystems based on data stored at the endsystem and this completeness data is transmitted to the sender of the message. The methods may be implemented using device-executable instructions which may be stored on device readable media.

Abstract: Resource optimization for online services is described. In one example, objects (such as mailboxes or other data associated with an online service) are assigned to network elements (such as servers) by inferring a relationship graph from log data relating to usage of the online service. The graph has a node for each object, and connections between each pair of objects having data items in common. Each connection has a weight relating to the number of common data items. The graph is partitioned into a set of clusters, such that each cluster has nodes joined by connections with a high weight relative to the weight of connections between nodes in different clusters. The objects are then distributed to the network elements such that objects corresponding to nodes in the same cluster are located on the same network element.

Abstract: Methods of memory management are described which can accommodate non- maskable failures in pages of physical memory. In an embodiment, when an impending non-maskable failure in a page of memory is identified, a pristine page of physical memory is used to replace the page containing the impending failure and memory mappings are updated to remap virtual pages from the failed page to the pristine page. When a new page of virtual memory is then allocated by a process, the failed page may be reused if the process identifies that it can accommodate failures and the process is provided with location information for impending failures. In another embodiment, a process may expose information on failure-tolerant regions of virtual address space such that a physical page of memory containing failures only in failure-tolerant regions may be used to store the data instead of using a pristine page.

Abstract: Improved storage systems which use write off-loading are described. When a request to store some data in a particular storage location is received, if the particular storage location is unavailable, the data is stored in an alternative location. In an embodiment, the particular storage location may be unavailable because it is powered down or because it is overloaded. The data stored in the alternative location may be subsequently recovered and written to the particular storage location once it becomes available.

Abstract: A prediction system may perform capacity planning for one or more resources of a database systems, such as by understanding how different workloads are using the system resources and/or predicting how the performance of the workloads will change when the hardware configuration of the resource is changed and/or when the workload changes. The prediction system may use a detailed, low-level tracing of a live database system running an application workload to monitor the performance of the current database system. In this manner, the current monitoring traces and analysis may be combined with a simulation to predict the workload's performance on a different hardware configuration. More specifically, performance may be indicated as throughput and/or latency, which may be for all transactions, for a particular transaction type, and/or for an individual transaction.

Abstract: A prediction system may perform capacity planning for one or more resources of a database systems, such as by understanding how different workloads are using the system resources and/or predicting how the performance of the workloads will change when the hardware configuration of the resource is changed and/or when the workload changes. The prediction system may use a detailed, low-level tracing of a live database system running an application workload to monitor the performance of the current database system. In this manner, the current monitoring traces and analysis may be combined with a simulation to predict the workload's performance on a different hardware configuration. More specifically, performance may be indicated as throughput and/or latency, which may be for all transactions, for a particular transaction type, and/or for an individual transaction.

Abstract: Improved storage systems which use write off-loading are described. When a request to store some data in a particular storage location is received, if the particular storage location is unavailable, the data is stored in an alternative location. In an embodiment, the particular storage location may be unavailable because it is powered down or because it is overloaded. The data stored in the alternative location may be subsequently recovered and written to the particular storage location once it becomes available.

Abstract: Methods of querying a large number of endsystems are described in which metadata is replicated between endsystems. When a query is injected, an available endsystem receives a message relating to the query which identifies a range of endsystems for which that available endsystem is responsible. The available endsystem then generates completeness data for the range of endsystems based on data stored at the endsystem and this completeness data is transmitted to the sender of the message. The methods may be implemented using device-executable instructions which may be stored on device readable media.

Abstract: A prediction system may perform capacity planning for one or more resources of a database systems, such as by understanding how different workloads are using the system resources and/or predicting how the performance of the workloads will change when the hardware configuration of the resource is changed and/or when the workload changes. The prediction system may use a detailed, low-level tracing of a live database system running an application workload to monitor the performance of the current database system. In this manner, the current monitoring traces and analysis may be combined with a simulation to predict the workload's performance on a different hardware configuration. More specifically, performance may be indicated as throughput and/or latency, which may be for all transactions, for a particular transaction type, and/or for an individual transaction.

Abstract: A data storage system is provided in which a migration plan generates migration plans for reaching a goal configuration from an initial configuration in a predetermined period of time. The migration plan initially makes terminal moves until no further terminal moves can be made. A shunt move is then made based on reducing the contention (the total size of the data stores that need to be moved onto a particular data stores device divided by the amount of excess capacity on the data storage device). The shunt is selected to lead to the lowest contention for the data storage system. Subsequently, the migration plan returns to making terminal moves to develop a migration plan. By determining existence and utilization dependencies of the various moves, independent moves are determined to be implemented in parallel with moves having dependencies. This produces parallelized migration plans which run much faster than sequential migration plans.

Abstract: A data storage system is provided in which a migration plan generates migration plans for reaching a goal configuration from an initial configuration in a predetermined period of time. The migration plan initially makes terminal moves until no further terminal moves can be made. A shunt move is then made based on reducing the contention (the total size of the data stores that need to be moved onto a particular data stores device divided by the amount of excess capacity on the data storage device). The shunt is selected to lead to the lowest contention for the data storage system. Subsequently, the migration plan returns to making terminal moves to develop a migration plan. Further migration plans are provided.