Abstract: A database management system receives a request to create an index for a virtual view of a database table. In response to the request, the database management system identifies relationships between the columns of the virtual view and columns of the database table. The database management system generates an index on the database table, where columns are included in the generated index based on the identified relationships and the definition of the virtual view. Queries of the virtual view are assisted by the generated index of the underlying table.

Abstract: Commitments against various resources can be dynamically adjusted for customers in a shared-resource environment. A customer can provision a data volume with a committed rate of Input/Output Operations Per Second (IOPS) and pay only for that commitment (plus any overage), for example, as well as the amount of storage requested. The customer can subsequently adjust the committed rate of IOPS by submitting an appropriate request, or the rate can be adjusted automatically based on any of a number of criteria. Data volumes for the customer can be migrated, split, or combined in order to provide the adjusted rate. The interaction of the customer with the data volume does not need to change, independent of adjustments in rate or changes in the data volume, other than the rate at which requests are processed.

Abstract: Techniques are described for managing access of executing programs to non-local block data storage. In some situations, a block data storage service uses multiple server storage systems to reliably store network-accessible block data storage volumes that may be used by programs executing on other physical computing systems. A group of multiple server block data storage systems that store block data volumes may in some situations be co-located at a data center, and programs that use volumes stored there may execute on other physical computing systems at that data center. If a program using a volume becomes unavailable, another program (e.g., another copy of the same program) may in some situations obtain access to and continue to use the same volume, such as in an automatic manner in some such situations.

Abstract: Customers of shared resources in a multi-tenant environment can have token buckets allocated that have an associated depth and fill rate, with each token enabling the customer to obtain an amount of work from a shared resource. A resource management system can monitor one or more system or output metrics, and can adjust a global fill rate based at least in part upon values of the monitored metrics. Such an approach can provide a fair distribution of work among the customers, while ensuring that the metrics stay within acceptable ranges and there are no drastic changes in performance levels of the system. The fill rate can update dynamically with changes in the monitored parameters, such that the system can float near an equilibrium point. Commitments for specific minimum service levels also can be met.

Abstract: A database management system stores an entry in a journal. The journal, upon storage of the entry, comprises an ordered plurality of leaf nodes and a hierarchy of interior nodes based upon the leaf nodes. A node in the ordered plurality of nodes comprises a hash value computed based on the hash of a corresponding entry and a hash of a prior leaf node. A cryptographic proof of the entry comprises successive application of a hash operator to one or more hashes of the leaf nodes.

Abstract: A database management system stores an entry in a journal. The journal, upon storage of the entry, comprises a leaf node with a hash value based at least in part on the entry, and a hierarchy of interior nodes based at least in part upon the leaf node. In response to a request to delete the entry, the entry is deleted but the hash value is retained. A cryptographic proof of a second entry stored in the journal is based at least in part on the retained hash value.

Abstract: A database management system stores an entry in a journal. The journal, upon storage of the entry, comprising a hierarchy of nodes. A node in the hierarchy comprises a hash value computed by application of a symmetric hash operator to hash values of first and second child nodes. The symmetric hash operator generates equivalent output irrespective of the order of the operands. A cryptographic proof of the entry comprises successive application of the symmetric hash operator to a list of hashes from the hierarchy.

Abstract: A database management system stores an entry in a journal. Upon storage of the entry, the journal comprises a plurality of threaded leaf nodes and a hierarchy of interior nodes comprising hash values computed from the threaded leaf nodes. A first set of hash values is provided to prove that the entry is stored in a first version of the journal. A second set is provided to prove that the entries stored in the first version of the journal are unmodified and stored in a second version of the journal.

Abstract: A snapshot manager examines a set of entries of a journal of a multi-data-store storage system. Each entry is formatted in a data-store-independent transaction language and various entries contain a respective indication of a committed change to a data object or to a journal schema. Corresponding to various data objects that are materialized at least at a data store of the storage system, respective compact state change representations are generated based on the set of journal entries affecting the data objects. A given compact state change representation, also formatted in the transaction language, may represent cumulative effects of several changes. A snapshot of the journal, comprising the compact state representations, is stored.

Abstract: A request to copy at least a portion of a first network that includes a first set of devices is received, the request including one or more filtering criteria, with at least one of the one or more filtering criteria specifying a tag assigned to a device of the first set of devices. At least the portion of the first network is copied by causing the system to create, according to the one or more filtering criteria, a second network by causing a second set of devices to be configured to be duplicative of devices of the first set of devices that are assigned the tag specified by the filtering criteria.

Abstract: Techniques are described for managing access of executing programs to non-local block data storage. In some situations, a block data storage service uses multiple server storage systems to reliably store copies of network-accessible block data storage volumes that may be used by programs executing on other physical computing systems, and snapshot copies of some volumes may also be stored (e.g., on remote archival storage systems). A group of multiple server block data storage systems that store block data volumes may in some situations be co-located at a data center, and programs that use volumes stored there may execute on other computing systems at that data center, while the archival storage systems may be located outside the data center. The snapshot copies of volumes may be used in various ways, including to allow users to obtain their own copies of other users' volumes (e.g., for a fee).

Abstract: A selective data restoration technique is disclosed. A request to recover data in a dataset is received along with an identifier of a particular portion of the data to be recovered. The request may specify a time that indicates a desired recovery state (e.g., recover the values as they were at the time). Change data is determined for the specified time. For example, data stream filters generate recovery data by applying filters (e.g., keys) to a time window batch of the stream data. The stream data may correspond to changes to a data set, and the recovery data may correspond to only a portion of the dataset (e.g., a portion of a table). The system performs a recovery action, based on the recovery data, using the recovery data to automatically restore data in a data store, or providing the recovery date to the requesting entity, for example.

Abstract: Commitments against various resources can be dynamically adjusted for customers in a shared-resource environment. A customer can provision a data volume with a committed rate of Input/Output Operations Per Second (IOPS) and pay only for that commitment (plus any overage), for example, as well as the amount of storage requested. The customer can subsequently adjust the committed rate of IOPS by submitting an appropriate request, or the rate can be adjusted automatically based on any of a number of criteria. Data volumes for the customer can be migrated, split, or combined in order to provide the adjusted rate. The interaction of the customer with the data volume does not need to change, independent of adjustments in rate or changes in the data volume, other than the rate at which requests are processed.

Abstract: Techniques are described for managing access of executing programs to non-local block data storage. In some situations, a block data storage service uses multiple server storage systems to reliably store network-accessible block data storage volumes that may be used by programs executing on other physical computing systems. A group of multiple server block data storage systems that store block data volumes may in some situations be co-located at a data center, and programs that use volumes stored there may execute on other physical computing systems at that data center. If a program using a volume becomes unavailable, another program (e.g., another copy of the same program) may in some situations obtain access to and continue to use the same volume, such as in an automatic manner in some such situations.

Abstract: Various systems and methods to efficiently store backups of partitions of a database are disclosed. An indication to take a current snapshot of a partition of the database is received. A log size and operation type statistics are determined for a portion of the change log for the partition. A snapshot type is determined for the partition based at least in part on the log size and the operation type statistics, where the snapshot type specifies a log-based snapshot or a full snapshot. The snapshot is stored as metadata indicating a point in the change log if the determined snapshot type is log-based snapshot or stored as complete data of the partition if the determined snapshot type is full snapshot.

Abstract: A set of virtualized computing services may include multiple types of virtualized data store differentiated by characteristics such as latency, throughput, durability and cost. A sequence of captures of a data set from one data store to another may be scheduled to achieve a variety of virtualized computing service user and provider goals such as lowering a probability of data loss, lowering costs, and computing resource load leveling. Data set captures may be scheduled according to policies specifying fixed and flexible schedules and conditions including flexible scheduling windows, target capture frequencies, probability of loss targets and/or cost targets. Capture lifetimes may also be managed with capture retention policies, which may specify fixed and flexible lifetimes and conditions including cost targets. Such data set capture policies may be specified with a Web-based administrative interface to a control plane of the virtualized computing services.

Abstract: An opportunistic throttling approach can be used for customers of shared resources in a multi-tenant environment. Each customer can have a respective token bucket with a guaranteed fill rate. When a request is received for an amount of work to be performed by a resource, the corresponding number of tokens are obtained from, or charged against, a global token bucket. If the global bucket has enough tokens, and if the customer has not exceeded a maximum work rate or other such metric, the customer can charge less than the full number of tokens against the customer's token bucket, in order to reduce the number of tokens that need to be taken from the customer bucket. Such an approach can enable the customer to do more work and enable the customer's bucket to fill more quickly as fewer tokens are charged against the customer bucket for the same amount of work.

Abstract: Techniques are described for managing access of executing programs to non-local block data storage. In some situations, a block data storage service uses multiple server storage systems to reliably store copies of network-accessible block data storage volumes that may be used by programs executing on other physical computing systems, and snapshot copies of some volumes may also be stored (e.g., on remote archival storage systems). A group of multiple server block data storage systems that store block data volumes may in some situations be co-located at a data center, and programs that use volumes stored there may execute on other computing systems at that data center, while the archival storage systems may be located outside the data center. The snapshot copies of volumes may be used in various ways, including to allow users to obtain their own copies of other users' volumes (e.g., for a fee).

Abstract: Disclosed are systems and methods for generating consistent backups. A central coordinator informs each node storing a partition of the time to perform a backup. At the designated time, each node blocks updates for a corresponding time interval measured by its local clock. Each node performs the backup operation according to its own local clock. Consistent backups may be generated in spite of clock skew between the local clocks as long as the time interval is at least as long as a maximum local clock skew (among the nodes performing the backup). In some systems the maximum local clock skew may be reduced, by a round-trip update latency from a client, for example.

Abstract: A client-side component of a journal-based multi-data-store database identifies an application schema corresponding to a first application whose data is stored at the database. The application schema includes a writes section indicating a data object to which writes are to be issued from the first application. A journal schema defining one or more data objects for which a concurrency-control protocol is implemented by a journal manager of the database is obtained. Before a transaction request of the application is submitted to the journal manager, the client-side component verifies that the application schema is compatible with the journal schema.