Abstract: A fleet of databases may comprise subsets of databases sharing a common schema for replication, horizontal partitioning and other applications. Events related to potential deviations from a desired schema may be monitored. Automated means may be employed to detect variations from a desired schema. An intermediate representation of database schema may be formed and used to compare database schemas. Subclasses and allowed versions of desired schemas may be defined. Security attributes related to schema inspection and modification may be mapped to enable selected operations on schema.

Abstract: A snapshot analysis system analyzes a plurality of data snapshots taken in connection with data stored on a block device allocated by a data storage system. The snapshot analysis system may include an ingestor capable of initially detecting new snapshots and adding a root node for the snapshots. The system may include a block device analyzer that analyzes each snapshot to determine its contents, the relationship within data structures extant within the snapshot, and the snapshot's relationship to other snapshots and/or that of other block devices. The system may also include a clustering analyzer capable of determining whether snapshots are associated with multipart block devices, such as LVM or MD RAID devices. The system may further include a block device emulator that exposes data associated with a given snapshot as an addressable block device without necessitating retrieval or exposure of the full block device to which the snapshot is associated.

Abstract: Methods and apparatus for profile-guided preloading for virtualized resources are described. A block-level storage volume whose contents are to be populated via data transfers from a repository service is programmatically attached to a compute instance. An indication of data transfers from the repository to a block storage service implementing the volume is obtained, corresponding to a particular phase of program execution at the compute instance. A storage profile is generated, based at least in part on the indication of data transfers. The storage profile is subsequently used to pre-load data from the repository service on behalf of other compute instances.

Abstract: A system includes a storage volume configured to store a data set in a plurality of data blocks, a data store configured to store a plurality of captures of the data set in a plurality of data chunks, and file retrieval logic. The data set includes a file stored in a first data block of the plurality of data blocks. The file retrieval logic is configured to identify a first data chunk of the plurality of data chunks in which the first data block as captured in a first capture is stored in the data store, retrieve the first data chunk from the data store, and read the first data block as captured in the first capture from the first data chunk.

Abstract: A system and method for maintaining data integrity during data transformation operations. The system and method include obtaining a message from a set of queues, obtaining, from a first data store, a data object indicated by the message, and generating, at a first host, a set of error detection codes corresponding to a transformation of the data object according to a transformation scheme. The system and method further include, transforming, at a second host different from the first host, the data object according to the transformation scheme into the transformation of the data object, verifying the transformation against the set of error detection codes, and storing the transformation in a second data store.

Abstract: A system includes a storage volume configured to store a data set in a plurality of data blocks, a data store configured to store a plurality of captures of the data set in a plurality of data chunks, and file retrieval logic. The data set includes a file stored in a data block of the plurality of data blocks. The plurality of captures includes the file captured at different points in time. The file retrieval logic is configured to identify the plurality of data chunks in which the data block as captured in the plurality of captures is stored in the data store, retrieve the plurality of data chunks from the data store, and read the data block as captured in the plurality of captures from the plurality of data chunks to produce a plurality of file versions.

Abstract: Systems and methods for performing diagnostic tests within a multi-tenant environment are described. Diagnostics are performed on one or more components, such as host computing devices. The one or more components send resulting diagnostic information to an ingester which collects the diagnostic information. The ingester then sends the diagnostic information to a database. A generator is able to query the information in the database and create metrics. Based at least in part on metrics, an alarm may be triggered.

Abstract: Systems and methods for performing diagnostic tests within a multi-tenant environment are described. Diagnostics are performed on one or more components, such as host computing devices. The one or more components send resulting diagnostic information to an ingester which collects the diagnostic information. The ingester then sends the diagnostic information to a database. A generator is able to query the information in the database and create metrics. Based at least in part on metrics, an alarm may be triggered.

Abstract: A snapshot analysis system analyzes a plurality of data snapshots taken in connection with data stored on a block device allocated by a data storage system. The snapshot analysis system may include an ingestor capable of initially detecting new snapshots and adding a root node for the snapshots. The system may include a block device analyzer that analyzes each snapshot to determine its contents, the relationship within data structures extant within the snapshot, and the snapshot's relationship to other snapshots and/or that of other block devices. The system may also include a clustering analyzer capable of determining whether snapshots are associated with multipart block devices, such as LVM or MD RAID devices. The system may further include a block device emulator that exposes data associated with a given snapshot as an addressable block device without necessitating retrieval or exposure of the full block device to which the snapshot is associated.

Abstract: A snapshot analysis system analyzes a plurality of data snapshots taken in connection with data stored on a block device allocated by a data storage system. The snapshot analysis system may include an ingestor capable of initially detecting new snapshots and adding a root node for the snapshots. The system may include a block device analyzer that analyzes each snapshot to determine its contents, the relationship within data structures extant within the snapshot, and the snapshot's relationship to other snapshots and/or that of other block devices. The system may also include a clustering analyzer capable of determining whether snapshots are associated with multipart block devices, such as LVM or MD-RAID devices. The system may further include a block device emulator that exposes data associated with a given snapshot as an addressable block device without necessitating retrieval or exposure of the full block device to which the snapshot is associated.

Abstract: A technology is described for determining a launch workflow for launching multiple computing instances on a physical host in a computing service environment using a predicted launch time. An example method may include receiving a launch plan to launch a plurality of computing instances on a physical host within a computing service environment. A first launch workflow and a second launch workflow may then be determined for the launch plan where the first launch workflow and the second launch workflow may specify different sequences of operations performed to launch the computing instance on the physical host. Predicted launch times for the first launch workflow and the second launch workflow may be obtained and the predicted launch times may be compared to determine a launch workflow used in executing the launch plan.

Abstract: Systems and methods for performing diagnostic tests within a multi-tenant environment are described. Diagnostics are performed on one or more components, such as host computing devices. The one or more components send resulting diagnostic information to an ingester which collects the diagnostic information. The ingester then sends the diagnostic information to a database. A generator is able to query the information in the database and create metrics. Based at least in part on metrics, an alarm may be triggered.

Abstract: A snapshot analysis system analyzes a plurality of data snapshots taken in connection with data stored on a block device allocated by a data storage system. The snapshot analysis system may include an ingestor capable of initially detecting new snapshots and adding a root node for the snapshots. The system may include a block device analyzer that analyzes each snapshot to determine its contents, the relationship within data structures extant within the snapshot, and the snapshot's relationship to other snapshots and/or that of other block devices. The system may also include a clustering analyzer capable of determining whether snapshots are associated with multipart block devices, such as LVM or MD RAID devices. The system may further include a block device emulator that exposes data associated with a given snapshot as an addressable block device without necessitating retrieval or exposure of the full block device to which the snapshot is associated.

Abstract: A technology is described for managing computing capacity within a computing service environment. An example method may include determining a state of a computing service environment having a group of physical hosts that execute computing instances and determining a predicted demand for computing instance launches on the group of physical hosts. A co-location of the computing instance launches on the group of physical hosts may then be determined by applying the predicted demand for computing instance launches to the state of the computing service environment. A predicted launch time for the computing instance launches may then be determined using the co-location of the computing instance launches as an input feature. An additional physical host may then be added to the group of physical hosts to decrease the predicted launch time.

Abstract: A technology is described for predicting a launch time for a computing instance. An example method may include receiving a request for a predicted launch time to launch a computing instance on a physical host within a computing service environment. Data associated with launch features of a computing instance may then be obtained, where the launch features may be determined to have an impact on a launch time of the computing instance on a physical host within a computing service environment. The launch features of the computing instance may then be input to a machine learning model that outputs the predicted launch time for launching the computing instance within the computing service environment.

Abstract: A snapshot analysis system analyzes a plurality of data snapshots taken in connection with data stored on a block device allocated by a data storage system. The snapshot analysis system may include an ingestor capable of initially detecting new snapshots and adding a root node for the snapshots. The system may include a block device analyzer that analyzes each snapshot to determine its contents, the relationship within data structures extant within the snapshot, and the snapshot's relationship to other snapshots and/or that of other block devices. The system may also include a clustering analyzer capable of determining whether snapshots are associated with multipart block devices, such as LVM or MD RAID devices. The system may further include a block device emulator that exposes data associated with a given snapshot as an addressable block device without necessitating retrieval or exposure of the full block device to which the snapshot is associated.

Abstract: A technology is described for providing a recommended launch configuration for a computing instance based on a predicted launch time. An example method may include receiving a launch plan to launch a computing instance on a physical host within a computing service environment, where the launch plan includes a launch configuration. Upon receiving the launch plan, a predicted launch time may be determined for the computing instance based on the launch configuration. The launch configuration may then be analyzed to identify changes to the launch configuration that may result in an improved predicted launch time as compared to the predicted launch time. A recommended change may then be provided for the launch configuration as a result of a determination that the change to the launch configuration results in the improved predicted launch time.

Abstract: A snapshot analysis system analyzes a plurality of data snapshots taken in connection with data stored on a block device allocated by a data storage system. The snapshot analysis system may include an ingestor capable of initially detecting new snapshots and adding a root node for the snapshots. The system may include a block device analyzer that analyzes each snapshot to determine its contents, the relationship within data structures extant within the snapshot, and the snapshot's relationship to other snapshots and/or that of other block devices. The system may also include a clustering analyzer capable of determining whether snapshots are associated with multipart block devices, such as LVM or MD RAID devices. The system may further include a block device emulator that exposes data associated with a given snapshot as an addressable block device without necessitating retrieval or exposure of the full block device to which the snapshot is associated.

Abstract: Technology is described for determining computing instance placement. A request to launch a computing instance in a computing service environment may be received. Estimated launch times for the computing instance to launch on individual physical hosts in a group of physical hosts may be identified. A physical host in the group of physical hosts that can provide placement of the computing instance may be selected based in part on the estimated launch times for the computing instance and additional factors related to placement of the computing instance.

Abstract: A system and method for using a dead letter queue to regulate host computing systems performing transformation or migration operations. The system and method including reading a message from a queue, the message including one or more references to a set of data objects, with the set of data objects including one or more data objects selected for a transformation operation according to a transformation scheme. The system and method further including performing a mitigation action upon determination that the message is one of a number of unsuccessfully processed messages in the queue that are associated with the computer system, when the number of unsuccessfully processed messages exceed a threshold.