Abstract: Various methods and systems for implementing request scheduling and processing in a multi-tenant distributed computing environment are provided. Requests to utilize system resources in the distributed computing environment are stored in account queues corresponding to tenant accounts. If storing a request in an account queue would exceed a throttling threshold such as a limit on the number of requests stored per account, the request is dropped to a throttling queue. A scheduler prioritizes processing requests stored in the processing queue before processing requests stored in the account queues. The account queues can be drained using dominant resource scheduling. In some embodiments, a request is not picked up from an account queue if processing the request would exceed a predefined hard limit on system resource utilization for the corresponding tenant account. In some embodiments, the hard limit is defined as a percentage of threads the system has to process requests.

Abstract: Various methods and systems for implementing request scheduling and processing in a multi-tenant distributed computing environment are provided. Requests to utilize system resources in the distributed computing environment are stored in account queues corresponding to tenant accounts. If storing a request in an account queue would exceed a throttling threshold such as a limit on the number of requests stored per account, the request is dropped to a throttling queue. A scheduler prioritizes processing requests stored in the processing queue before processing requests stored in the account queues. The account queues can be drained using dominant resource scheduling. In some embodiments, a request is not picked up from an account queue if processing the request would exceed a predefined hard limit on system resource utilization for the corresponding tenant account. In some embodiments, the hard limit is defined as a percentage of threads the system has to process requests.

Abstract: A distributed storage system includes table controllers collectively storing partitions of a logical table having multiple rows. For each of the partitions, a corresponding table controller stores a subset of the rows of the logical table across individual pages. A management controller updates a schema of the logical table from a first to a second schema definition by, for each of the partitions, sending an update command indicating the second schema definition to the corresponding table controller. The first table controller corresponds to a first partition that includes a first subset of the rows. The first table controller, prior to receiving the update command, includes an identifier of the first schema definition within each page of data for the first subset. Subsequent to receiving the update command, the first table controller, when writing each new page of data for the first subset, includes an identifier of the second schema definition.

Abstract: Described herein is a system method for point in time blob restore. In response to receipt of a request to restore a particular blob range to a particular point in time, read and/or write operations on the particular blob range requested to be restored are blocked. Blob(s) in the particular blob range which have changed since the particular point in time are determined based, at least in part, upon information stored in a change log and/or a deleted blob store. A determination is made as to whether the requested restore operation can be performed, for example, based upon a restoration policy. When it is determined that the requested restore operation can be performed, blob(s) determined to have changed since the particular point in time based, at least in part, upon information stored in the change log and/or the deleted blob store are restored.

Abstract: A distributed storage system includes table controllers collectively storing partitions of a logical table having multiple rows. For each of the partitions, a corresponding table controller stores a subset of the rows of the logical table across individual pages. A management controller updates a schema of the logical table from a first to a second schema definition by, for each of the partitions, sending an update command indicating the second schema definition to the corresponding table controller. The first table controller corresponds to a first partition that includes a first subset of the rows. The first table controller, prior to receiving the update command, includes an identifier of the first schema definition within each page of data for the first subset. Subsequent to receiving the update command, the first table controller, when writing each new page of data for the first subset, includes an identifier of the second schema definition.

Abstract: Techniques are described that enable a user to upgrade a stack of resources by providing a template that reflects the desired end state of the stack. Upon receiving a new template, the system automatically detects changes that should be performed and determines the order in which the changes should be performed. The system can also detect if the desired changes to the stack are a result of direct modifications; because parameters to the resources have changed; or the changes are indirectly caused by other dependency/attribute changing. Additionally, the system determines whether the changes require creating new resources or if the changes can be made to the resources live. In case of replacement of resources, the system will first create the new resource, move that new resource into the stack and remove the old resource(s). In case of failures, the system ensures that the stack rolls back to the initial state.

Abstract: A snapshot of data from a table associated with a particular user may be generated. Tree data structures that are distributed across multiple computer systems may be accessed. Each of the tree structures may include data associated with one or more users. At least one tree data structure of the tree data structures that includes data associated with the particular user of the one or more users may be identified. The at least one tree data structure may then be filtered. Filtering may comprise identifying only data that is associated with the particular user. A snapshot of the data associated with the particular user may be generated. Generating the snapshot of the data associated with the particular user comprises generating a data structure that is configured to map to each data page of the at least one tree data structure that includes data associated with the particular user.

Abstract: Described herein is a system method for point in time blob restore. In response to receipt of a request to restore a particular blob range to a particular point in time, read and/or write operations on the particular blob range requested to be restored are blocked. Blob(s) in the particular blob range which have changed since the particular point in time are determined based, at least in part, upon information stored in a change log and/or a deleted blob store. A determination is made as to whether the requested restore operation can be performed, for example, based upon a restoration policy. When it is determined that the requested restore operation can be performed, blob(s) determined to have changed since the particular point in time based, at least in part, upon information stored in the change log and/or the deleted blob store are restored.

Abstract: Various methods and systems for implementing request scheduling and processing in a multi-tenant distributed computing environment are provided. Requests to utilize system resources in the distributed computing environment are stored in account queues corresponding to tenant accounts. If storing a request in an account queue would exceed a throttling threshold such as a limit on the number of requests stored per account, the request is dropped to a throttling queue. A scheduler prioritizes processing requests stored in the processing queue before processing requests stored in the account queues. The account queues can be drained using dominant resource scheduling. In some embodiments, a request is not picked up from an account queue if processing the request would exceed a predefined hard limit on system resource utilization for the corresponding tenant account. In some embodiments, the hard limit is defined as a percentage of threads the system has to process requests.

Abstract: Various methods and systems for implementing request scheduling and processing in a multi-tenant distributed computing environment are provided. Requests to utilize system resources in the distributed computing environment are stored in account queues corresponding to tenant accounts. If storing a request in an account queue would exceed a throttling threshold such as a limit on the number of requests stored per account, the request is dropped to a throttling queue. A scheduler prioritizes processing requests stored in the processing queue before processing requests stored in the account queues. The account queues can be drained using dominant resource scheduling. In some embodiments, a request is not picked up from an account queue if processing the request would exceed a predefined hard limit on system resource utilization for the corresponding tenant account. In some embodiments, the hard limit is defined as a percentage of threads the system has to process requests.

Abstract: A distributed storage system includes table controllers collectively storing partitions of a logical table having multiple rows. For each of the partitions, a corresponding table controller stores a subset of the rows of the logical table across individual pages. A management controller updates a schema of the logical table from a first to a second schema definition by, for each of the partitions, sending an update command indicating the second schema definition to the corresponding table controller. The first table controller corresponds to a first partition that includes a first subset of the rows. The first table controller, prior to receiving the update command, includes an identifier of the first schema definition within each page of data for the first subset. Subsequent to receiving the update command, the first table controller, when writing each new page of data for the first subset, includes an identifier of the second schema definition.

Abstract: Methods for dequeue optimizations in queues are performed by systems and apparatuses. The methods optimize dequeue operations using aggregation of expired messages enqueued in a queue and conditional iteration over enqueued messages based on the aggregation to service dequeue commands. Queues utilize page hierarchies such as root pages, index pages, and data pages. The aggregation of expired messages for pages in the queue determines the latest expired time for messages for a given page, and these latest expired times are stored in their respective pages, including data pages, index pages, and root pages, for use in the conditional iteration. The conditional iteration bypasses pages for which a latest expired time for all messages is prior to the current time when servicing dequeue requests for the queue.

Abstract: Techniques are described that enable a user to upgrade a stack of resources by providing a template that reflects the desired end state of the stack. Upon receiving a new template, the system automatically detects changes that should be performed and determines the order in which the changes should be performed. The system can also detect if the desired changes to the stack are a result of direct modifications; because parameters to the resources have changed; or the changes are indirectly caused by other dependency/attribute changing. Additionally, the system determines whether the changes require creating new resources or if the changes can be made to the resources live. In case of replacement of resources, the system will first create the new resource, move that new resource into the stack and remove the old resource(s). In case of failures, the system ensures that the stack rolls back to the initial state.

Abstract: A resource stack managed by a resource stack provider is created based on a resource stack template that integrates a custom resource from a second provider into the resource stack using a notification system with the second provider. For example, a customer may create a template that defines a resource stack that comprises resources available from the resource stack provider and one or more custom resources provided by a second provider. When a resource stack is created, resources available from the resource stack provider may be provisioned. Custom resources may be initialized by notifying the provider of the custom resource of the requested integration of the custom resource with the resource stack and requested configuration details. The custom resource provider may respond with an indication of successful integration when the custom resource has been successfully initialized. After initializing the resources, the resource stack may be enabled for use.

Abstract: Various methods and systems for implementing request scheduling and processing in a multi-tenant distributed computing environment are provided. Requests to utilize system resources in the distributed computing environment are stored in account queues corresponding to tenant accounts. If storing a request in an account queue would exceed a throttling threshold such as a limit on the number of requests stored per account, the request is dropped to a throttling queue. A scheduler prioritizes processing requests stored in the processing queue before processing requests stored in the account queues. The account queues can be drained using dominant resource scheduling. In some embodiments, a request is not picked up from an account queue if processing the request would exceed a predefined hard limit on system resource utilization for the corresponding tenant account. In some embodiments, the hard limit is defined as a percentage of threads the system has to process requests.

Abstract: Methods for dequeue optimizations in queues are performed by systems and apparatuses. The methods optimize dequeue operations using aggregation of expired messages enqueued in a queue and conditional iteration over enqueued messages based on the aggregation to service dequeue commands. Queues utilize page hierarchies such as root pages, index pages, and data pages. The aggregation of expired messages for pages in the queue determines the latest expired time for messages for a given page, and these latest expired times are stored in their respective pages, including data pages, index pages, and root pages, for use in the conditional iteration. The conditional iteration bypasses pages for which a latest expired time for all messages is prior to the current time when servicing dequeue requests for the queue.

Abstract: A snapshot of data from a table associated with a particular user may be generated. Tree data structures that are distributed across multiple computer systems may be accessed. Each of the tree structures may include data associated with one or more users. At least one tree data structure of the tree data structures that includes data associated with the particular user of the one or more users may be identified. The at least one tree data structure may then be filtered. Filtering may comprise identifying only data that is associated with the particular user. A snapshot of the data associated with the particular user may be generated. Generating the snapshot of the data associated with the particular user comprises generating a data structure that is configured to map to each data page of the at least one tree data structure that includes data associated with the particular user.

Abstract: A resource stack managed by a resource stack provider is created based on a resource stack template that integrates a custom resource from a second provider into the resource stack using a notification system with the second provider. For example, a customer may create a template that defines a resource stack that comprises resources available from the resource stack provider and one or more custom resources provided by a second provider. When a resource stack is created, resources available from the resource stack provider may be provisioned. Custom resources may be initialized by notifying the provider of the custom resource of the requested integration of the custom resource with the resource stack and requested configuration details. The custom resource provider may respond with an indication of successful integration when the custom resource has been successfully initialized. After initializing the resources, the resource stack may be enabled for use.

Abstract: A resource stack managed by a resource stack provider is created based on a resource stack template that integrates a custom resource from a second provider into the resource stack using a notification system with the second provider. For example, a customer may create a template that defines a resource stack that comprises resources available from the resource stack provider and one or more custom resources provided by a second provider. When a resource stack is created, resources available from the resource stack provider may be provisioned. Custom resources may be initialized by notifying the provider of the custom resource of the requested integration of the custom resource with the resource stack and requested configuration details. The custom resource provider may respond with an indication of successful integration when the custom resource has been successfully initialized. After initializing the resources, the resource stack may be enabled for use.

Abstract: A resource stack managed by a resource stack provider is created based on a resource stack template that integrates a custom resource from a second provider into the resource stack using a notification system with the second provider. For example, a customer may create a template that defines a resource stack that comprises resources available from the resource stack provider and one or more custom resources provided by a second provider. When a resource stack is created, resources available from the resource stack provider may be provisioned. Custom resources may be initialized by notifying the provider of the custom resource of the requested integration of the custom resource with the resource stack and requested configuration details. The custom resource provider may respond with an indication of successful integration when the custom resource has been successfully initialized. After initializing the resources, the resource stack may be enabled for use.