Abstract: A scaling manager manages deques that track groups of preinitialized instances used to scale respective groups of active compute instances. Various techniques for deque management include a technique where a total instance quantity is preconfigured for the total number of instances assigned to both the group and the deque of preinitialized instances. As the size of the group grows for scale-ups, the size of the deque may go down. For example, the deque is not replenished when the group scales, but does expand when the group scales down. The total instance quantity may be bounded, in some examples, and an additional “buffer amount” of preinitialized instances may be implemented to provide a safety margin for burst scaling, which can be further enhanced by transferring instances between data structures of different groups of instances in some cases.

Abstract: A scaling manager manages deques that track groups of preinitialized instances used to scale respective groups of active compute instances. Various techniques for deque management include a rate-based technique that uses a historical scale-up rate for a particular group and adjusts the size of the deque of preinitialized instances for that group based on the monitored scale-up rate and based on an instance preinitialization time for instances for that group. A total instance quantity may be bounded, in some examples, and an additional “buffer amount” of preinitialized instances may be implemented to provide a safety margin for burst scaling, which can be further enhanced by transferring instances between data structures of different groups of instances in some cases.