Abstract: A database service that receives and manages time series data into a set of generational data tables. The database service generates a set of generational data tables configured to receive a subset of time series data. Each generational data table is created with an initial set of access parameters that facilitates the receipt of incoming time series data. Thereafter, the database service creates additional generational data tables to receive the continuous received time series data and then modifies the access parameters of the previously created data table, such as for read only rights.

Abstract: Techniques for predicting the availability of a resource are described. An exemplary method includes obtaining capacity data indicating an amount of capacity available in a cloud provider network to satisfy the request; generating, using a machine learning model that has been trained based at least in part on an output of an automated historical hindsight learner that is an integer linear program, an approval prediction, wherein the approval prediction indicates that the request is to be approved based on one or more launch parameters of the request and the capacity data; receiving information from a downstream component that controls the resource that the approval prediction is incorrect; and evaluating the incorrect approval prediction using a hindsight learner and predictor explainer.

Abstract: Techniques for managing dynamically scalable virtualized compute instances within a provider network are described. A dynamically scalable instance has a baseline performance level and a maximum performance level associated with a computing resource, such as a processor, a memory, a network interface, etc. In response to receive a request to launch a dynamically scalable instance, one or more services select a computer system to host the requested instance from a pool of instance-hosting computer systems of a provider network. The instance-hosting computer systems include a monitoring agent that reports computing resource usage data to the one or more services. The one or more services monitor the resource usage of the instance-hosting computer systems and migrate instances so that the dynamically scalable instances can operate up to the maximum level of performance.

Abstract: Techniques for managing compute resource usage by virtualized compute instances within a provider network are described. A computer system connected to a provider network hosts one or more compute instances. An agent associated with the computer system obtains per-compute instance resource usage information of computer system resources such as processors, memory, and network interfaces. The agent sends the usage information to a usage management service of the provider network. The usage management service generates usage limits based on the usage information from the agent and on usage targets and sends the usage limits to the computer system. The computer system limits the resource usage of the compute instance based on the usage limits.

Abstract: A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.

Abstract: A partitioning policy, comprising an indication of an initial mapping of data records of a stream to a plurality of partitions, is selected to distribute data records of a data stream among a plurality of nodes of a stream management service. Data ingestion nodes and storage nodes are configured according to the initial mapping. In response to a determination that a triggering criterion for dynamically repartitioning the data stream has been met, a modified mapping is generated, and a different set of ingestion and storage nodes are configured. For at least some time during which arriving data records are stored in accordance with the modified mapping, data records stored at the first set of storage nodes in accordance with the initial mapping are retained.

Abstract: A control node of a multi-tenant stream processing service receives a request indicating an operation to be performed on data records of a particular data stream. Based on a stream partitioning policy, the control node determines an initial number of worker nodes to be used. The control node configures a worker node to perform the operation on received records. In response to a determination that the worker node is in an unhealthy state, the control node configures a replacement worker node.

Abstract: Systems for processing requests to use virtual computing resources in communicatively isolated computing environments apply a multi-tier framework for limiting the amount of virtual computing resources that can be allocated to a user in a computing environment. A bottom tier is associated with a user account that has not been analyzed for security risks, a top tier is associated with a user account that has been validated to use a corresponding computing environment; an intermediate tier, having limits between the bottom and top tiers, is applied to computing environments where the user has not requested resources. Initially, all computing environments are on the bottom tier. In response to a valid request for resources, validated computing environments are moved to the top tier and all other computing environments are moved to the intermediate tier. Requests in an intermediate-tier computing environment trigger validity review; the computing environment can move to top tier.

Abstract: A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.

Abstract: A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.

Abstract: A partitioning policy, comprising an indication of an initial mapping of data records of a stream to a plurality of partitions, is selected to distribute data records of a data stream among a plurality of nodes of a stream management service. Data ingestion nodes and storage nodes are configured according to the initial mapping. In response to a determination that a triggering criterion for dynamically repartitioning the data stream has been met, a modified mapping is generated, and a different set of ingestion and storage nodes are configured. For at least some time during which arriving data records are stored in accordance with the modified mapping, data records stored at the first set of storage nodes in accordance with the initial mapping are retained.

Abstract: Monitoring data can be sent, received, stored, and accessed with zero setup. Customers can send monitoring data to be stored without the customers having to perform any setup ahead-of-time, such as arbitrary lists of key-value pairs. Customers can perform zero setup pivot queries using stored monitoring data. For example, users can perform pivot queries using any available stored information, such as available metric types and dimensions, without any setup needed ahead-of-time.

Abstract: A control node of a multi-tenant stream management service receives a request to initialize a data stream to be comprised of a plurality of data records. The control node determines, based on a partitioning policy, parameters to be used to configure subsystems for ingestion, storage and retrieval of the records. The control node identifies resources to be used for a node of retrieval subsystem The retrieval node is configured to implement programmatic record retrieval interfaces, including respective interfaces to implement non-sequential and sequential access patterns. The control node configures the retrieval node using the selected resources.

Abstract: A partitioning policy, comprising an indication of an initial mapping of data records of a stream to a plurality of partitions, is selected to distribute data records of a data stream among a plurality of nodes of a stream management service. Data ingestion nodes and storage nodes are configured according to the initial mapping. In response to a determination that a triggering criterion for dynamically repartitioning the data stream has been met, a modified mapping is generated, and a different set of ingestion and storage nodes are configured. For at least some time during which arriving data records are stored in accordance with the modified mapping, data records stored at the first set of storage nodes in accordance with the initial mapping are retained.

Abstract: A configuration request comprising a security option selected for a particular data stream is received. Nodes of a plurality of functional categories, such as a data ingestion category and a data retrieval category are to be configured for the stream. The security option indicates a security profile of a resource to be used for nodes of at least one functional category. In accordance with the configuration request, a node of a first functional category is configured at a resource with a first security profile, and configuration of a node of a second functional category is initiated at a different resource with a different security profile.

Abstract: Monitoring data can be received and managed in a multi-tenant environment. Monitoring data can be received for multiple tenants. The received monitoring data can be sorted. Sorting can be performed based on various criteria, such as by tenant. The sorted monitoring data can be written to data files. Each data file can store monitoring data for a plurality of tenants. Receiving, sorting, and writing monitoring data can be performed within a multi-tenant monitoring system.

Abstract: Pivot queries can be performed using stored monitoring data. Pivot queries can be received. Monitoring data can be read from data files. Each data file can store monitoring data for a plurality of tenants. The read monitoring data can be aggregated based on metric types across various dimensions, according to the pivot queries. Results can be presented in various formats, such as charts and graphs. Reading, aggregating, and presenting results can be performed within a multi-tenant monitoring system.

Abstract: A control node of a multi-tenant stream management service receives a request to initialize a data stream to be comprised of a plurality of data records. The control node determines, based on a partitioning policy, parameters to be used to configure subsystems for ingestion, storage and retrieval of the records. The control node identifies resources to be used for a node of retrieval subsystem The retrieval node is configured to implement programmatic record retrieval interfaces, including respective interfaces to implement non-sequential and sequential access patterns. The control node configures the retrieval node using the selected resources.

Abstract: A control node of a multi-tenant stream processing service receives a request indicating an operation to be performed on data records of a particular data stream. Based on a stream partitioning policy, the control node determines an initial number of worker nodes to be used. The control node configures a worker node to perform the operation on received records. In response to a determination that the worker node is in an unhealthy state, the control node configures a replacement worker node.

Abstract: A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.