Abstract: Automated reconfiguration of real time data stream processing may be implemented. A processing function that describes one or more operations to be performed with respect to one or more data streams may be executed at one or more processing nodes. Performance metrics describing the performance of the processing function at the processing nodes may be collected and monitored. A reconfiguration event may be detected for the processing function. A different execution configuration for the processing function may be determined and initiated in response to detecting the reconfiguration event.

Abstract: Automated reconfiguration of real time data stream processing may be implemented. A processing function that describes one or more operations to be performed with respect to one or more data streams may be executed at one or more processing nodes. Performance metrics describing the performance of the processing function at the processing nodes may be collected and monitored. A reconfiguration event may be detected for the processing function. A different execution configuration for the processing function may be determined and initiated in response to detecting the reconfiguration event.

Abstract: A robotic device management service obtains, from a customer, a first set of parameters of a robotic device and a second set of parameters for a simulation environment for testing a robotic device application installable on the robotic device. The set of parameters are used to indicate a storage location of the application and a selection of a simulation environment for testing the application. In response to the request, the robotic device management service selects a set of resources on which to execute the simulation in the simulation environment. The robotic device management service obtains the robotic device application from the storage location and loads the application on to the set of resources to execute the simulation.

Abstract: Automated reconfiguration of real time data stream processing may be implemented. A processing function that describes one or more operations to be performed with respect to one or more data streams may be executed at one or more processing nodes. Performance metrics describing the performance of the processing function at the processing nodes may be collected and monitored. A reconfiguration event may be detected for the processing function. A different execution configuration for the processing function may be determined and initiated in response to detecting the reconfiguration event.

Abstract: Windowed queries with event-based open times are described for use in data analytics for streaming data. A window semantic may be used to define windowed queries of an analytics application that, when executed against an incoming data stream, open a window at a time when a first event associated with a grouping key is received by a windowed query. The window remains open for a predefined duration specified in the windowed query, and query results may be generated based on any events associated with the grouping key that are received while the window remains open. After a lapse of the predefined duration, the window closes, and the query results may be output in association with the window.

Abstract: Idempotent processing of data may be implemented for data records retrieved from a data stream. A data stream may receive data records as input and distribute the ingestion, storage, and processing of the data records amongst one or more partitions of the data stream. Partition metadata may be maintained which includes checkpoint metadata for retrieving, processing, and sending data records in the data stream to a specified destination. When assigned a partition for processing, checkpoint metadata for partition may be accessed to determine whether a pending checkpoint for the partition exists. If not pending checkpoint exists, new data records may be retrieved, processed, and sent from the partition of the data stream to a specified destination. If a checkpoint is pending, then the data records identified by the checkpoint metadata as pending may be retrieved, processed, and sent to the specified destination.

Abstract: Disclosed are various embodiments for consuming fragments of time-associated data streams. A streaming gateway receives a request from a consumer via a network to obtain a stream of time-associated data. The gateway determines that a first fragment of the stream of time-associated data is stored in a data store. The first fragment is obtained from the data store and sent to the consumer via the network. The gateway determines that the stream of time-associated data is currently being received by an endpoint from a producer. A second segment of the stream is obtained from the endpoint and sent to the consumer via the network.

Abstract: Disclosed are various embodiments for fragmentation of time-associated streaming data. A producer application receives a stream of time-associated data. The producer application then generates a first fragment of the stream of time-associated data and assigns a first unique fragment identifier and a first timestamp to the first fragment. The producer application sends the first fragment via a network using an application-layer protocol to an endpoint. The producer application receives at least one acknowledgement for the first fragment from the endpoint via the network using the application-layer protocol.

Abstract: Techniques are described for facilitating use of sequences of invocable services in a configurable manner, including by providing a metering service that tracks the use of invocable services by other invocable services or applications. In at least some situations, the invocable services are fee-based Web services or other fee-based network-accessible services, and are made available by providers of the services for use by others in exchange for fees defined by the service providers. The described techniques facilitate use of a sequence of such invocable services by applications in a manner configured by the creators of the applications and providers of the services, including to allow the application creators to define pricing terms that will be used to determine fees that end users of the applications will be charged for use of the invocable services via the applications.

Abstract: Various embodiments replicate and index fragments of a stream of time-associated data. In one embodiment, a stream endpoint application receives a first fragment of a stream of time-associated data from a producer via a network using an application-layer protocol. The stream endpoint application determines a producer timestamp from the first fragment. The stream endpoint application sends at least one acknowledgment to the producer via the network using the application-layer protocol. The stream endpoint application indexes the first fragment and sends the first fragment to a destination.

Abstract: Techniques are described for facilitating use of invocable services by applications in a configurable manner. In at least some situations, the invocable services are Web services or other network-accessible services that are made available by providers of the services for use by others in exchange for fees defined by the service providers. The described techniques facilitate use of such invocable services by applications in a manner configured by the developers of the applications, including to allow the application developers to configure pricing terms that will be used to determine fees that end users of the applications will be charged for use of the invocable services via the applications. In some situations, the configured pricing terms for a service specify fees for end users that differ in one or more ways from the defined fees charged by the provider of that service.

Abstract: A programmatic interface is implemented, enabling a client of a stream management service to select a data ingestion policy for a data stream. A client request selecting an at-least-once ingestion policy is received. In accordance with the at-least-once policy, a client may transmit an indication of a data record one or more times to the service until a positive acknowledgement is received. In response to receiving a plurality of transmissions indicating a particular data record, respective positive acknowledgements are sent to the client. Based on a persistence policy selected for the stream, copies of the data record are stored at one or more storage locations in response to one particular transmission of the plurality of transmissions.

Abstract: Automated reconfiguration of real time data stream processing may be implemented. A processing function that describes one or more operations to be performed with respect to one or more data streams may be executed at one or more processing nodes. Performance metrics describing the performance of the processing function at the processing nodes may be collected and monitored. A reconfiguration event may be detected for the processing function. A different execution configuration for the processing function may be determined and initiated in response to detecting the reconfiguration event.

Abstract: Disclosed are various embodiments for a framework for time-associated data stream storage, processing, and replication. A plurality of streams of time-associated data are received from a plurality of sources via a network using an application-layer protocol. Each of the plurality of streams is divided into a plurality of fragments. An acknowledgement is sent to each of the plurality of sources for each of the plurality of fragments via the network using the application-layer protocol. Processing is performed on each of the plurality of fragments for individual ones of the plurality of streams. An action is implemented relative to a respective fragment based at least in part on a result of the processing.

Abstract: Techniques are described for facilitating use of invocable services by applications in a configurable manner. In at least some situations, the invocable services are Web services or other network-accessible services that are made available by providers of the services for use by others in exchange for fees defined by the service providers. The described techniques facilitate use of such invocable services by applications in a manner configured by the application providers and the service providers, including to track use of third-party invocable services by applications on behalf of end users and to allocate fees that are charged end users between the applications and the services as configured by the providers of the applications and services. In some situations, the configured pricing terms for a service specify fees for end users that differ in one or more ways from the defined fees charged by the provider of that service.

Abstract: Idempotent processing of data may be implemented for data records retrieved from a data stream. A data stream may receive data records as input and distribute the ingestion, storage, and processing of the data records amongst one or more partitions of the data stream. Partition metadata may be maintained which includes checkpoint metadata for retrieving, processing, and sending data records in the data stream to a specified destination. When assigned a partition for processing, checkpoint metadata for partition may be accessed to determine whether a pending checkpoint for the partition exists. If not pending checkpoint exists, new data records may be retrieved, processed, and sent from the partition of the data stream to a specified destination. If a checkpoint is pending, then the data records identified by the checkpoint metadata as pending may be retrieved, processed, and sent to the specified destination.

Abstract: A partitioning policy, comprising an indication of an initial mapping of data records of a stream to a plurality of partitions, is selected to distribute data records of a data stream among a plurality of nodes of a stream management service. Data ingestion nodes and storage nodes are configured according to the initial mapping. In response to a determination that a triggering criterion for dynamically repartitioning the data stream has been met, a modified mapping is generated, and a different set of ingestion and storage nodes are configured. For at least some time during which arriving data records are stored in accordance with the modified mapping, data records stored at the first set of storage nodes in accordance with the initial mapping are retained.

Abstract: A robotic device management service obtains, from a customer, a first set of parameters of a robotic device and a second set of parameters for a simulation environment for testing a robotic device application installable on the robotic device. The set of parameters are used to indicate a storage location of the application and a selection of a simulation environment for testing the application. In response to the request, the robotic device management service selects a set of resources on which to execute the simulation in the simulation environment. The robotic device management service obtains the robotic device application from the storage location and loads the application on to the set of resources to execute the simulation.

Abstract: A robotic device management service obtains a set of parameters of a simulation environment and a set of components for execution of a simulation of a robotic device application. Based on these parameters, the robotic device management service selects a set of resources for executing the application in a simulation environment. The robotic device management service may launch the set of components among the set of resources and monitor execution of the application in the simulation environment to ensure completion of the simulation.

Abstract: A control node of a multi-tenant stream processing service receives a request indicating an operation to be performed on data records of a particular data stream. Based on a stream partitioning policy, the control node determines an initial number of worker nodes to be used. The control node configures a worker node to perform the operation on received records. In response to a determination that the worker node is in an unhealthy state, the control node configures a replacement worker node.