Abstract: The disclosed embodiments provide a system for managing a stream-processing application. During operation, the system allocates a first host for an active instance of the stream-processing application that maintains a state during processing of one or more input streams. Next, the system allocates a second host for a first backup instance that recreates the state on the active instance by consuming changes to the state replicated from the active instance without consuming the input stream(s). During a failure on the first host, the system moves the active instance to the first host by stopping the first backup instance on the second host. Finally, the system launches the active instance on the second host to resume processing of the input stream(s) by the active instance using the recreated state from the first backup instance.

Abstract: Techniques are provided for automatically resizing applications. In one technique, policy data that indicates an order of multiple policies is stored. The policies include (1) a first policy that corresponds to a first computer resource and a first resizing action and (2) a second policy that is lower in priority than the first policy and that corresponds to a second resizing action and a second computer resource. Resource utilization data is received from at least one application executing in a cloud environment. Based on the order, the first policy is identified. Based on the resource utilization data, it is determined whether criteria associated with the first policy are satisfied with respect to the application. If satisfied, then the first resizing action is performed with respect to the application; otherwise, based on the computer resource utilization data, it is determined whether criteria associated with the second policy are satisfied.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system receives records of activity within a stream-processing system over a set of event streams, wherein each event stream in the set of event streams contains events related to a corresponding job in the stream-processing system. Next, the system indexes data in the records under a set of keys that include a first key related to jobs in the stream-processing system and a second key related to errors in the stream-processing system. The system then outputs the indexed data for use in analyzing the execution of the stream-processing system.

Abstract: Techniques are provided for automatically resizing applications. In one technique, policy data that indicates an order of multiple policies is stored. The policies include (1) a first policy that corresponds to a first computer resource and a first resizing action and (2) a second policy that is lower in priority than the first policy and that corresponds to a second resizing action and a second computer resource. Resource utilization data is received from at least one application executing in a cloud environment. Based on the order, the first policy is identified. Based on the resource utilization data, it is determined whether criteria associated with the first policy are satisfied with respect to the application. If satisfied, then the first resizing action is performed with respect to the application; otherwise, based on the computer resource utilization data, it is determined whether criteria associated with the second policy are satisfied.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system receives records of activity within a stream-processing system over a set of event streams, wherein each event stream in the set of event streams contains events related to a corresponding job in the stream-processing system. Next, the system indexes data in the records under a set of keys that include a first key related to jobs in the stream-processing system and a second key related to errors in the stream-processing system. The system then outputs the indexed data for use in analyzing the execution of the stream-processing system.

Abstract: The disclosed embodiments provide a system for managing a stream-processing application. During operation, the system allocates a first host for an active instance of the stream-processing application that maintains a state during processing of one or more input streams. Next, the system allocates a second host for a first backup instance that recreates the state on the active instance by consuming changes to the state replicated from the active instance without consuming the input stream(s). During a failure on the first host, the system moves the active instance to the first host by stopping the first backup instance on the second host. Finally, the system launches the active instance on the second host to resume processing of the input stream(s) by the active instance using the recreated state from the first backup instance.