Abstract: The disclosed embodiments provide a system for managing a stream-processing application. During operation, the system allocates a first host for an active instance of the stream-processing application that maintains a state during processing of one or more input streams. Next, the system allocates a second host for a first backup instance that recreates the state on the active instance by consuming changes to the state replicated from the active instance without consuming the input stream(s). During a failure on the first host, the system moves the active instance to the first host by stopping the first backup instance on the second host. Finally, the system launches the active instance on the second host to resume processing of the input stream(s) by the active instance using the recreated state from the first backup instance.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system receives records of activity within a stream-processing system over a set of event streams, wherein each event stream in the set of event streams contains events related to a corresponding job in the stream-processing system. Next, the system indexes data in the records under a set of keys that include a first key related to jobs in the stream-processing system and a second key related to errors in the stream-processing system. The system then outputs the indexed data for use in analyzing the execution of the stream-processing system.

Abstract: A geographically distributed system using a hybrid, multi-master design is disclosed. The system may use global tables, for which both reads and writes may be performed locally, but with a single region serving as a master region. All strongly consistent reads and writes may be sent to the master region, whereas eventually consistent reads and writes may be performed in any region with a replica. The master region may not be statically assigned but may move from region to region. The master region may be assigned manually by client preference or may be assigned and updated automatically based on the time of day or workload characteristics.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system receives records of activity within a stream-processing system over a set of event streams, wherein each event stream in the set of event streams contains events related to a corresponding job in the stream-processing system. Next, the system indexes data in the records under a set of keys that include a first key related to jobs in the stream-processing system and a second key related to errors in the stream-processing system. The system then outputs the indexed data for use in analyzing the execution of the stream-processing system.

Abstract: The disclosed embodiments provide a system for managing a stream-processing application. During operation, the system allocates a first host for an active instance of the stream-processing application that maintains a state during processing of one or more input streams. Next, the system allocates a second host for a first backup instance that recreates the state on the active instance by consuming changes to the state replicated from the active instance without consuming the input stream(s). During a failure on the first host, the system moves the active instance to the first host by stopping the first backup instance on the second host. Finally, the system launches the active instance on the second host to resume processing of the input stream(s) by the active instance using the recreated state from the first backup instance.

Abstract: A geographically distributed system using a hybrid, multi-master design is disclosed. The system may use global tables, for which both reads and writes may be performed locally, but with a single region serving as a master region. All strongly consistent reads and writes may be sent to the master region, whereas eventually consistent reads and writes may be performed in any region with a replica. The master region may not be statically assigned but may move from region to region. The master region may be assigned manually by client preference or may be assigned and updated automatically based on the time of day or workload characteristics.