Abstract: Processing of continuously generated data using a rolling transaction procedure is described. For instance, a system can process a data stream comprising a first segment and a second segment. A transaction associated with the data stream can be initiated and in response to the transaction being initiated, a first transaction segment for the first segment and a second transaction segment for the second segment are generated. Further, a scaling event that modifies the second segment into a third segment and a fourth segment can be detected, and a data stream transaction procedure is executed to end the transaction.

Abstract: Event loading is individually tracked for each of the segments of a stream. Event loading trends of each of the segments of the stream based on event loading changes are identified and reported. Selected segments are automatically scaled-up or scaled-down based on the event loading trends. Individual segment event loading is measured by computing an exponential weighted moving average of data, measured either in events or bytes, received per fixed interval of time. Exponential functions are used to assign exponentially decreasing weights over time. Multiple different running averages are computed for different time intervals, each with a different weight that represents a decay factor.

Abstract: Event loading is individually tracked for each of the segments of a stream. Event loading trends of each of the segments of the stream based on event loading changes are identified and reported. Selected segments are automatically scaled-up or scaled-down based on the event loading trends. Individual segment event loading is measured by computing an exponential weighted moving average of data, measured either in events or bytes, received per fixed interval of time. Exponential functions are used to assign exponentially decreasing weights over time. Multiple different running averages are computed for different time intervals, each with a different weight that represents a decay factor.

Abstract: Reporting time progress for written events on a stream is disclosed. The system comprises operations for obtaining time report information comprising respective timestamp information and respective positional offset information for respective writers with respect to events written to a stream by the respective writers. The operations further comprise aggregating the time report information and determining a lowerbound, e.g., a minimum timestamp before which new events to be written by the stream storage system will no longer arrive, and determining an upperbound, e.g., a stream offset representative of an overall offset of the events written thus far in the stream. The operations further comprise storing the minimum timestamp and the stream offset in a data structure for the events written thus far.