Abstract: A system receives a set of log data generated from one or more computing services. The system identifies a first timestamp and a second timestamp associated with a set of log data. The system generates an index by encoding the quantized first and second timestamps and identifies a prefix length between the first and second timestamps. The prefix length is then used as a basis to interweave the encoded bits associated with the first and second timestamps to generate an index value. The index value may then be used as a key in connection with the index to locate log data to satisfy a query request.

Abstract: Systems and methods are disclosed to implement a re-sharding technique that redistributes data in a first set of shards of a first stream into a second set of shards of a second stream to reduce the entropy of the data. In embodiments, individual ones of a number of re-sharding nodes determine statistics on an individual shard. The statistics include, for individual data keys in the shard, a frequency of the key and the cardinality of the key. The re-sharding node then selects a list of high-frequency, low-cardinality keys to be used generate a re-sharding scheme that is used to redistribute data items in the first set of shards to the second set. The re-sharding scheme assigns data items having the same key value combinations to the same range of shards in the second set. Accordingly, upon redistribution, data entropy in the second set of shards is reduced.