Abstract: A method for compressing a sequence of records, each record comprising a sequence of fields, comprises steps of buffering a record in a line of a matrix, reordering the lines of the matrix according to locality sensitive hash values of the buffered records such that records with similar contents in corresponding fields are placed in proximity, and consolidating fields in columns of the matrix into a block of codes. In this, consolidating yields codes of one of a first type comprising a sequence of individual fields and a second type comprising a sequence of fields with at least one repetition. The second type of code comprises a presence field indicating repeated fields and an iteration field indicating a number of respective repetitions. Decompression of the records from the block codes compressed above is also described.

Abstract: A data traffic monitor for determining a heavy distinct hitter (HDH) in a data stream, the data stream comprising a plurality of element-value (e,v) pairs, includes a HDH module, the HDH module configured to receive the plurality of (e,v) pairs from the data stream; and a counter block in communication with the HDH module, the counter block comprising a plurality of hash functions, and further comprising a respective pair of distinct counting primitives associated with each hash function of the plurality of hash functions, wherein each of the plurality of (e,v) pairs is added to one of the distinct counting primitives of the respective pair of distinct counting primitives for each of the plurality of hash functions in each of the plurality of counter blocks.

Abstract: A method for determining a heavy distinct hitter (HDH) in a data stream by a data traffic monitor, the data stream comprising a plurality of element-value (e,v) pairs, includes receiving the plurality of (e,v) pairs from the data stream by an HDH module in the data traffic monitor, the HDH module being in communication with a counter block, the counter block comprising a plurality of hash functions, and further comprising a respective pair of distinct counting primitives associated with each hash function of the plurality of hash functions; and adding each of the plurality of (e,v) pairs to one of the distinct counting primitives of the respective pair of distinct counting primitives for each of the plurality of hash functions in the counter block.

Abstract: A method for compressing a sequence of records, each record comprising a sequence of fields, comprises steps of buffering a record in a line of a matrix, reordering the lines of the matrix according to locality sensitive hash values of the buffered records such that records with similar contents in corresponding fields are placed in proximity, and consolidating fields in columns of the matrix into a block of codes. In this, consolidating yields codes of one of a first type comprising a sequence of individual fields and a second type comprising a sequence of fields with at least one repetition. The second type of code comprises a presence field indicating repeated fields and an iteration field indicating a number of respective repetitions. Decompression of the records from the block codes compressed above is also described.

Abstract: A data traffic monitor for determining a heavy distinct hitter (HDH) in a data stream, the data stream comprising a plurality of element-value (e,v) pairs, includes a HDH module, the HDH module configured to receive the plurality of (e,v) pairs from the data stream; and a counter block in communication with the HDH module, the counter block comprising a plurality of hash functions, and further comprising a respective pair of distinct counting primitives associated with each hash function of the plurality of hash functions, wherein each of the plurality of (e,v) pairs is added to one of the distinct counting primitives of the respective pair of distinct counting primitives for each of the plurality of hash functions in each of the plurality of counter blocks.

Abstract: A data traffic monitor for determining a heavy distinct hitter (HDH) in a data stream, the data stream comprising a plurality of element-value (e,v) pairs, includes a HDH module, the HDH module configured to receive the plurality of (e,v) pairs from the data stream; and a counter block in communication with the HDH module, the counter block comprising a plurality of hash functions, and further comprising a respective pair of distinct counting primitives associated with each hash function of the plurality of hash functions, wherein each of the plurality of (e,v) pairs is added to one of the distinct counting primitives of the respective pair of distinct counting primitives for each of the plurality of hash functions in each of the plurality of counter blocks.