Abstract: A computer-implemented method to sample a large data set of traffic records, including receiving a traffic record associated with a traffic flow from a source of a large data set of traffic records, incrementing a flow counter representing a number of traffic flows received for one address of a pair of addresses identified by a traffic record, adding a traffic size of the traffic flow associated with the received traffic record to a total traffic size of all flows received in previous iterations. If the flow counter is less than a predetermined sampling threshold, then storing a traffic record sample associated with the traffic record. If the flow counter is more than the predetermined sampling threshold, then determining whether or not to sample the received traffic record by applying an exponentially decreasing probability function. Storing the traffic record sample as sampled data associated with the traffic record only if the determination is to sample the received traffic record.

Abstract: A computer-implemented method to sample a large data set of traffic records, including receiving a traffic record associated with a traffic flow from a source of a large data set of traffic records, incrementing a flow counter representing a number of traffic flows received for one address of a pair of addresses identified by a traffic record, adding a traffic size of the traffic flow associated with the received traffic record to a total traffic size of all flows received in previous iterations. If the flow counter is less than a predetermined sampling threshold, then storing a traffic record sample associated with the traffic record. If the flow counter is more than the predetermined sampling threshold, then determining whether or not to sample the received traffic record by applying an exponentially decreasing probability function. Storing the traffic record sample as sampled data associated with the traffic record only if the determination is to sample the received traffic record.