Abstract: A system and method of recording packets and packet streams to random-access block-oriented recording media, and retrieving the packets and packet streams from said recording media is disclosed. Incoming packets are copied into very large fixed sized blocks, which in turn are distributed to a plurality of disk drives for recording. Supplementary blocks of index data constructed as packets are received and recorded. The index data reduces the time required to locate and read specific packets and packet streams from the recording media. Recorded packets are retrieved and converted to standard pcap format files.

Abstract: The present invention describes a system and method of extracting and storing data elements from network packets, thus performing the task of data mining. In one embodiment of the present invention incoming packets are decomposed one protocol layer at a time to extract data elements contained in the protocol headers. Layer-specific parsers perform deep packet inspection in order to extract data elements from upper-level protocols. Extracted data is arranged in rows, which are subsequently stored into a memory-based accumulator. After some length of time the accumulator is flushed to disk files. Another process reads the flushed disk files row-by-row, inserting each row into a relational database. Standard SQL operations are performed on the relational database in order to generate and display reports of the collected data.