Abstract: Deep packet inspection (DPI) techniques are utilized to provide data compression, particularly necessary in many bandwidth-limited communication systems. A separate processor is initially used within a transmission source to scan, in real time, a data packet stream and recognize repetitive patterns that are occurring in the data. The processor builds a dictionary (ruleset), storing the set, of repetitive patterns and defining a unique token ID to be associated with each pattern. Thereafter, the DPI engine uses this ruleset to recognize the repetitive data patterns and replace each relatively long data pattern with its short token ID, creating a compressed data packet.