Abstract: A system for gathering, formatting, validating, compressing, processing and storing a large volume of transactional data is presented. The system preferably analyzes pharmaceutical drug transactions. Importantly, the method of compressing the gathered data of the present invention detects repetitive behavior, and data patterns, to more efficiently process and store the large volume of transactional data. Also, the stored data is processed and analyzed more quickly due to its reduced size. Further, the present invention retains all useful information represented by the transactional data after compression of the data (i.e., it does not filter data for the purpose of reducing the quantity of data). The results of the process give users a truer sense of market activity. Additionally, the present invention allows new data to be added to historical data to allow a progressive analysis of the data.