Abstract: Techniques for epoch pipelining are disclosed. In one particular exemplary embodiment, the techniques may be realized as a method for epoch pipelining comprising the steps of beginning a first epoch, determining for one or more pages of memory if the pages of memory are likely to be modified in a subsequent epoch, performing a first operation on the pages of memory that are likely to be modified in a subsequent epoch, beginning a second epoch, subsequent to the first epoch, performing a second operation on the pages of memory that are not likely to be modified in a subsequent epoch, and ending the first epoch, wherein the first operation and the second operation are dissimilar.

Abstract: Methods and systems for efficient and accurate re-starting of data transfers using volatile data transfer mechanisms, such as pipes, following an error. According to one embodiment, portions of the transferred data can be saved in a fallback file. In addition, checkpoint commands can be periodically provided, upon which much of the data in the fallback file can be overwritten. During the original transmission, the data that has passed through the mechanism can be counted, and a signature for the data that has passed can be accumulated. After certain error conditions, the data in the fallback file can be recovered and passed to the receiving process and the transmission can then be resumed at the point where it had left off. After other errors requiring a complete re-start, a second signature of the re-transmitted data can be accumulated and original and second signatures compared to determine if the data is valid.

Abstract: One embodiment of the present invention relates to a system for extracting online data from a variety of web-sources in real-time and transmitting the data to a data warehouse in real-time. The system comprises an extractor plug-in having instructions configured to integrate with a pre-determined type of host server and extracting data in real-time from any variety of web source in communication with the host server. The system also comprises a transformer engine in communication with the extractor plug-in and configured to transmit the extracted data in real-time into a data warehouse for analysis thereof.