Abstract: A method includes receiving first data from a first data source that continuously generates a first set of records with first common fields. The first data is filtered by selecting a first sub-set of fields of the first common fields to form first filtered data. Second data is received from a second data source that continuously generates a second set of records with second common fields. The second data is filtered by selecting a second sub-set of fields of the second common fields to form second filtered data, where the first filtered data and the second filtered data are generated simultaneously. Rules are applied to the first filtered data and the second filtered data in real-time to identify selected real-time events. The selected real-time events are reported prior to persistently storing the first filtered data, the second filtered data and the selected real-time events.