Abstract: A system obtains data logs from a set of applications that each output data of different data types and in different formats. Data is obtained from the applications as an input message stream and processed into an output message stream with messages in a standardized format for processing by an engine. The data source is detected from analysis of the data and a corresponding filter is applied to generate the output message stream. An alert is provided to an administrative interface when a pattern indicative of malicious activity is detected in the output data steam.

Abstract: A method of ingesting data to enable selection between a fast query mode and a low-cost query mode includes receiving data from one or more data sources, generating records based on the received data, adding the records to a current record file, and adding index metadata to the records. The index metadata for a given record includes a path and an offset, the path identifies a file in the datastore that includes the given record and the offset indicates a location of the given record in the file. The method further includes saving the current record file to a datastore and loading the records in the current record file into a table in a cloud database.

Abstract: A security system obtains data logs from a set of security applications that each output data of different data types and in different formats. A filtering module obtains the data from the security applications as an input message stream and processes the into message stream into an output message stream with messages in a standardized format for processing by a security engine. The filtering module includes a set of filters each tailored to process data from a different data source. The filtering module detects the data source from analysis of the data and applies the corresponding filter to generate the output message stream. The security engine then detects patterns in the output data stream and provides alerts to an administrative interface when it detects a pattern indicative of malicious activity.

Abstract: A query is received from a client device and a mode is selected to process the query from a set of possible modes. The possible modes include a fast mode and a low-cost mode. If the fast mode is selected, the query is forwarded to a cloud database to retrieve responsive records. If the low-cost mode is selected, the cloud database is queried for index metadata of responsive records and the index metadata is used to retrieve the responsive records from a datastore other than the cloud database. Regardless of the mode selected, the responsive records are provided to the client device.

Abstract: A process is provided to allow session state data to be used across sessions. In the process, a first session is established. The first session includes session state data. Then, a second session is established. It is then determined if the second session desires to access session state data established by the first session. If so, at least some of the session state data from the first session is used during the second session to establish the initial session state during the second session.