Abstract: Disclosed are some examples of systems, methods, apparatus and computer program products for scheduling jobs to process log files. In some implementations, a scheduler can be configured to identify a final state of a previously scheduled first job as a failure in execution by one or more processors. For instance, the first job may have been defined to process a first input log file saved to a database during a first timeframe of a sequence of timeframes of a designated interval. The scheduler can detect a second input log file saved to the database. The scheduler can then define a second job to process the first input log file and the second input log file and schedule the second job in association with a second timeframe of the sequence.

Abstract: Disclosed are some examples of database systems, methods, and computer program products for run-time schema for event records. In some implementations, event records satisfying a query can be identified. Attribute-value pairs of data of those identified event records can be stored in an unstructured data field of an event record.

Abstract: A database system may determine that a database is unavailable to generate an identifier for an event or that events may not be written to the database. As a result, the database system may write the event to a low-latency, supplementary file system. The database system may determine that the database is available at a later time, and read the event from the supplementary file system, store the event in a memory component, and then delete the event from the supplementary file system. The database system may then access the database to generate an identifier for the event, write the event to the database, and then delete the event from the memory component.

Abstract: In an example, a processing system of a database system may categorize event data taken from logged interactions of users with a multi-tenant information system to provide a metric. Event roll-up aggregate metrics used to provide the metric may be generated in connection with event capture. The processing system of the database system may periodically calculate the metric for a particular one of the tenants, and electronically store the periodically calculated metrics for accessing responsive to a query of the particular tenant.

Abstract: A capture service running on an application server receives events from a client application running on an application server to be stored in a data store and stores the events in an in-memory bounded buffer on the application server, the in-memory bounded buffer comprising a plurality of single-threaded segments, the capture service to write events to each segment in parallel. The in-memory bounded buffer provides a notification to a buffer flush regulator when a number of events stored in the in-memory bounded buffer reaches a predefined limit. The in-memory bounded buffer receive a request to flush the events in the in-memory bounded buffer from a consumer executor service. The consumer executor service consumes the events in the in-memory bounded buffer using a dynamically sized thread pool of consumer threads to read the segments of the bounded buffer in parallel, wherein consuming the events comprises writing the events directly to the data store.

Abstract: Computer-implemented methods and systems are provided for writing events to a data store. An application server generates events, the data store that stores the events, and a temporary events file storage system (TEFSS) temporarily stores groups of events as events files. When events are unable to be written directly to the data store, an indirect events writer is invoked that includes event capture threads each being configured to generate a particular events file, and write it to the TEFSS. Each events file includes a plurality of events flushed from an in-memory buffer service. An events file uploader service reads the events file(s) from the TEFSS, and then writes the events from each of the events files to the data store.

Abstract: Computer-implemented methods and systems are provided. The system includes a data store that is configured to store events in an event table, a temporary events file storage system (TEFSS), and a cluster of application servers. The cluster includes a first application server that generates events, and a second application server that includes an events file uploader service. When the first application server is unable to directly write events to the data store, an indirect events writer generates events file(s), and writes the events file(s) to the TEFSS. Each events file includes a plurality of events flushed from an in-memory buffer service at the first application server. When the events file uploader service determines that the first application server is inactive, it reads the events file(s) from the TEFSS, and writes the events from each of the events files to the data store.

Abstract: Computer-implemented methods and systems are provided for writing events to a data store. An application server generates events, the data store that stores the events, and a temporary events file storage system (TEFSS) temporarily stores groups of events as events files. When events are unable to be written directly to the data store, an indirect events writer is invoked that includes event capture threads each being configured to generate a particular events file, and write it to the TEFSS. Each events file includes a plurality of events flushed from an in-memory buffer service. An events file uploader service reads the events file(s) from the TEFSS, and then writes the events from each of the events files to the data store.

Abstract: Disclosed are some examples of database systems, methods, and computer program products for processing log files. In some implementations, a server of a database system accesses a metadata file indicating algorithms that can be applied to data of log files. The server generates customer-facing log files using the log file and metadata file. The customer-facing log files include new data derived from using the algorithms and the data of the log files.

Abstract: Computer-implemented methods and systems are provided for writing events to a data store. An application server generates events, the data store that stores the events, and a temporary events file storage system (TEFSS) temporarily stores groups of events as events files. When events are unable to be written directly to the data store, an indirect events writer is invoked that includes event capture threads each being configured to generate a particular events file, and write it to the TEFSS. Each events file includes a plurality of events flushed from an in-memory buffer service. An events file uploader service reads the events file(s) from the TEFSS, and then writes the events from each of the events files to the data store.

Abstract: Computer-implemented methods and systems are provided. The system includes a data store that is configured to store events in an event table, a temporary events file storage system (TEFSS), and a cluster of application servers. The cluster includes a first application server that generates events, and a second application server that includes an events file uploader service. When the first application server is unable to directly write events to the data store, an indirect events writer generates events file(s), and writes the events file(s) to the TEFSS. Each events file includes a plurality of events flushed from an in-memory buffer service at the first application server. When the events file uploader service determines that the first application server is inactive, it reads the events file(s) from the TEFSS, and writes the events from each of the events files to the data store.

Abstract: A capture service running on an application server receives events from a client application running on an application server to be stored in a data store and stores the events in an in-memory bounded buffer on the application server, the in-memory bounded buffer comprising a plurality of single-threaded segments, the capture service to write events to each segment in parallel. The in-memory bounded buffer provides a notification to a buffer flush regulator when a number of events stored in the in-memory bounded buffer reaches a predefined limit. The in-memory bounded buffer receive a request to flush the events in the in-memory bounded buffer from a consumer executor service. The consumer executor service consumes the events in the in-memory bounded buffer using a dynamically sized thread pool of consumer threads to read the segments of the bounded buffer in parallel, wherein consuming the events comprises writing the events directly to the data store.

Abstract: In an example, a processing system of a database system may categorize event data taken from logged interactions of users with a multi-tenant information system to provide a metric. Event roll-up aggregate metrics used to provide the metric may be generated in connection with event capture. The processing system of the database system may periodically calculate the metric for a particular one of the tenants, and electronically store the periodically calculated metrics for accessing responsive to a query of the particular tenant.

Abstract: A capture service running on an application server receives events from a client application running on an application server to be stored in a data store and stores the events in an in-memory bounded buffer on the application server, the in-memory bounded buffer comprising a plurality of single-threaded segments, the capture service to write events to each segment in parallel. The in-memory bounded buffer provides a notification to a buffer flush regulator when a number of events stored in the in-memory bounded buffer reaches a predefined limit. The in-memory bounded buffer receive a request to flush the events in the in-memory bounded buffer from a consumer executor service. The consumer executor service consumes the events in the in-memory bounded buffer using a dynamically sized thread pool of consumer threads to read the segments of the bounded buffer in parallel, wherein consuming the events comprises writing the events directly to the data store.

Abstract: A database system captures custom information of a header section associated with a logged interaction of a user. The database system may receive a hypertext transfer protocol (http) message including the header section and determine whether the header section includes a predetermined data pattern associated with the custom information added by a second application that is different than a first application which initiated the http message. The database system may extract the custom information from the header section in response to determining that the header section includes the predetermined data pattern. The custom information and event data extracted from the logged interaction of the user may be stored on the database system as a storage element.

Abstract: In an example, a processing system of a database system may categorize event data taken from logged interactions of users with a multi-tenant information system to provide a metric. The processing system of the database system may periodically calculate the metric for a particular one of the tenants, and electronically store the periodically calculated metrics for accessing responsive to a query of the particular tenant.

Abstract: A capture service running on an application server receives events from a client application running on an application server to be stored in a data store and stores the events in an in-memory bounded buffer on the application server, the in-memory bounded buffer comprising a plurality of single-threaded segments, the capture service to write events to each segment in parallel. The in-memory bounded buffer provides a notification to a buffer flush regulator when a number of events stored in the in-memory bounded buffer reaches a predefined limit. The in-memory bounded buffer receive a request to flush the events in the in-memory bounded buffer from a consumer executor service. The consumer executor service consumes the events in the in-memory bounded buffer using a dynamically sized thread pool of consumer threads to read the segments of the bounded buffer in parallel, wherein consuming the events comprises writing the events directly to the data store.

Abstract: In an example, composite keys for an event log may be provided. A partitioner may be configured to extract a natural key from the composite keys and distribute log lines of event log files to a plurality of reducer nodes based on a value of the natural key. A comparator may use a log time of the composite key to sort a received portion of the distributed log lines.

Abstract: Disclosed are some examples of database systems, methods, and computer program products for processing log files. In some implementations, a server of a database system accesses a log file having a release version. The log file includes data entries identifying system events corresponding to user activity. The server generates or updates, based on the release version of the log file, a metadata file indicating approved entry types and approved data. The server also generates or updates, based on the log file and the metadata file, customer-facing log files. Each customer-facing log file is associated with a corresponding customer entity capable of being serviced by the database system. Each customer-facing log file includes a subset of the entries and a subset of the data items of at least one of the entries.

Abstract: Disclosed are methods, apparatus, systems, and computer program products for provisioning log file data. A server may parse through log files by selecting particular log entries and data fields. Customer-facing log files may be generated based on the selected log entries and data fields.