Abstract: Methods, systems, and computer program products for categorizing log records at run-time are provided herein. A computer-implemented method includes generating one or more template signatures to be associated with each of multiple templates, wherein each of the multiple templates comprises a concatenation of one or more words; processing each of multiple log records derived from a data stream to determine a composition of each of the multiple log records; matching one or more of the generated template signatures to each of the multiple log records based on the determined composition of each of the multiple log records; and outputting an identification of (i) each of the multiple log records and (ii) the one or more generated template signatures matched thereto.

Abstract: Sequences of hierarchical records are aggregated and summarized. A capture log that includes a plurality of operations of a workload is received. A first data structure that models transaction types as sequences of nodes is created. The nodes identify operations in the workload. A present operation and a transaction identifier are read from the capture log. The transaction identifier is dissociated from a first node that identifies a prior operation. The transaction identifier is associated with a second node that identifies the present operation. In a second data structure that associates nodes with transaction identifiers, the first node is dissociated from the transaction identifier and the second node is associated with the transaction identifier. A summary of the workload is generated based, at least in part, on the first and second data structures. The summary includes signatures of transaction types and counts of instances of the transaction types.

Abstract: Sequences of hierarchical records are aggregated and summarized. A capture log that includes a plurality of operations of a workload is received. A first data structure that models transaction types as sequences of nodes is created. The nodes identify operations in the workload. A present operation and a transaction identifier are read from the capture log. The transaction identifier is dissociated from a first node that identifies a prior operation. The transaction identifier is associated with a second node that identifies the present operation. In a second data structure that associates nodes with transaction identifiers, the first node is dissociated from the transaction identifier and the second node is associated with the transaction identifier. A summary of the workload is generated based, at least in part, on the first and second data structures. The summary includes signatures of transaction types and counts of instances of the transaction types.

Abstract: Methods, systems, and computer program products for categorizing log records at run-time are provided herein. A computer-implemented method includes generating one or more template signatures to be associated with each of multiple templates, wherein each of the multiple templates comprises a concatenation of one or more words; processing each of multiple log records derived from a data stream to determine a composition of each of the multiple log records; matching one or more of the generated template signatures to each of the multiple log records based on the determined composition of each of the multiple log records; and outputting an identification of (i) each of the multiple log records and (ii) the one or more generated template signatures matched thereto.

Abstract: Sequences of hierarchical records are aggregated and summarized. A capture log that includes a plurality of operations of a workload is received. A first data structure that models transaction types as sequences of nodes is created. The nodes identify operations in the workload. A present operation and a transaction identifier are read from the capture log. The transaction identifier is dissociated from a first node that identifies a prior operation. The transaction identifier is associated with a second node that identifies the present operation. In a second data structure that associates nodes with transaction identifiers, the first node is dissociated from the transaction identifier and the second node is associated with the transaction identifier. A summary of the workload is generated based, at least in part, on the first and second data structures. The summary includes signatures of transaction types and counts of instances of the transaction types.

Abstract: Sequences of hierarchical records are aggregated and summarized. A capture log that includes a plurality of operations of a workload is received. A first data structure that models transaction types as sequences of nodes is created. The nodes identify operations in the workload. A present operation and a transaction identifier are read from the capture log. The transaction identifier is dissociated from a first node that identifies a prior operation. The transaction identifier is associated with a second node that identifies the present operation. In a second data structure that associates nodes with transaction identifiers, the first node is dissociated from the transaction identifier and the second node is associated with the transaction identifier. A summary of the workload is generated based, at least in part, on the first and second data structures. The summary includes signatures of transaction types and counts of instances of the transaction types.

Abstract: A method, a computer program product and a system identify partition locations within an extended markup language (XML) document without parsing so as to process portions of said document in parallel. The XML document includes sections required to remain continuous. The document is scanned for continuous sections without parsing, and boundaries of the initial partitions are adjusted to reside outside the continuous sections to determine resulting partitions for the document. The resulting partitions may be processed in parallel to provide the document information for storage.

Abstract: A system for receiving a declarative specification including a plurality of stages. Each stage specifies an atomic operation, a data input to the atomic operation, and a data output from the atomic operation. The data input is characterized by a data type. Links between at least two of the stages are generated to create a data integration workflow. The data integration workflow is compiled to generate computer code for execution on a parallel processing platform. The computer code configured to perform at least one of data preparation and data analysis.

Abstract: A method for receiving a declarative specification including a plurality of stages. Each stage specifies an atomic operation, a data input to the atomic operation, and a data output from the atomic operation. The data input is characterized by a data type. Links between at least two of the stages are generated to create a data integration workflow. The data integration workflow is compiled to generate computer code for execution on a parallel processing platform. The computer code configured to perform at least one of data preparation and data analysis.

Abstract: Database operation records are sequentially read from two or more log files. If the transaction identifier is new and the record is not an end-of-transaction record, an open transactions list entry is created. If the transaction identifier is new and the record is an end-of-transaction record, a transaction type list entry is created or updated. If the transaction identifier is not new and is not an end-of-transaction record, an open transactions list entry is updated. If the transaction identifier is not new and the record is an end-of-transaction record, a transaction type list entry is created or updated. When all log file records are read, analytical comparison between the information associated with two or more of the log files in data fields in the transaction type list entries is performed.

Abstract: Identifying matching transactions between two log files. First and second log files contain operation records of transactions in a transaction workload. The first and second log files are split into first and second corresponding partition files, based on distinct sequences of operation record types beginning operation records of the transactions in each of the log files. A record location in a first partition file, and a window of sequential record locations in a corresponding second partition file at a defined offset relative to the record location in the first file are advanced one record location at a time. If each operation record of a complete transaction at a record location in a first file has a matching record in the associated window of record locations in a second file, the corresponding transactions match.

Abstract: Identifying matching transactions between two log files. First and second log files contain operation records of transactions in a transaction workload. The first and second log files are split into first and second corresponding partition files, based on distinct sequences of operation record types beginning operation records of the transactions in each of the log files. A record location in a first partition file, and a window of sequential record locations in a corresponding second partition file at a defined offset relative to the record location in the first file are advanced one record location at a time. If each operation record of a complete transaction at a record location in a first file has a matching record in the associated window of record locations in a second file, the corresponding transactions match.

Abstract: Arrangements and methods for employing empirical evidence to estimate the performance of applications with very few data samples, in complex environments such as dynamic SDP environments, using one or more effective data-plotting models.

Abstract: Database operation records are sequentially read from two or more log files. If the transaction identifier is new and the record is not an end-of-transaction record, an open transactions list entry is created. If the transaction identifier is new and the record is an end-of-transaction record, a transaction type list entry is created or updated. If the transaction identifier is not new and is not an end-of-transaction record, an open transactions list entry is updated. If the transaction identifier is not new and the record is an end-of-transaction record, a transaction type list entry is created or updated. When all log file records are read, analytical comparison between the information associated with two or more of the log files in data fields in the transaction type list entries is performed.

Abstract: Arrangements and methods for employing empirical evidence to estimate the performance of applications with very few data samples, in complex environments such as dynamic SDP environments, using one or more effective, data-plotting models.

Abstract: Identifying matching transactions. First and second log files contain operation records of transactions in a transaction workload, each file recording a respective execution of the transaction workload, the method comprising. A first record location in the first file and an associated window of a defined number of sequential second record locations in the second file are advanced one record location at a time. Whether each operation record of a complete transaction at a first record location has a matching operation record at one of the record locations in the associated window of second record locations is determined. If so, the complete transaction in the first file and the transaction that includes the matching operation records in the second file are identified as matching transactions.

Abstract: Techniques for running an Extract Transform Load (ETL) job in parallel on one or more processors wherein the ETL job comprises use of an extensible markup language (XML) document are provided. The techniques include receiving an XML document input, identifying a node in the XML document at which partitioning of the XML document is to begin, sending partition information to each respective processor, performing a shallow parsing of the XML document in parallel on the one or more processors, wherein each processor performs shallow parsing using the identified partition node until it reaches its identified partition, using the shallow parsing to generate the partition of the input XML document, wherein each processor generates a different partition of the same XML document, and sending each partition in streaming format to an ETL job instance.

Abstract: A predefined number of matches is identified between records in a first file and records in a second file. For the matches, determine the span of the actual range of record positions in the second file relative to the positions of the operation records in the first file within which all matches were found. If the actual span is smaller than the span of a current defined range of record positions by at least a first threshold value, decrease the span of the current defined range. If the actual span is within a second threshold value of the span of the current defined range, increase the span of the current defined range. If an amount above a third threshold value of operation records in the first file are not matched to operation records in the second file, increasing the span of the current defined range.

Abstract: A log file contains operation records, each operation record is of a certain type, and each operation record is associated with a transaction. A plurality of operation records is read from the log file into a record store. Records of the plurality of operation records of each operation record type are sampled at a predefined sampling rate. Operation records in the plurality of operations records are identified that are associated with completed transactions of which the sampled operation records are associated. The identified operation records are then extracted from the record store into a data store.

Abstract: A log file contains operation records, each operation record is of a certain type, and each operation record is associated with a transaction. A plurality of operation records is read from the log file into a record store. Records of the plurality of operation records of each operation record type are sampled at a predefined sampling rate. Operation records in the plurality of operations records are identified that are associated with completed transactions of which the sampled operation records are associated. The identified operation records are then extracted from the record store into a data store.