Abstract: A workflow includes a sequence of tools and a logic controller. A data file is input into the workflow and processed by the tools. A log file and a metadata file are generated for a tool. The log file includes data describing events that occurred during the operation of the tool in the processing of the data file. The metadata file includes data generated from the performance of the data analytics function of the tool. The logic controller uses the log file to determine whether to trigger a subsequent tool. Responsive to a determination of trigging the subsequent tool, the metadata file is sent to the subsequent tool and processed by a subset of the sequence of tools to generate a result item. The subset starts with the subsequent tool. The result item represents an outcome of the processing of the data file by the workflow.

Abstract: A data analytics system stores a plurality of sequential data blocks in a block cache. Each data block contains one or more data records. Each data record of a data record contains one or more values and a record marker marking the end of the data record. The data analytics system allocates a data block in the block cache to a worker thread. The worker thread parses the data block to identify the one or more data records contained within the data block. The data analytics system determines whether the data block was correctly parsed. Responsive to determining that the data block was correctly parsed, the data analytics system provides the one or more data records contained within the data block to a data analytics workflow.

Abstract: A module manages usage of a memory in a computer by scheduling data processing tasks. The module determines whether a memory associated with a computer is available for performing a data processing task. In response to determining that the memory is not available for performing the data processing task, the module generates a factory object for the data processing task. The factory object specifies instructions to produce the data processing task. After the factory object is generated, the module receives a request to perform the data processing task. The module produces the data processing task by using the factory object in response to the request and determine again whether the memory is available for performing the data processing task. In response to determining that the memory is now available for performing the data processing task, the module performs the data processing task by using the memory.

Abstract: A module manages memory in a computer. The module monitors usage of a primary memory associated with the computer. The primary memory stores memory blocks in a ready state. In response to primary memory usage by the memory blocks in the ready state exceeding a ready state threshold, the module compresses at least some of the memory blocks in the ready state to form memory blocks in a ready and compressed state. In response to primary memory usage by the memory blocks in the ready and compressed state exceeding a release threshold, the module releases at least some of the memory blocks in the ready and compressed state. In response to primary memory usage by the memory blocks in the compressed state exceeding a compressed threshold, the module transfers at least some memory blocks in the compressed state to a secondary memory associated with the computer.

Abstract: A data analytics system stores a plurality of sequential data blocks in a block cache. Each data block contains one or more data records. Each data record of a data record contains one or more values and a record marker marking the end of the data record. The data analytics system allocates a data block in the block cache to a worker thread. The worker thread parses the data block to identify the one or more data records contained within the data block. The data analytics system determines whether the data block was correctly parsed. Responsive to determining that the data block was correctly parsed, the data analytics system provides the one or more data records contained within the data block to a data analytics workflow.

Abstract: A module manages memory in a computer. The module monitors usage of a primary memory associated with the computer. The primary memory stores memory blocks in a ready state. In response to primary memory usage by the memory blocks in the ready state exceeding a ready state threshold, the module compresses at least some of the memory blocks in the ready state to form memory blocks in a ready and compressed state. In response to primary memory usage by the memory blocks in the ready and compressed state exceeding a release threshold, the module releases at least some of the memory blocks in the ready and compressed state. In response to primary memory usage by the memory blocks in the compressed state exceeding a compressed threshold, the module transfers at least some memory blocks in the compressed state to a secondary memory associated with the computer.

Abstract: A module manages memory in a computer. The module monitors usage of a primary memory associated with the computer. The primary memory stores memory blocks in a ready state. In response to primary memory usage by the memory blocks in the ready state exceeding a ready state threshold, the module compresses at least some of the memory blocks in the ready state to form memory blocks in a ready and compressed state. In response to primary memory usage by the memory blocks in the ready and compressed state exceeding a release threshold, the module releases at least some of the memory blocks in the ready and compressed state. In response to primary memory usage by the memory blocks in the compressed state exceeding a compressed threshold, the module transfers at least some memory blocks in the compressed state to a secondary memory associated with the computer.

Abstract: A data analytics system stores a plurality of sequential data blocks in a block cache. Each data block contains one or more data records. Each data record of a data record contains one or more values and a record marker marking the end of the data record. The data analytics system allocates a data block in the block cache to a worker thread. The worker thread parses the data block to identify the one or more data records contained within the data block. The data analytics system determines whether the data block was correctly parsed. Responsive to determining that the data block was correctly parsed, the data analytics system provides the one or more data records contained within the data block to a data analytics workflow.

Abstract: A data analytics system stores a plurality of sequential data blocks in a block cache. Each data block contains one or more data records. Each data record of a data record contains one or more values and a record marker marking the end of the data record. The data analytics system allocates a data block in the block cache to a worker thread. The worker thread parses the data block to identify the one or more data records contained within the data block. The data analytics system determines whether the data block was correctly parsed. Responsive to determining that the data block was correctly parsed, the data analytics system provides the one or more data records contained within the data block to a data analytics workflow.

Abstract: A module manages memory in a computer. The module monitors usage of a primary memory associated with the computer. The primary memory stores memory blocks in a ready state. In response to primary memory usage by the memory blocks in the ready state exceeding a ready state threshold, the module compresses at least some of the memory blocks in the ready state to form memory blocks in a ready and compressed state. In response to primary memory usage by the memory blocks in the ready and compressed state exceeding a release threshold, the module releases at least some of the memory blocks in the ready and compressed state. In response to primary memory usage by the memory blocks in the compressed state exceeding a compressed threshold, the module transfers at least some memory blocks in the compressed state to a secondary memory associated with the computer.