Abstract: An intermediate representation of a workflow of one or more modules may be generated to decouple language implementations of the one or more modules. In response to receiving a workflow of one or more modules, the workflow may be analyzed to determine an optimal implementation language for each of the one or more modules to thereby reduce effects of data marshalling. An intermediate representation of the workflow that is configured to decouple any implementation languages associated with the one or more modules may be generated. To allow for decoupling, the intermediate representation may be written in a declarative language. The generated intermediate representation may then be compiled to generate an executable program that corresponds to the workflow and is implemented in the determined optimal language for each of the one or more modules.

Abstract: Accommodating a particular kind of out-of-order data; namely, when data arrives out-of-order because the data is generated in systems that may have different application timelines. As data arrives, it arrives with a particular partition key. For instance, the partition key might identify the source of the data. The data from each partition key is kept in-order with respect to all other data from that same partition key. That said, data from one key is permitted to have a different timeline as compared to data from any other key. Thus, the data may not be in-order when viewed globally across keys. Rather, data is tolerated to be out-of-order globally so long as the lag in the arrived data is within some tolerated lag. If the data arrives having a time that exceeds the maximum tolerated lag, then the system applies some policy to determine what to do with the delayed data.

Abstract: An intermediate representation of a workflow of one or more modules may be generated to decouple language implementations of the one or more modules. In response to receiving a workflow of one or more modules, the workflow may be analyzed to determine an optimal implementation language for each of the one or more modules to thereby reduce effects of data marshalling. An intermediate representation of the workflow that is configured to decouple any implementation languages associated with the one or more modules may be generated. To allow for decoupling, the intermediate representation may be written in a declarative language. The generated intermediate representation may then be compiled to generate an executable program that corresponds to the workflow and is implemented in the determined optimal language for each of the one or more modules.

Abstract: Accommodating a particular kind of out-of-order data; namely, when data arrives out-of-order because the data is generated in systems that may have different application timelines. As data arrives, it arrives with a particular partition key. For instance, the partition key might identify the source of the data. The data from each partition key is kept in-order with respect to all other data from that same partition key. That said, data from one key is permitted to have a different timeline as compared to data from any other key. Thus, the data may not be in-order when viewed globally across keys. Rather, data is tolerated to be out-of-order globally so long as the lag in the arrived data is within some tolerated lag. If the data arrives having a time that exceeds the maximum tolerated lag, then the system applies some policy to determine what to do with the delayed data.