Abstract: An apparatus includes a processor to: based on data dependencies specified in a job flow definition, identify first and second tasks of the corresponding job flow to be performed sequentially, wherein the first task outputs a data object used as an input to the second; store, within a task queue, at least one message conveying at least an identifier of the first task, and an indication that the data object is to be exchanged through a shared memory space; within a task container, in response to storage of the at least one message within the task queue, sequentially execute first and second task routines to sequentially perform the first and second tasks, respectively, and instantiate the shared memory space to be accessible to the first and second task routines during their executions; and upon completion of the job flow, transmit an indication of completion to another device via a network.

Abstract: An apparatus includes a processor to: use an identifier of a requesting device or operator thereof to identify federated area(s) to which access is authorized; based on data dependencies among a set of tasks of a job flow, derive an order of performance specifying the first task to be performed; store, within a task queue, a task routine execution request message including an identifier associated with the first task, and federated area identifier(s) of the identified federated area(s); within a resolver container, in response to storage of the task routine execution request message, use the identifier associated with the first task and identifier(s) of the federated area(s) to identify one in which a first task routine is stored; within a task container, execute the first task routine to perform the first task; and upon completion of the job flow, transmit an indication of completion to the requesting device.

Abstract: An apparatus includes at least on processor to: parse a job flow definition for a job flow to identify an implicit expression of a data dependency arising from a data object output by a first task of the job flow and input to a second task thereof, wherein the first task is referred to as a data object required by the second as an input; in response to identifying the implicit expression, derive an order of performance of the tasks of the job flow that includes performing the first task before the second to ensure generation of the data object prior to performance of the second; for each task, retrieve a corresponding task routine; execute the task routines in an order that follows the order of performance of the tasks; and transmit, to the requesting device via the network, an indication of successful performance of the job flow.

Abstract: An apparatus includes a processor to: based on data dependencies specified in a job flow definition, identify first and second tasks of the corresponding job flow to be performed sequentially, wherein the first task outputs a data object used as an input to the second; store, within a task queue, at least one message conveying at least an identifier of the first task, and an indication that the data object is to be exchanged through a shared memory space; within a task container, in response to storage of the at least one message within the task queue, sequentially execute first and second task routines to sequentially perform the first and second tasks, respectively, and instantiate the shared memory space to be accessible to the first and second task routines during their executions; and upon completion of the job flow, transmit an indication of completion to another device via a network.

Abstract: An apparatus includes a processor to: use an identifier of a requesting device or operator thereof to identify federated area(s) to which access is authorized; based on data dependencies among a set of tasks of a job flow, derive an order of performance specifying the first task to be performed; store, within a task queue, a task routine execution request message including an identifier associated with the first task, and federated area identifier(s) of the identified federated area(s); within a resolver container, in response to storage of the task routine execution request message, use the identifier associated with the first task and identifier(s) of the federated area(s) to identify one in which a first task routine is stored; within a task container, execute the first task routine to perform the first task; and upon completion of the job flow, transmit an indication of completion to the requesting device.

Abstract: An apparatus includes at least on processor to: parse a job flow definition for a job flow to identify an implicit expression of a data dependency arising from a data object output by a first task of the job flow and input to a second task thereof, wherein the first task is referred to as a data object required by the second as an input; in response to identifying the implicit expression, derive an order of performance of the tasks of the job flow that includes performing the first task before the second to ensure generation of the data object prior to performance of the second; for each task, retrieve a corresponding task routine; execute the task routines in an order that follows the order of performance of the tasks; and transmit, to the requesting device via the network, an indication of successful performance of the job flow.

Abstract: An apparatus includes a processor to: receive a request to generate a superset job flow replacing multiple job flows including an output job flow and preceding job flows previously performed to generate an output data object; identify a first subset of mid-flow data object(s) generated by preceding job flow(s) as input(s) to the output job flow to generate the output data object; identify a second subset of the mid-flow data object(s) generated by preceding job flow(s) as input(s) to other preceding job flow(s) generating the first subset; in response to a lack of a second subset, derive the superset job flow and/or corresponding DAG to include at least one task of the output job flow and at least one task of each preceding job flow that generated the first subset; and transmit an indication of the generation of the superset job flow.

Abstract: An apparatus includes a processor to: receive a job flow definition; retrieve the most recent versions of a set of task routines for the defined job flow; translate, into an intermediate representation, executable instructions of each task routine implementing an interface for data input and/or output during execution; translate executable instructions of the job flow definition that defines the interface for each task routine into an intermediate representation; compare each intermediate representation from a task routine to the corresponding intermediate representation from the job flow definition to determine if there is a match; and in response to there being a match for each comparison and to the executable instructions of the job flow definition being written in a secondary programming language, translate the executable instructions of the job flow definition into a primary programming language, and store the resulting translated form of the job flow definition in a federated area.

Abstract: An apparatus includes a processor to: receive a request to generate a superset job flow replacing multiple job flows including an output job flow and preceding job flows previously performed to generate an output data object; identify a first subset of mid-flow data object(s) generated by preceding job flow(s) as input(s) to the output job flow to generate the output data object; identify a second subset of the mid-flow data object(s) generated by preceding job flow(s) as input(s) to other preceding job flow(s) generating the first subset; in response to a lack of a second subset, derive the superset job flow and/or corresponding DAG to include at least one task of the output job flow and at least one task of each preceding job flow that generated the first subset; and transmit an indication of the generation of the superset job flow.

Abstract: An apparatus includes a processor to: receive a job flow definition; retrieve the most recent versions of a set of task routines for the defined job flow; translate, into an intermediate representation, executable instructions of each task routine implementing an interface for data input and/or output during execution; translate executable instructions of the job flow definition that defines the interface for each task routine into an intermediate representation; compare each intermediate representation from a task routine to the corresponding intermediate representation from the job flow definition to determine if there is a match; and in response to there being a match for each comparison and to the executable instructions of the job flow definition being written in a secondary programming language, translate the executable instructions of the job flow definition into a primary programming language, and store the resulting translated form of the job flow definition in a federated area.

Abstract: An apparatus includes a processor to: receive a job flow definition; retrieve the most recent versions of a set of task routines for the defined job flow; translate, into an intermediate representation, executable instructions of each task routine implementing an interface for data input and/or output during execution; translate executable instructions of the job flow definition that defines the interface for each task routine into an intermediate representation; compare each intermediate representation from a task routine to the corresponding intermediate representation from the job flow definition to determine if there is a match; and in response to there being a match for each comparison and to the executable instructions of the job flow definition being written in a secondary programming language, translate the executable instructions of the job flow definition into a primary programming language, and store the resulting translated form of the job flow definition in a federated area.

Abstract: An apparatus includes a processor to: receive a job flow definition; retrieve the most recent versions of a set of task routines for the defined job flow; translate, into an intermediate representation, executable instructions of each task routine implementing an interface for data input and/or output during execution; translate executable instructions of the job flow definition that defines the interface for each task routine into an intermediate representation; compare each intermediate representation from a task routine to the corresponding intermediate representation from the job flow definition to determine if there is a match; and in response to there being a match for each comparison and to the executable instructions of the job flow definition being written in a secondary programming language, translate the executable instructions of the job flow definition into a primary programming language, and store the resulting translated form of the job flow definition in a federated area.

Abstract: An apparatus includes a processor to: receive a request for a federated area (FA) package including a FA storing a job flow definition; retrieve an instance log of a past performance that includes identifiers of the job flow definition, a data object employed as an input to a past performance, and a task routine executed in the past performance; employ an identifier to identify an FA storing a data object employed as an input; employ an identifier to identify an FA storing an executed task routine; generate the FA package to include the FAs storing the job flow definition, the instance log, the data object and the task routine, and include a copy of each in its respective federated area within the FA package; include an integrity value for each FA in the FA package; and transmit the FA package to the requesting device.

Abstract: An apparatus includes a processor to: receive a request for a federated area (FA) package including a FA storing a job flow definition; retrieve an instance log of a past performance that includes identifiers of the job flow definition, a data object employed as an input to a past performance, and a task routine executed in the past performance; employ an identifier to identify an FA storing a data object employed as an input; employ an identifier to identify an FA storing an executed task routine; generate the FA package to include the FAs storing the job flow definition, the instance log, the data object and the task routine, and include a copy of each in its respective federated area within the FA package; include an integrity value for each FA in the FA package; and transmit the FA package to the requesting device.

Abstract: An apparatus includes a processor to: receive a request from a first remote device to provide a second remote device with access to an existing federated area; allocate storage for a new private federated accessible to the second remote device and not to the first remote device; maintain an inheritance relationship between the existing and new private federated areas in which an object stored in the existing federated area is made accessible to the second remote device to the same extent as an object in the new private federated area; and maintain a priority relationship between the existing and new private federated areas as an exception to the inheritance relationship such that priority is given to providing the second remote device with access to a task routine stored in the new private federated area over a task routine stored in the existing federated area that performs the same task.

Abstract: An apparatus includes a processor to: grant a first device access to a first federated area, but not a second device; grant the second device access to a second federated area, but not the first device; grant the first and second devices access to a base federated area; maintain inheritance relationships among the federated areas so an object stored in the base federated area is as accessible to the first device as an object in the first federated area and is as accessible to the second device as an object in the second federated area; and maintain priority relationships among the federated areas so priority is given to providing the first device access to a task routine stored in the first federated area, and to providing the second device access to a task routine stored in the second federated area, over a task routine stored in the base federated area.

Abstract: An apparatus includes a processor and a storage storing instructions causing the processor to: maintain a federated area; receive a request to perform a job flow with a data set from a remote device; retrieve a job flow definition specifying the tasks of the job flow from the federated area; determine whether there is an instance log in the federated area generated by a previous performance of the job flow with the data set; in response to there being such an instance log, retrieve the version specified in the instance log of each task routine for each task from the federated area; in response to there being no such instance log, retrieve the most recent version of each task routine; perform the job flow with the retrieved versions of the task routines and the data set to generate a result report; and provide the result report to the remote device.

Abstract: An apparatus includes a processor to: receive a request from a first remote device to provide a second remote device with access to an existing federated area; allocate storage for a new private federated accessible to the second remote device and not to the first remote device; maintain an inheritance relationship between the existing and new private federated areas in which an object stored in the existing federated area is made accessible to the second remote device to the same extent as an object in the new private federated area; and maintain a priority relationship between the existing and new private federated areas as an exception to the inheritance relationship such that priority is given to providing the second remote device with access to a task routine stored in the new private federated area over a task routine stored in the existing federated area that performs the same task.

Abstract: An apparatus includes a processor to: grant a first device access to a first federated area, but not a second device; grant the second device access to a second federated area, but not the first device; grant the first and second devices access to a base federated area; maintain inheritance relationships among the federated areas so an object stored in the base federated area is as accessible to the first device as an object in the first federated area and is as accessible to the second device as an object in the second federated area; and maintain priority relationships among the federated areas so priority is given to providing the first device access to a task routine stored in the first federated area, and to providing the second device access to a task routine stored in the second federated area, over a task routine stored in the base federated area.

Abstract: An apparatus includes a processor and a storage storing instructions causing the processor to: maintain a federated area; receive a request to perform a job flow with a data set from a remote device; retrieve a job flow definition specifying the tasks of the job flow from the federated area; determine whether there is an instance log in the federated area generated by a previous performance of the job flow with the data set; in response to there being such an instance log, retrieve the version specified in the instance log of each task routine for each task from the federated area; in response to there being no such instance log, retrieve the most recent version of each task routine; perform the job flow with the retrieved versions of the task routines and the data set to generate a result report; and provide the result report to the remote device.