Abstract: A distributed system with multiple nodes, each of which processes input data streams according to a set of independent, asynchronous processing components connected in a manner represented by a directed graph, is dynamically updated. Such a dynamic update occurs while the system is processing input data by pausing the input data stream, making changes to the system configuration, and then restarting the input data stream. If the update fails for any reason, the system configuration is rolled back to its prior state.

Abstract: Messages are processed to guarantee proper work scheduling and allow for cyclic and conditional processing. A processing node requiring ordered processing of messages is configured to operate within a plurality of successive processing nodes not requiring ordering processing of messages. Nodes upstream of the ordered node (i) monitor skew between the messages being processed and messages that are ordered after the messages still being processed and that have been forwarded downstream of the node and (ii) pause processing upon reaching skew limits. The nodes upstream of the ordered node limit output of messages to be within a delta of other messages as defined by message IDs. In addition, messages removed from a flow path are replaced by Null messages for ordered processing. Child messages are encoded to incorporate parent IDs preserving ordering systems.

Abstract: A data flow processing system represents processing by a directed graph including processing nodes that generate, and process, groups of dependent messages and that can process such groups of messages in order. To identify a group of messages, the system applies message identifiers to messages. Processing of a message may generate child messages. A child message is assigned a message identifier that incorporates the message identifier of the parent message. The message identifier of the parent message is annotated to indicate the number of related child messages. The processing node maintains a buffer to store messages in the group. When a message is received, its message identifier indicates whether it is in a group and information about the group. From this information the processing node determines whether all messages within the group have been received, at which time the processing node can process the messages in order.

Abstract: A data flow processing system represents processing by a directed graph including processing nodes that generate, and process, groups of dependent messages and that can process such groups of messages in order. To identify a group of messages, the system applies message identifiers to messages. Processing of a message may generate child messages. A child message is assigned a message identifier that incorporates the message identifier of the parent message. The message identifier of the parent message is annotated to indicate the number of related child messages. The processing node maintains a buffer to store messages in the group. When a message is received, its message identifier indicates whether it is in a group and information about the group. From this information the processing node determines whether all messages within the group have been received, at which time the processing node can process the messages in order.

Abstract: A highly parallel, asynchronous data flow processing system in which processing is represented by a directed graph model, can include processing nodes that generate, and process, groups of dependent messages and that process messages within such groups in order. Other messages can be processed in whatever order they are received by a processing node. To identify a group of dependent messages, message identifiers are applied to a message. Processing of a message may generate child messages. A child message is assigned a message identifier that incorporates the associated message identifier of the parent message. The message identifier of the parent message is annotated to indicate the number of related child messages. When a group of messages is to be processed by a processing node in order, the processing node maintains a buffer in which messages in the group are stored.

Abstract: A highly parallel, asynchronous data flow processing system in which processing is represented by a directed graph model, can include processing nodes that generate, and process, groups of dependent messages and that process messages within such groups in order. Other messages can be processed in whatever order they are received by a processing node. To identify a group of dependent messages, message identifiers are applied to a message. Processing of a message may generate child messages. A child message is assigned a message identifier that incorporates the associated message identifier of the parent message. The message identifier of the parent message is annotated to indicate the number of related child messages. When a group of messages is to be processed by a processing node in order, the processing node maintains a buffer in which messages in the group are stored.

Abstract: Messages are processed to guarantee proper work scheduling and allow for cyclic and conditional processing. A processing node requiring ordered processing of messages is configured to operate within a plurality of successive processing nodes not requiring ordering processing of messages. Nodes upstream of the ordered node (i) monitor skew between the messages being processed and messages that are ordered after the messages still being processed and that have been forwarded downstream of the node and (ii) pause processing upon reaching skew limits. The nodes upstream of the ordered node limit output of messages to be within a delta of other messages as defined by message IDs. In addition, messages removed from a flow path are replaced by Null messages for ordered processing. Child messages are encoded to incorporate parent IDs preserving ordering systems.