Abstract: An event processing system for processing events in an event stream is disclosed. The system can launch a first CQL engine in a cluster of CQL engines using a CQL engine tracking engine. The system can schedule, using the CQL engine tracking engine, the first CQL engine to process a batch of a continuous stream of input events related to an application. The system can track, using the CQL engine tracking engine, the first CQL engine to be scheduled for execution. The system can then execute, using the CQL engine tracking engine, the first CQL engine to process the batch of the continuous stream of input events to generate a set of output events related to the application.

Abstract: An event processing system for processing events in an event stream is disclosed. The system can execute instructions to receive a micro-batch stream of input events, process the input events using the CQL engine to generate a set of output events, generate, using a snapshot management algorithm implemented by the CQL engine, a snapshot of a current state of a system based at least in part on the set of output events related to the application, generate a first directory structure to access snapshot information associated with the snapshot of the current state of the system, generate a second directory structure to generate a list of snapshots associated with the current state of the system, and determine based at least in part on the snapshot management algorithm, a process to get, add, or clean the list of snapshots associated with the current state of the system.

Abstract: An event processing system for processing events in an event stream is disclosed. The system can execute instructions to receive a continuous data stream related to an application, identify an input geometry associated with the continuous data stream, generate a cluster of geometries based at least in part on the input geometry, generate an output geometry based at least in part on the cluster of geometries and a number of geometries in each cluster of the cluster of geometries, determining one or more partitions for the output geometry based on the cluster of geometries and the number of geometries in each cluster of the cluster of geometries, dynamically change a size of the one or more partitions for the output geometry, and transmitting the output geometry associated with the continuous data stream.

Abstract: An event processing system for processing events in an event stream is disclosed. The system can execute instructions to receive a continuous query, apply a transformation to the continuous query to generate a query plan for the continuous query, monitor the continuous query using a monitoring transformation process, receive a micro-batch stream of input events related to an application, and process the input events of the micro-batch stream based at least in part on the monitoring transformation process to generate a set of output events related to the application.

Abstract: A distributed event processing system is disclosed that receives a batch of events via a continuous data stream and performs the serialization of data in the batch of events. In certain embodiments, the system identifies a first data type of a first attribute for each event in a batch of events and determines a first type of data compression to be performed on data values represented by the first attribute. The system determines a first type of data compression to be performed on data values represented by the first attribute based on the first data type of the first attribute. The system then generates a first set of serialized data values for the first attribute. The system processes the first set of serialized data values against a set of one or more continuous queries to generate a first set of output events.

Abstract: An event processing system for processing events in an event stream is disclosed. The system is configured for configuring a stream processor to micro-batch incoming events from a stream source. The system is also configured for generating a single timestamp for a micro-batch of the incoming events and/or receiving the micro-batch of the incoming events from the stream source. The system can also be configured for assigning the single timestamp to each event of the micro-batch and/or generating separate timestamp values for each respective event of the micro-batch. In some examples, the system can also be configured for assigning, for each respective event of the micro-batch, an individual one of the separate timestamp values.

Abstract: An event processing system for processing events in an event stream is disclosed. The system can execute instructions to receive a continuous query, apply a transformation to the continuous query to generate a query plan for the continuous query, transform the query plan using a transformation algorithm to generate a transformed query plan, receive a micro-batch stream of input events related to an application, process the input events of the micro-batch stream based at least in part on the transformed query plan to generate a set of output events related to the application, and store the set of output events related to the application in an output queue.

Abstract: An event processing system for processing events in an event stream is disclosed. The system is configured for determining a stage for a continuous query language (CQL) query being processed by an event processing system and/or determining a stage type associated with the stage. The system is also configured for determining a transformation to be computed for the stage based at least in part on the stage type and/or determining a classification for the CQL query based at least in part on a plurality of rules. The system can also be configured for generating a transformation in a Directly Acyclic Graph (DAG) of a data transformation pipeline for the stage based at least in part on the partitioning criteria for the stage. In some examples, the system can also be configured for determining a partitioning of the stage based at least in part on the transformation.

Abstract: Some event ordering requirements can be determined based on continuous event processing queries. Other event ordering requirements can be determined based on distribution flow types being used to distribute events from event streams to node executing the queries. Events from event streams can be ordered according to ordering semantics that are based on a combination of all of these event ordering requirements. Additionally, virtual computing nodes can be associated with constraints, and computing processors can be associated with capabilities. Virtual computing nodes for processing event streams can be assigned to execute on various computing processors based on both these constraints and capabilities. Additionally, for each of several events in an event stream, a ratio between a total latency and a communication latency can be for determined. Based on an average of these ratios, a quantity of reducing nodes that will be involved in a map-reduce operation can be selected.

Abstract: Embodiments of the present invention provide for supporting definition of services in the SOA using templates for definition of processes, tasks, and/or components and wizards associated to the templates for guiding users in defining values to variables in the templates. Embodiments of the present invention provide developers with a guided flow for an integration project as well as guidance from the functional decomposition from the solution architect within an integrated development environment. Embodiments provide pre-built templates that may also contain wizard based interfaces to help the developers to create services that follow best-practices and include necessary plumbing tasks.

Abstract: Systems and methods for guaranteeing the event order for multi-stage processing in distributed systems are disclosed. In some examples, a warm-up period is used to accurately determine a starting point for ordered events of an event stream. Skip-beats may be utilized as dummy events so that the event processor does not wait too long for events that were filtered out at earlier stages.

Abstract: Systems and methods for query decomposition for scalability of continuous query processing are disclosed. In some examples, it is determined whether a query is fully stateful. If it is, the query may be divided into two functions, an aggregate function and a global function. The aggregate functions may be distributed across multiple distributed nodes and executed in parallel. The global function may then receive the results from the distributed nodes and join the results to achieve the fully stateful result.

Abstract: Systems and methods for extracting tuples using dynamically generated extractor classes are disclosed. In some examples, an optimized tuple extraction class can be dynamically generated to enable more efficient tuple extraction.

Abstract: Systems and methods for guaranteeing the event order for multi-stage processing in distributed systems are disclosed. In some examples, a warm-up period is used to accurately determine a starting point for ordered events of an event stream. Skip-beats may be utilized as dummy events so that the event processor does not wait too long for events that were filtered out at earlier stages.

Abstract: An event processing system is disclosed that processes events of an event stream, performs the recovery of events during system failure and preserves the state of the system reliably and accurately while achieving desired system performance. In an embodiment, the event processing system processes a first batch of events of a continuous input stream of events using a continuous query and generates an output stream of events related to an application. The event processing system identifies one or more operators of the continuous query and determines that an operator is a journaled operator. The event processing system generates a journaled snapshot of a current state of the system based on execution of the journaled operator on at least the first batch of events and stores the journaled snapshot of the current state of the system.

Abstract: An event processing system is disclosed that processes events of an event stream and performs the recovery of events during system failure. The recovery of events is performed by generating output sequence numbers for each deterministic output event generated by the event processing system. In an embodiment, the event processing system determines a current output sequence number of a most recently transmitted output event in a output stream of events, a last output sequence number of an output event corresponding to a most recently processed batch of input events and determines a set of one or more output events of the output stream to be transmitted based on the current sequence number and the last output sequence number. The event processing system then transmits the output events related to the application.

Abstract: Techniques for encoding node locations are provided. In some examples, a node in a hierarchy such as a document object model (DOM) may be expressed as a location path from the root node of the hierarchy to the node. Each node on the location path may be encoded using a mapping between the type and/or index of the node and a set of characters such as ASCII characters. The encoding of each node on the path be concatenated to form the encoded location path. The encoding seeks to minimize the length of the encoded location path. For example, in an embodiment, a single ASCII character may be used to encode both the type and the index of a node. In an embodiment, breakpoint locations may be encoded using methods provided to reduce storage and computing resource required.

Abstract: Some event ordering requirements can be determined based on continuous event processing queries. Other event ordering requirements can be determined based on distribution flow types being used to distribute events from event streams to node executing the queries. Events from event streams can be ordered according to ordering semantics that are based on a combination of all of these event ordering requirements. Additionally, virtual computing nodes can be associated with constraints, and computing processors can be associated with capabilities. Virtual computing nodes for processing event streams can be assigned to execute on various computing processors based on both these constraints and capabilities. Additionally, for each of several events in an event stream, a ratio between a total latency and a communication latency can be for determined. Based on an average of these ratios, a quantity of reducing nodes that will be involved in a map-reduce operation can be selected.

Abstract: Some event ordering requirements can be determined based on continuous event processing queries. Other event ordering requirements can be determined based on distribution flow types being used to distribute events from event streams to node executing the queries. Events from event streams can be ordered according to ordering semantics that are based on a combination of all of these event ordering requirements. Additionally, virtual computing nodes can be associated with constraints, and computing processors can be associated with capabilities. Virtual computing nodes for processing event streams can be assigned to execute on various computing processors based on both these constraints and capabilities. Additionally, for each of several events in an event stream, a ratio between a total latency and a communication latency can be for determined. Based on an average of these ratios, a quantity of reducing nodes that will be involved in a map-reduce operation can be selected.

Abstract: Some event ordering requirements can be determined based on continuous event processing queries. Other event ordering requirements can be determined based on distribution flow types being used to distribute events from event streams to node executing the queries. Events from event streams can be ordered according to ordering semantics that are based on a combination of all of these event ordering requirements. Additionally, virtual computing nodes can be associated with constraints, and computing processors can be associated with capabilities. Virtual computing nodes for processing event streams can be assigned to execute on various computing processors based on both these constraints and capabilities. Additionally, for each of several events in an event stream, a ratio between a total latency and a communication latency can be for determined. Based on an average of these ratios, a quantity of reducing nodes that will be involved in a map-reduce operation can be selected.