Abstract: A method of transforming relational queries of a database into on a data processing system includes receiving a series of relational queries, transforming first parts of the queries into a continuous query embodied as a streaming application, sending parameters in second parts of the queries in the series to the streaming application as a data stream, and executing the continuous query based on the received data stream to generate query results for the series of relational queries. Each query in the series includes a first part and a second part. The first parts are a pattern common to all the queries in the series and the second parts each have one or more parameters that are not common to all of the queries in the series.

Abstract: An Open Database Connectivity (ODBC) proxy infrastructure to transparently route incoming queries to one or more selected query engines. The ODBC proxy receives a query from an application, and determines based on the characteristics of the query and the capabilities of the query engines which one or more query engines are to perform the query. The proxy then routes the query to the one or more query engines, which perform the query. The results are then returned to the proxy, which provides the results to the application.

Abstract: A keyed delay is used in the processing of streaming data to decrease the processing performed and the output provided. A first event, within a particular window, having a particular key starts a delay condition. Arriving events with the same key replace the previous arrival for that key until the delay condition is satisfied. In response thereto, the latest event with that key is output.

Abstract: Techniques for composing an application resource model are disclosed.

Abstract: Techniques for generating a distributed stream processing application are provided. The techniques include obtaining a declarative description of one or more data stream processing tasks, wherein the declarative description expresses at least one stream processing task, and generating one or more execution units from the declarative description of one or more data stream processing tasks, wherein the one or more execution units are deployable across one or more distributed computing nodes, and comprise a distributed data stream processing application.

Abstract: Techniques for generating a distributed stream processing application are provided. The techniques include obtaining a declarative description of one or more data stream processing tasks from a graph of operators, wherein the declarative description expresses at least one stream processing task, generating one or more containers that encompass a combination of one or more stream processing operators, and generating one or more execution units from the declarative description of one or more data stream processing tasks, wherein the one or more execution units are deployable across one or more distributed computing nodes, and comprise a distributed data stream processing application binary.

Abstract: Techniques for dynamically modifying inter-connections between components in an application are provided. The techniques include receiving a data producer profile for each output port within a software application to be executed on one or more processors, receiving a data subscription profile for each input port of each component of the application, establishing connections between the output ports and the input ports of the components in the application based on a comparison of each data producer profile and each data subscription profile, executing the application on one or more processors to process streams of data, receiving either or both of a new data producer profile or a new data subscription profile during the execution of the application, and establishing at least one new connection between an output port and an input port based upon a revised comparison of the received data profiles that include the new data profile.

Abstract: State sharing is facilitated in stream processing environments, including distributed stream processing environments. A customized shared state implementation representing the state to be shared is automatically created based on at least one of user preferences, hints of usage, and system performance.

Abstract: Techniques for composing an application resource model in a data stream processing system are disclosed. The application resource model may be used to understand what resources will be consumed by an application when executed by the data stream processing system. For example, a method for composing an application resource model for a data stream processing system comprises the following steps. One or more operator-level metrics are obtained from an execution of a data stream processing application in accordance with a first configuration. The application is executed by one or more nodes of the data stream processing system, and the application is comprised of one or more processing elements that are comprised of one or more operators. One or more operator-level resource functions are generated based on the obtained one or more operator-level metrics. A processing element-level resource function is generated based on the one or more generated operator-level resource functions.

Abstract: Fault tolerance is provided in a distributed system. The complexity of replicas and rollback requests are avoided; instead, a local failure in a component of a distributed system is tolerated. The local failure is tolerated by storing state related to a requested operation on the component, persisting that stored state in a data store, such as a relational database, asynchronously processing the operation request, and if a failure occurs, restarting the component using the stored state from the data store.

Abstract: An Open Database Connectivity (ODBC) proxy infrastructure to transparently route incoming queries to one or more selected query engines. The ODBC proxy receives a query from an application, and determines based on the characteristics of the query and the capabilities of the query engines which one or more query engines are to perform the query. The proxy then routes the query to the one or more query engines, which perform the query. The results are then returned to the proxy, which provides the results to the application.

Abstract: A method of transforming relational queries of a database into on a data processing system includes receiving a series of relational queries, transforming first parts of the queries into a continuous query embodied as a streaming application, sending parameters in second parts of the queries in the series to the streaming application as a data stream, and executing the continuous query based on the received data stream to generate query results for the series of relational queries. Each query in the series includes a first part and a second part. The first parts are a pattern common to all the queries in the series and the second parts each have one or more parameters that are not common to all of the queries in the series.

Abstract: A method for cross-triggering transitions in independent finite state machines is provided. For a given plurality of finite state machine definitions having a plurality of states and a plurality of transitions among the states, two or more independent instances of the plurality of finite state machine definitions are created. In addition, associations between two or more of the independent finite state machine instances are identified. The method uses cross-triggering rules that identify a condition in a first one of the associated independent finite state machine instances that triggers a transition action in a second one of the associated independent finite state machine instances. Each cross-triggering rule is triggered upon an occurrence of the cross-triggering rule condition, and the transition action in the second associated independent finite state machine instance is performed in response to the triggering of the cross-triggering rule.

Abstract: State sharing is facilitated in stream processing environments, including distributed stream processing environments. A customized shared state implementation representing the state to be shared is automatically created based on at least one of user preferences, hints of usage, and system performance.

Abstract: A method is provided for tracing dataflow in a distributed computing application. For example, the method includes incrementally advancing a dataflow in a dataflow path of one or more dataflow paths according to two or more directives encoded in two or more data messages. The method further includes performing the two or more directives. The dataflow path includes one or more operators including at least one merge operator operative to merge the two or more data messages and merge the two or more directives. One or more of the incrementally advancing of the dataflow and the performing of the two or more directives are implemented as instruction code performed on a processor device.

Abstract: Processing streaming data in a data processing system is facilitated by: declaring and defining, by a processor, a request-response interface as part of a stream processing operator defined using a stream processing language; processing a stream of data using the stream processing operator with the request-response interface defined as a part thereof; and communicating with the stream processing operator through the request-response interface via a communication path separate from the stream of data, the communicating accessing or controlling a state of the stream processing operator while the stream processing operator is processing the stream of data.

Abstract: A method and computer program product for selecting an expression evaluation technique for domain-specific language (DSL) compilation. An application written in DSL for a programming task is provided, the application including a plurality of components configured by expressions. A technique that most quickly implements the programming task is selected from a plurality of techniques for evaluating the expressions. The DSL application is compiled in accordance with the selected expression evaluation technique to generate general-purpose programming language (GPL) code.

Abstract: In one embodiment, the invention comprises partial fault tolerant stream processing applications. One embodiment of a method for implementing partial fault tolerance in a stream processing application comprising a plurality of stream operators includes: defining a quality score function that expresses how well the application is performing quantitatively, injecting a fault into at least one of the plurality of operators, assessing an impact of the fault on the quality score function, and selecting at least one partial fault-tolerant technique for implementation in the application based on the quantitative metric-driven assessment.

Abstract: Data analysis applications include model building components and stream processing components. To increase utility of the data analysis application, in one embodiment, the model building component of the data analysis application is managed. Management includes resource allocation and/or configuration adaptation of the model building component, as examples.

Abstract: A virtual execution environment (VEE) for a streaming Intermediate Language (IL), wherein the streaming IL represents a streaming program, communicates streaming data in queues, stores data-at-rest in variables, and determines data by functions, where inputs are read from the queues and the variables, and outputs are written to the queues and the variables.