Abstract: A conversational analytics system may provide for a conversational interface to any relational database. A controlled natural language may be constructed in an automated manner from a given database (e.g., from schema and values associated with a relational database). For instance, a user natural language expression may be converted to an expression in the constructed controlled natural language and the controlled natural language expression may be converted into a sequence of one or more queries in a query language (e.g., queries in structured query language (SQL)). Such an intermediate controlled natural language may provide queries without ambiguity (e.g., as each expression or phrase in the controlled natural language may be mapped to one sequence of SQL queries). Accordingly, any natural language user utterance that ultimately follows the controlled natural language may be automatically converted into a sequence of one or more SQL queries sent to the database.

Abstract: A conversational analytics system may provide for a conversational interface to any relational database. A controlled natural language may be constructed in an automated manner from a given database (e.g., from schema and values associated with a relational database). For instance, a user natural language expression may be converted to an expression in the constructed controlled natural language and the controlled natural language expression may be converted into a sequence of one or more queries in a query language (e.g., queries in structured query language (SQL)). Such an intermediate controlled natural language may provide queries without ambiguity (e.g., as each expression or phrase in the controlled natural language may be mapped to one sequence of SQL queries). Accordingly, any natural language user utterance that ultimately follows the controlled natural language may be automatically converted into a sequence of one or more SQL queries sent to the database.

Abstract: A keyed join is used in the processing of streaming data to streamline processing to provide higher throughput and decreased use of resources. The most recent event for each unique replacement key value(s) is maintained substituting older events with the same key. An incoming event is joined with the data received from one or more other data sources, and the correlations are output.

Abstract: Data analysis applications include model building components and stream processing components. To increase utility of the data analysis application, in one embodiment, the model building component of the data analysis application is managed. Management includes resource allocation and/or configuration adaptation of the model building component, as examples.

Abstract: One embodiment of the present method and apparatus adaptive in-operator load shedding includes receiving at least two data streams (each comprising a plurality of tuples, or data items) into respective sliding windows of memory. A throttling fraction is then calculated based on input rates associated with the data streams and on currently available processing resources. Tuples are then selected for processing from the data streams in accordance with the throttling fraction, where the selected tuples represent a subset of all tuples contained within the sliding window.

Abstract: An embodiment of the invention provides a method for exploiting stateless and stateful data parallelism in a streaming application, wherein a compiler determines whether an operator of the streaming application is safe to parallelize based on a definition of the operator and an instance of the definition. The operator is not safe to parallelize when the operator has selectivity greater than 1, wherein the selectivity is the number of output tuples generated for each input tuple. Parallel regions are formed within the streaming application with the compiler when the operator is safe to parallelize. Synchronization strategies for the parallel regions are determined with the compiler, wherein the synchronization strategies are determined based on the definition of the operator and the instance of the definition. The synchronization strategies of the parallel regions are enforced with a runtime system.

Abstract: An embodiment of the invention provides a method for exploiting stateless and stateful data parallelism in a streaming application, wherein a compiler determines whether an operator of the streaming application is safe to parallelize based on a definition of the operator and an instance of the definition. The operator is not safe to parallelize when the operator has selectivity greater than 1, wherein the selectivity is the number of output tuples generated for each input tuple. Parallel regions are formed within the streaming application with the compiler when the operator is safe to parallelize. Synchronization strategies for the parallel regions are determined with the compiler, wherein the synchronization strategies are determined based on the definition of the operator and the instance of the definition. The synchronization strategies of the parallel regions are enforced with a runtime system.

Abstract: Techniques are disclosed for adaptive source filtering and load shedding in such data stream processing systems. For example, in one aspect of the invention, a method for use in filtering data in a distributed data stream processing system, wherein a server receives and processes one or more data streams from one or more data sources, comprises the steps of the server periodically re-configuring one or more filters and sending the one or more periodically re-configured filters to the one or more data sources, and the one or more data sources performing data filtering based on the one or more periodically re-configured filters received from the server.

Abstract: Techniques for composing an application resource model are disclosed.

Abstract: A method for adjusting a data parallel region of a stream processing application includes measuring congestion of each parallel channel of the data parallel region, measuring a total throughput of all the parallel channels, and adjusting the number of parallel channels based on the current measured congestion and throughput.

Abstract: A method for adjusting a data parallel region of a stream processing application includes measuring congestion of each parallel channel of the data parallel region, measuring a total throughput of all the parallel channels, and adjusting the number of parallel channels based on the current measured congestion and throughput.

Abstract: An embodiment of the invention provides a method for adaptive auto-pipelining of a stream processing application, wherein the stream processing application includes one or more threads. Runtime of the stream processing application is initiated with a stream processing application manager. The stream processing application is monitored with a monitoring module during the runtime, wherein the monitoring of the stream processing application includes identifying threads in the stream processing application that execute operators in a data flow graph, and determining an amount of work that each of the threads are performing on operators of the logical data flow graph. A processor identifies one or more operators in the data flow graph to add one or more additional threads based on the monitoring of the stream processing application during the runtime.

Abstract: Embodiments of the disclosure include a method for providing stream processing with runtime adaptation includes registering one or more events, wherein each of the events is associated with a stream processing application. The method also includes monitoring, by a processor, for an occurrence of the one or more events associated with the stream processing application, wherein each of the one or more events is associated with one or more runtime metrics. The method further includes receiving an event notification, wherein the event notification includes event identification and an event context and executing an adaptation of the stream processing application.

Abstract: Embodiments of the disclosure include a system for providing stream processing with runtime adaptation, having a stream processing application that receives an incoming data stream and a runtime infrastructure configured to execute the stream processing application. The system also includes an orchestrator configured to communicate with the runtime infrastructure and the stream processing application, the orchestrator configured to perform a method. The method includes registering one or more events, wherein each of the events is associated with a stream processing application. The method also includes monitoring, by a processor, for an occurrence of the one or more events associated with the stream processing application, wherein each of the one or more events is associated with one or more runtime metrics.

Abstract: In one embodiment, the invention comprises partial fault tolerant stream processing applications. One embodiment of a method for implementing partial fault tolerance in a stream processing application comprising a plurality of stream operators includes: defining a quality score function that expresses how well the application is performing quantitatively, injecting a fault into at least one of the plurality of operators, assessing an impact of the fault on the quality score function, and selecting at least one partial fault-tolerant technique for implementation in the application based on the quantitative metric-driven assessment.

Abstract: Techniques for optimizing data stream processing are provided. The techniques include employing a pattern, wherein the pattern facilitates splitting of one or more incoming streams and distributing processing across one or more operators, obtaining one or mote operators, wherein the one or more operators support at least one group-independent aggregation and join operation on one or more streams, generating code, wherein the code facilitates mapping of the application onto a computational infrastructure to enable workload partitioning, using the one or more operators to decompose each of the application into one or more granular components, and using the code to reassemble the one or more granular components into one or more deployable blocks to map the application to a computational infrastructure, wherein reassembling the one or more granular components to map the application to the computational infrastructure optimizes data stream processing of the application.

Abstract: In one embodiment, the invention is a method and apparatus for failure recovery for stream processing applications. One embodiment of a method for providing a failure recovery mechanism for a stream processing application includes receiving source code for the stream processing application, wherein the source code defines a fault tolerance policy for each of the components of the stream processing application, and wherein respective fault tolerance policies defined for at least two of the plurality of components are different, generating a sequence of instructions for converting the state(s) of the component(s) into a checkpoint file comprising a sequence of storable bits on a periodic basis, according to a frequency defined in the fault tolerance policy, initiating execution of the stream processing application, and storing the checkpoint file, during execution of the stream processing application, at a location that is accessible after failure recovery.

Abstract: One embodiment of a method for constructing executable code for a component-based application includes receiving a request to compile source code for the component-based application, wherein the request identifies the source code, and wherein the source code comprises a plurality of source code components, each of the source code components implementing a different component of the application, and performing a series of steps for each source code component where the series of steps includes: deriving a signature for the source code component, retrieving a stored signature corresponding to a currently available instance of executable code for the source code component, comparing the derived signature with the stored signature, compiling the source code component into the executable code when the derived signature does not match the stored signature, and obtaining the executable code for the source code component from a repository when the derived signature matches the stored signature.

Abstract: An information processing system, computer readable storage medium, and method for documenting the execution of long running applications. A processor of the information processing system operates to continuously collect snapshots monitored from an executing application on a runtime system. The processor selects, based at least on user specified policies, collected snapshots that indicate a significant change in runtime behavior of the executing application on the runtime system. The processor annotates each of the selected snapshots with description of the significant change in runtime behavior of the executing application. The processor operates to store and update documentation of an execution history of the executing application from the annotated snapshots.

Abstract: A method for adjusting a data parallel region of a stream processing application includes measuring congestion of each parallel channel of the data parallel region, measuring a total throughput of all the parallel channels, and adjusting the number of parallel channels based on the current measured congestion and throughput.