Abstract: Architecture introduces a new pattern operator referred to as called an augmented transition network (ATN), which is a streaming adaptation of non-reentrant, fixed-state ATNs for dynamic patterns. Additional user-defined information is associated with automaton states and is accessible to transitions during execution. ATNs are created that directly model complex pattern continuous queries with arbitrary cycles in a transition graph. The architecture can express the desire to ignore some events during pattern detection, and can also detect the absence of data as part of a pattern. The architecture facilitates efficient support for negation, ignorable events, and state cleanup based on predicate punctuations.

Abstract: A streaming operator assignment system and method for determining a streaming operator assignment that minimizes overload in a data processing system. Embodiments of the streaming operator assignment system include an optimization goals definition module, which defines optimization goals in terms of fundamental quantities that system administrators and application writers want to control, such as minimizing the worst case latency over all periods of time, or minimizing how much the system is backlogged with work. Embodiments of the streaming operator assignment system also include an optimization goals solution module that optimizes and solves a selected optimization goal. A specialized optimization technique is used to find the best operator (or load) assignment using the optimization goals to measure of the value of the assignment.

Abstract: Parameterized queries are optimized by a transformational optimizer. The optimizer produces a dynamic plan that embeds multiple plan options that may be selected to execute a particular query. Parameter distribution improves query execution efficiency and performance by exploring a sample parameter space representative of the parameter values actually used. The dynamic plans can be simplified while maintaining an acceptable level of optimality by reducing the number of plan options. The reduction is achieved by eliminating switch unions to alternatives that are close in cost. Both approaches of parameter space exploration and dynamic plan generation are deeply integrated into the query optimizer.

Abstract: The described implementations relate to recursive streaming queries. One technique processes a recursive streaming query through a query graph. The technique also detects when output produced by executing the query graph advances to a specific point.

Abstract: A streaming operator assignment system and method for determining a streaming operator assignment that minimizes overload in a data processing system. Embodiments of the streaming operator assignment system include an optimization goals definition module, which defines optimization goals in terms of fundamental quantities that system administrators and application writers want to control, such as minimizing the worst case latency over all periods of time, or minimizing how much the system is backlogged with work. Embodiments of the streaming operator assignment system also include an optimization goals solution module that optimizes and solves a selected optimization goal. A specialized optimization technique is used to find the best operator (or load) assignment using the optimization goals to measure of the value of the assignment.

Abstract: A new approach for handling stream imperfections based on speculative execution involves the retraction of incorrect events facilitated using operators to remove speculatively produced incorrect output. Additionally, parameters are disclosed that define a spectrum of consistency levels. A first parameter, maximum blocking time, exposes a tradeoff between a degree of speculation and latency. A second parameter, the maximum time data is remembered before being purged from the system, exposes a tradeoff between state size and correctness. Varying these two parameters produces a spectrum of consistency levels (e.g., strong, middle, weak) which address the specific tradeoffs built into other systems. Retraction is accomplished using operators that include Select, AlterLifetime, Join, Sum, Align, and Finalize.

Abstract: Disclosed is a temporal stream model that provides support both for query language semantics and consistency guarantees, simultaneously. A data stream is modeled as a time varying relation. The data stream model incorporates a temporal data perspective, and defines a clear separation in different notions of time in streaming applications. The temporal stream model further refines the conventional application time into two temporal dimensions of valid time and occurrence time, and utilizes system time (the clock of the stream processor) for modeling out-of-order event delivery but thereby providing three temporal dimensions. The methods for assigning timestamps and quantifying latency form the basis for defining a spectrum of consistency levels. Based on the selected consistency level, an output can be produced. The utilization of system time facilitates the retraction of incorrect output and the insertion of the correct revised output.

Abstract: Parameterized queries are optimized by a transformational optimizer. The optimizer produces a dynamic plan that embeds multiple plan options that may be selected to execute a particular query. Parameter distribution improves query execution efficiency and performance by exploring a sample parameter space representative of the parameter values actually used. The dynamic plans can be simplified while maintaining an acceptable level of optimality by reducing the number of plan options. The reduction is achieved by eliminating switch unions to alternatives that are close in cost. Both approaches of parameter space exploration and dynamic plan generation are deeply integrated into the query optimizer.

Abstract: Prior to searching a multidimensional feature space populated with data objects, each dimension in the feature space is divided into a number of intervals. When a query is received, a single interval that is overlapped by the query is selected from each dimension. A reduced set of data objects is then selected that includes only those data objects that overlap the selected intervals. This reduced set of data objects, rather than the entire set of data objects in the feature space, is then used to determine matches for the query.

Abstract: Prior to searching a multidimensional feature space populated with data objects, each dimension in the feature space is divided into a number of intervals. When a query is received, a single interval that is overlapped by the query is selected from each dimension. A reduced set of data objects is then selected that includes only those data objects that overlap the selected intervals. This reduced set of data objects, rather than the entire set of data objects in the feature space, is then used to determine matches for the query.

Abstract: Prior to searching a multidimensional feature space populated with data objects, each dimension in the feature space is divided into a number of intervals. When a query is received, a single interval that is overlapped by the query is selected from each dimension. A reduced set of data objects is then selected that includes only those data objects that overlap the selected intervals. This reduced set of data objects, rather than the entire set of data objects in the feature space, is then used to determine matches for the query.