Abstract: Herein are machine learning (ML) feature processing and analytic techniques to detect anomalies in parse trees of logic statements, database queries, logic scripts, compilation units of general-purpose programing language, extensible markup language (XML), JAVASCRIPT object notation (JSON), and document object models (DOM). In an embodiment, a computer identifies an operational trace that contains multiple parse trees. Values of explicit features are generated from a single respective parse tree of the multiple parse trees of the operational trace. Values of implicit features are generated from more than one respective parse tree of the multiple parse trees of the operational trace. The explicit and implicit features are stored into a same feature vector. With the feature vector as input, an ML model detects whether or not the operational trace is anomalous, based on the explicit features of each parse tree of the operational trace and the implicit features of multiple parse trees of the operational trace.

Abstract: Techniques described herein propose a ROWID Elimination Rewrite that uses functional dependencies to substitute and/or eliminate ROWID pseudo-columns referenced in a query in order to reduce memory pressure and speed up processing.

Abstract: Techniques described herein propose a new RIDDecode operator in a QEP that uses ROWID lookup and fetch, instead of dictionary decoding, to retrieve decoded values, in order to reduce memory pressure and speed up processing.

Abstract: Techniques are provided for vectorizing Heapsort. A K-heap is used as the underlying data structure for indexing values being sorted. The K-heap is vectorized by storing values in a contiguous memory array containing a beginning-most side and end-most side. The vectorized Heapsort utilizes horizontal aggregation SIMD instructions for comparisons, shuffling, and moving data. Thus, the number of comparisons required in order to find the maximum or minimum key value within a single node of the K-heap is reduced resulting in faster retrieval operations.

Abstract: Approaches herein transparently delegate data access from a relational database management system (RDBMS) onto an offload engine (OE). The RDBMS receives a database statement referencing a user defined function (UDF). In an execution plan, the RDBMS replaces the UDF reference with an invocation of a relational operator in the OE. Execution invokes the relational operator in the OE to obtain a result based on data in the OE. Thus, the UDF is bound to the OE, and almost all of the RDBMS avoids specially handling the UDF. The UDF may be a table function that offloads a relational table for processing. User defined objects such as functions and types provide metadata about the table. Multiple tables can be offloaded and processed together, such that some or all offloaded tables are not materialized in the RDBMS. Offloaded tables may participate in standard relational algebra such as in a database statement.

Abstract: Herein are techniques that extend a software system to embed new guest programing languages (GPLs) that interoperate in a transparent, modular, and configurable way. In embodiments, a computer inserts an implementation of a GPL into a deployment of the system. A command registers the GPL, define subroutines for the GPL, generates a guest virtual environment, and adds a binding of a dependency to a guest module. In an embodiment, a native programing language invokes a guest programing language to cause importing intra- or inter-language dependencies. An embodiment defines a guest object that is implemented in a first GPL and accessed from a second GPL. In an embodiment, dependencies are retrieved from a virtual file system having several alternative implementation mechanisms that include: an archive file or an actual file system, and a memory buffer or a column of a database table.

Abstract: The present invention relates to execution optimization of database queries. Herein are techniques for optimal execution based on query interpretation by translation to a domain specific language (DSL), with optimizations such as partial evaluation, abstract syntax tree (AST) rewriting, just in time (JIT) compilation, dynamic profiling, speculative logic, and Futamura projection. In an embodiment, a database management system (DBMS) that is hosted on a computer generates a query tree that represents a database query that contains an expression that is represented by a subtree of the query tree. The DBMS generates a sequence of DSL instructions that represents the subtree. The sequence of DSL instructions is executed to evaluate the expression during execution of the database query. In an embodiment, an AST is generated from the sequence of DSL instructions. In an embodiment, the DSL AST is optimally rewritten based on a runtime feedback loop that includes dynamic profiling information.

Abstract: Herein are techniques that extend a software system to embed new guest programing languages (GPLs) that interoperate in a transparent, modular, and configurable way. In embodiments, a computer inserts an implementation of a GPL into a deployment of the system. A command registers the GPL, define subroutines for the GPL, generates a guest virtual environment, and adds a binding of a dependency to a guest module. In an embodiment, a native programing language invokes a guest programing language to cause importing intra- or inter-language dependencies. An embodiment defines a guest object that is implemented in a first GPL and accessed from a second GPL. In an embodiment, dependencies are retrieved from a virtual file system having several alternative implementation mechanisms that include: an archive file or an actual file system, and a memory buffer or a column of a database table.

Abstract: Herein are techniques that extend a software system to embed new guest programing languages (GPLs) that interoperate in a transparent, modular, and configurable way. In embodiments, a computer inserts an implementation of a GPL into a deployment of the system. A command registers the GPL, define subroutines for the GPL, generates a guest virtual environment, and adds a binding of a dependency to a guest module. In an embodiment, a native programing language invokes a guest programing language to cause importing intra- or inter-language dependencies. An embodiment defines a guest object that is implemented in a first GPL and accessed from a second GPL. In an embodiment, dependencies are retrieved from a virtual file system having several alternative implementation mechanisms that include: an archive file or an actual file system, and a memory buffer or a column of a database table.

Abstract: Techniques are described herein for influencing plan generation in context of the two phase query optimization approach. Types of pruning criteria including method pruning criteria, total cost pruning criteria, and permutation pruning criteria exist in cost-based plan generators to determine what parts of a query statement should be offloaded to a query offload engine. Method pruning criteria is responsible to determine an optimal joining method. Total cost pruning criteria compares accumulated costs with a lowest plan cost determined so far. Permutation pruning criteria is responsible for selecting the cheapest query execution plan from all considered query execution plans. Each type of pruning criteria is modified to favor offload engine execution upon request.

Abstract: According to an embodiment, a method includes rewriting a particular query to generate a rewritten query. The particular query specifies a window function operator, a particular input to the window function operator, and an analytical function. Rewriting the particular query includes assigning the particular input to an intermediate relation and replacing the window function operator with a replacement operator. The method further includes assigning to the replacement operator an aggregate function corresponding to the analytical function, and the intermediate relation. In this embodiment, the method also includes placing a join operator that joins the intermediate relation with an output of the replacement operator.

Abstract: Approaches herein transparently delegate data access from a relational database management system (RDBMS) onto an offload engine (OE). The RDBMS receives a database statement referencing a user defined function (UDF). In an execution plan, the RDBMS replaces the UDF reference with an invocation of a relational operator in the OE. Execution invokes the relational operator in the OE to obtain a result based on data in the OE. Thus, the UDF is bound to the OE, and almost all of the RDBMS avoids specially handling the UDF. The UDF may be a table function that offloads a relational table for processing. User defined objects such as functions and types provide metadata about the table. Multiple tables can be offloaded and processed together, such that some or all offloaded tables are not materialized in the RDBMS. Offloaded tables may participate in standard relational algebra such as in a database statement.

Abstract: Approaches herein transparently delegate data access from a relational database management system (RDBMS) onto an offload engine (OE). The RDBMS receives a database statement referencing a user defined function (UDF). In an execution plan, the RDBMS replaces the UDF reference with an invocation of a relational operator in the OE. Execution invokes the relational operator in the OE to obtain a result based on data in the OE. Thus, the UDF is bound to the OE, and almost all of the RDBMS avoids specially handling the UDF. The UDF may be a table function that offloads a relational table for processing. User defined objects such as functions and types provide metadata about the table. Multiple tables can be offloaded and processed together, such that some or all offloaded tables are not materialized in the RDBMS. Offloaded tables may participate in standard relational algebra such as in a database statement.

Abstract: Techniques are provided for obtaining generic vectorized d-heaps for any data type for which horizontal aggregation SIMD instructions are not available, including primitive as well as complex data types. A generic vectorized d-heap comprises a prefix heap and a plurality of suffix heaps. Each suffix heap of the plurality of suffix heaps comprises a d-heap. A plurality of key values stored in the heap are split into key prefix values and key suffix values. Key prefix values are stored in the prefix heap and key suffix values are stored in the plurality of suffix heaps. Each entry in the prefix heap includes a key prefix value of the plurality of key values and a reference to the suffix heap of the plurality of suffix heaps that includes all key suffix values of the plurality of key values that share the respective key prefix value.

Abstract: The manner in which tables are joined can affect the outcome of the query and database performance. Example types of join operations include semi-join and inner-join. The techniques described herein are approaches that may be used to substitute a semi-join operator with an inner-join operator and may be used to transform and optimize representations of queries.

Abstract: Techniques are described herein for computing columnar information during join enumeration in a database system. The computation occurs in two phases: the first phase involves a pre-computational phase that is only run once per query block to initialize and prepare a set of data structures. The second phase is an incremental approach that takes place for every query sub-plan. Upon completion of the second phase, the generated projected attributes of a query sub-plan are associated as columnar information associated with the query sub-plan, and used to compute the query execution cost. Subsequently, based on the computed query execution cost, the query sub-plan may be executed as part of the query execution plan.

Abstract: Techniques are provided for vectorizing Heapsort. A K-heap is used as the underlying data structure for indexing values being sorted. The K-heap is vectorized by storing values in a contiguous memory array containing a beginning-most side and end-most side. The vectorized Heapsort utilizes horizontal aggregation SIMD instructions for comparisons, shuffling, and moving data. Thus, the number of comparisons required in order to find the maximum or minimum key value within a single node of the K-heap is reduced resulting in faster retrieval operations.

Abstract: Techniques are described herein for introducing transcode operators into a generated operator tree during query processing. Setting up the transcode operators with correct encoding type at runtime is performed by inferring correct encoding type information during compile time. The inference of the correct encoding type information occurs in three phases during compile time: the first phase involves collecting, consolidating, and propagating the encoding-type information of input columns up the expression tree. The second phase involves pushing the encoding-type information down the tree for nodes in the expression tree that do not yet have any encoding-type assigned. The third phase involves determining which inputs to the current relational operator need to be pre-processed by a transcode operator.

Abstract: Unsorted sparse dictionary encodings are transformed into unsorted-dense or sorted-dense dictionary encodings. Sparse domain codes have large gaps between codes that are adjacent in order. Unlike spare codes, dense codes have smaller gaps between adjacent codes; consecutive codes are dense codes that have no gaps between adjacent codes. The techniques described herein are relational approaches that may be used to generate sparse composite codes and sorted codes.

Abstract: Techniques for maintaining d-heap property and speeding up retrieval operations, such as top or pop, by vectorizing the d-heap and utilizing horizontal aggregation SIMD instructions across the retrieval operations. A d-heap is vectorized by storing it in a contiguous memory array containing a beginning-most side and end-most side. Horizontal aggregation SIMD instructions are utilized to aggregate the values of the vectorized d-heap. Thus, the number of comparisons required in order to find the maximum or minimum key value within a single node of the d-heap is reduced resulting in faster retrieval operations.