Abstract: Herein are techniques that extend a software system to embed new guest programing languages (GPLs) that interoperate in a transparent, modular, and configurable way. In embodiments, a computer inserts an implementation of a GPL into a deployment of the system. A command registers the GPL, define subroutines for the GPL, generates a guest virtual environment, and adds a binding of a dependency to a guest module. In an embodiment, a native programing language invokes a guest programing language to cause importing intra- or inter-language dependencies. An embodiment defines a guest object that is implemented in a first GPL and accessed from a second GPL. In an embodiment, dependencies are retrieved from a virtual file system having several alternative implementation mechanisms that include: an archive file or an actual file system, and a memory buffer or a column of a database table.

Abstract: According to an embodiment, a method includes rewriting a particular query to generate a rewritten query. The particular query specifies a window function operator, a particular input to the window function operator, and an analytical function. Rewriting the particular query includes assigning the particular input to an intermediate relation and replacing the window function operator with a replacement operator. The method further includes assigning to the replacement operator an aggregate function corresponding to the analytical function, and the intermediate relation. In this embodiment, the method also includes placing a join operator that joins the intermediate relation with an output of the replacement operator.

Abstract: Approaches herein transparently delegate data access from a relational database management system (RDBMS) onto an offload engine (OE). The RDBMS receives a database statement referencing a user defined function (UDF). In an execution plan, the RDBMS replaces the UDF reference with an invocation of a relational operator in the OE. Execution invokes the relational operator in the OE to obtain a result based on data in the OE. Thus, the UDF is bound to the OE, and almost all of the RDBMS avoids specially handling the UDF. The UDF may be a table function that offloads a relational table for processing. User defined objects such as functions and types provide metadata about the table. Multiple tables can be offloaded and processed together, such that some or all offloaded tables are not materialized in the RDBMS. Offloaded tables may participate in standard relational algebra such as in a database statement.

Abstract: Approaches herein transparently delegate data access from a relational database management system (RDBMS) onto an offload engine (OE). The RDBMS receives a database statement referencing a user defined function (UDF). In an execution plan, the RDBMS replaces the UDF reference with an invocation of a relational operator in the OE. Execution invokes the relational operator in the OE to obtain a result based on data in the OE. Thus, the UDF is bound to the OE, and almost all of the RDBMS avoids specially handling the UDF. The UDF may be a table function that offloads a relational table for processing. User defined objects such as functions and types provide metadata about the table. Multiple tables can be offloaded and processed together, such that some or all offloaded tables are not materialized in the RDBMS. Offloaded tables may participate in standard relational algebra such as in a database statement.

Abstract: Techniques are provided for obtaining generic vectorized d-heaps for any data type for which horizontal aggregation SIMD instructions are not available, including primitive as well as complex data types. A generic vectorized d-heap comprises a prefix heap and a plurality of suffix heaps. Each suffix heap of the plurality of suffix heaps comprises a d-heap. A plurality of key values stored in the heap are split into key prefix values and key suffix values. Key prefix values are stored in the prefix heap and key suffix values are stored in the plurality of suffix heaps. Each entry in the prefix heap includes a key prefix value of the plurality of key values and a reference to the suffix heap of the plurality of suffix heaps that includes all key suffix values of the plurality of key values that share the respective key prefix value.

Abstract: The manner in which tables are joined can affect the outcome of the query and database performance. Example types of join operations include semi-join and inner-join. The techniques described herein are approaches that may be used to substitute a semi-join operator with an inner-join operator and may be used to transform and optimize representations of queries.

Abstract: Techniques are provided for vectorizing Heapsort. A K-heap is used as the underlying data structure for indexing values being sorted. The K-heap is vectorized by storing values in a contiguous memory array containing a beginning-most side and end-most side. The vectorized Heapsort utilizes horizontal aggregation SIMD instructions for comparisons, shuffling, and moving data. Thus, the number of comparisons required in order to find the maximum or minimum key value within a single node of the K-heap is reduced resulting in faster retrieval operations.

Abstract: Computer-implemented techniques described herein provide efficient data decoding using runtime specialization. In an embodiment, a method comprises a virtual machine executing a body of code of a dynamically typed language, wherein executing the body of code includes: querying a relational database, and in response to the query, receiving table metadata indicating data types of one or more columns of a first table in the relational database. In response to receiving the table metadata: for a first column of the one or more columns, generating decoding machine code to decode the first column based on the data type of the first column, and executing the decoding machine code to decode the first column of the one or more columns.

Abstract: Techniques for maintaining d-heap property and speeding up retrieval operations, such as top or pop, by vectorizing the d-heap and utilizing horizontal aggregation SIMD instructions across the retrieval operations. A d-heap is vectorized by storing it in a contiguous memory array containing a beginning-most side and end-most side. Horizontal aggregation SIMD instructions are utilized to aggregate the values of the vectorized d-heap. Thus, the number of comparisons required in order to find the maximum or minimum key value within a single node of the d-heap is reduced resulting in faster retrieval operations.

Abstract: Computer-implemented techniques described herein provide efficient data decoding using runtime specialization. In an embodiment, a method comprises a virtual machine executing a body of code of a dynamically typed language, wherein executing the body of code includes: querying a relational database, and in response to the query, receiving table metadata indicating data types of one or more columns of a first table in the relational database. In response to receiving the table metadata: for a first column of the one or more columns, generating decoding machine code to decode the first column based on the data type of the first column, and executing the decoding machine code to decode the first column of the one or more columns.

Abstract: Computer-implemented techniques described herein provide fast access to structured, semi-structured, and unstructured data using a virtual machine that provides support for dynamic code generation. In an embodiment, a virtual machine allows for the efficient encoding of data objects into hierarchically marked up data. A virtual machine is configured to dynamically create attributes of objects referenced by a body of code and hidden classes used to store attributes of objects referenced by the body of code. The virtual machine will determine whether a hidden class of an object satisfies one or more encoding criteria for encoding instances of objects with the hidden class. Encoding machine code is generated and executed to encode instances of objects with the hidden class. In another embodiment, direct structure decoding enables the generation of efficient machine code that is specialized for accessing only a subset of the input data that is used by an application.

Abstract: Techniques related to code dictionary generation based on non-blocking operations are disclosed. In some embodiments, a column of tokens includes a first token and a second token that are stored in separate rows. The column of tokens is correlated with a set of row identifiers including a first row identifier and a second row identifier that is different from the first row identifier. Correlating the column of tokens with the set of row identifiers involves: storing a correlation between the first token and the first row identifier, storing a correlation between the second token and the second row identifier if the first token and the second token have different values, and storing a correlation between the second token and the first row identifier if the first token and the second token have identical values. After correlating the column of tokens with the set of row identifiers, duplicate correlations are removed.

Abstract: Herein are computerized techniques for deploying JavaScript and TypeScript stored procedures and user-defined functions into a database management system (DBMS). In an embodiment, a computer generates a SQL call specification for each subroutine of one or more subroutines encoded in a scripting language. The generating is based on a signature declaration of the subroutine. Each subroutine comprises a definition of a stored procedure or a user-defined function. The computer packages the definition and the SQL call specification of each subroutine into a single bundle file. The definition and the SQL call specification of each subroutine are deployed into a DBMS from the single bundle file. Eventually, the SQL call specification of at least one subroutine is invoked to execute the definition of the subroutine in the DBMS.

Abstract: Computer-implemented techniques described herein provide fast access to structured, semi-structured, and unstructured data using a virtual machine that provides support for dynamic code generation. In an embodiment, a virtual machine allows for the efficient encoding of data objects into hierarchically marked up data. A virtual machine is configured to dynamically create attributes of objects referenced by a body of code and hidden classes used to store attributes of objects referenced by the body of code. The virtual machine will determine whether a hidden class of an object satisfies one or more encoding criteria for encoding instances of objects with the hidden class. Encoding machine code is generated and executed to encode instances of objects with the hidden class. In another embodiment, direct structure decoding enables the generation of efficient machine code that is specialized for accessing only a subset of the input data that is used by an application.

Abstract: Systems and methods disclosed herein are used to store and interact with data from heterogeneous data sources. In some embodiments, a method of improved querying of data includes: identifying a data value in a first data source of a plurality of heterogeneous data sources. The method further includes: (i) extracting, from the first data source, one or more characteristics associated with the data value, (ii) creating a cell that includes the data value and the extracted one or more characteristics, and (iii) storing, in an unordered collection of cells, the created cell. The method also includes: receiving a query that specifies a set of one or more required characteristics. In accordance with a determination that the created cell includes each required characteristic in the set, the method includes locating the created cell and presenting, on an electronic device's display, a tabular representation that at least includes the data value.

Abstract: A method for updating a block on disk is provided. The method involves one or more clients committing log records to queues corresponding to blocks. The method further involves checkpointing, which includes obtaining a flushing lock on the queue by a client, receiving log records from the queue, applying the log records to a block on disk, and deleting the log records from the queue. A block on disk may be updated by first applying the log records from a queue to a locally cached version of the block corresponding to the queue. The locally cached version may then be stored on the disk replacing the original block on disk or being stored as a new version of the original block.