Abstract: The present invention relates to optimized access of a database. Herein are techniques to accelerate execution of any combination of ad hoc query, heterogenous hardware, and fluctuating workload. In an embodiment, a computer receives a data access request for data tuples and compiles the data access request into relational operators. A particular implementation of a particular relational operator is dynamically selected from multiple interchangeable implementations. Each interchangeable implementation contains respective physical operators. A particular hardware operator for a particular physical operator is selected from multiple interchangeable hardware operators that include: a first hardware operator that executes on first processing hardware, and a second hardware operator that executes on second processing hardware that is functionally different from the first processing hardware.

Abstract: Methods and apparatuses for determining set-membership using Single Instruction Multiple Data (“SIMD”) architecture are presented herein. Specifically, methods and apparatuses are discussed for determining, in parallel, whether multiple values in a first set of values are members of a second set of values. Many of the methods and systems discussed herein are applied to determining whether one or more rows in a dictionary-encoded column of a database table satisfy one or more conditions based on the dictionary-encoded column. However, the methods and systems discussed herein may apply to many applications executed on a SIMD processor using set-membership tests.

Abstract: Methods and apparatuses for determining set-membership using Single Instruction Multiple Data (“SIMD”) architecture are presented herein. Specifically, methods and apparatuses are discussed for determining, in parallel, whether multiple values in a first set of values are members of a second set of values. Many of the methods and systems discussed herein are applied to determining whether one or more rows in a dictionary-encoded column of a database table satisfy one or more conditions based on the dictionary-encoded column. However, the methods and systems discussed herein may apply to many applications executed on a SIMD processor using set-membership tests.

Abstract: Techniques are described herein for maintaining two copies of the same semi-structured data, where each copy is organized in a different format. One copy is in a first-format that may be convenient for storage, but inefficient for query processing. For example, the first-format may be a textual format that needs to be parsed every time a query needs to access individual data items within a semi-structured object. The database system intelligently loads semi-structured first-format data into volatile memory and, while doing so, converts the semi-structured first-format data to a second-format. Because the data in volatile memory is in the second-format, processing queries against the second-format data both allows disk I/O to be avoided, and increases the efficiency of the queries themselves. For example, the parsing that may be necessary to run a query against a cached copy of the first-format data is avoided.

Abstract: A method, apparatus, and system for OZIP, a data compression and decompression codec, is provided. OZIP utilizes a fixed size static dictionary, which may be generated from a random sampling of input data to be compressed. Compression by direct token encoding to the static dictionary streamlines the encoding and avoids expensive conditional branching, facilitating hardware implementation and high parallelism. By bounding token definition sizes and static dictionary sizes to hardware architecture constraints such as word size or processor cache size, hardware implementation can be made fast and cost effective. For example, decompression may be accelerated by using SIMD instruction processor extensions. A highly granular block mapping in optional stored metadata allows compressed data to be accessed quickly at random, bypassing the processing overhead of dynamic dictionaries. Thus, OZIP can support low latency random data access for highly random workloads, such as for OLTP systems.

Abstract: Techniques herein use in-memory column vectors to process data that is external to a database management system (DBMS) and logically join the external data with data that is native to the DBMS. In an embodiment, a computer maintains a data dictionary for native data that is durably stored in an DBMS and external data that is not durably stored in the DBMS. From a client through a connection to the DBMS, the computer receives a query. The computer loads the external data into an in-memory column vector that resides in random access memory of the DBMS. Based on the query and the data dictionary, the DBMS executes a data join of the in-memory column vector with the native data. To the client through said connection, the computer returns results of the query based on the data join.

Abstract: Techniques are described herein for storing and processing codes included in dictionary-encoded data. In an embodiment, for each respective code of a plurality of codes in the dictionary-encoded data: a plurality of bits from a first portion of the respective code is contiguously stored. One or more bits from a second portion of the respective code is stored in one or more slices. Each respective slice of the one or more slices stores a bit from the one or more bits with a corresponding bit position in the respective code. In another embodiment, a bit-vector is generated based on at least one slice by loading each respective bit of the plurality of bits into different respective partitions in a register at a bit position corresponding to the at least one slice. A plurality of codes may be reconstructed by combining the bit-vector with one or more other bit-vectors.

Abstract: Methods and apparatuses for determining set-membership using Single Instruction Multiple Data (“SIMD”) architecture are presented herein. Specifically, methods and apparatuses are discussed for determining, in parallel, whether multiple values in a first set of values are members of a second set of values. Many of the methods and systems discussed herein are applied to determining whether one or more rows in a dictionary-encoded column of a database table satisfy one or more conditions based on the dictionary-encoded column. However, the methods and systems discussed herein may apply to many applications executed on a SIMD processor using set-membership tests.

Abstract: Methods and apparatuses for determining set-membership using Single Instruction Multiple Data (“SIMD”) architecture are presented herein. Specifically, methods and apparatuses are discussed for determining, in parallel, whether multiple values in a first set of values are members of a second set of values. Many of the methods and systems discussed herein are applied to determining whether one or more rows in a dictionary-encoded column of a database table satisfy one or more conditions based on the dictionary-encoded column. However, the methods and systems discussed herein may apply to many applications executed on a SIMD processor using set-membership tests.

Abstract: A method, apparatus, and system for OZIP, a data compression and decompression codec, is provided. OZIP utilizes a fixed size static dictionary, which may be generated from a random sampling of input data to be compressed. Compression by direct token encoding to the static dictionary streamlines the encoding and avoids expensive conditional branching, facilitating hardware implementation and high parallelism. By bounding token definition sizes and static dictionary sizes to hardware architecture constraints such as word size or processor cache size, hardware implementation can be made fast and cost effective. For example, decompression may be accelerated by using SIMD instruction processor extensions. A highly granular block mapping in optional stored metadata allows compressed data to be accessed quickly at random, bypassing the processing overhead of dynamic dictionaries. Thus, OZIP can support low latency random data access for highly random workloads, such as for OLTP systems.

Abstract: A method, apparatus, and system for OZIP, a data compression and decompression codec, is provided. OZIP utilizes a fixed size static dictionary, which may be generated from a random sampling of input data to be compressed. Compression by direct token encoding to the static dictionary streamlines the encoding and avoids expensive conditional branching, facilitating hardware implementation and high parallelism. By bounding token definition sizes and static dictionary sizes to hardware architecture constraints such as word size or processor cache size, hardware implementation can be made fast and cost effective. For example, decompression may be accelerated by using SIMD instruction processor extensions. A highly granular block mapping in optional stored metadata allows compressed data to be accessed quickly at random, bypassing the processing overhead of dynamic dictionaries. Thus, OZIP can support low latency random data access for highly random workloads, such as for OLTP systems.

Abstract: Techniques are described herein for maintaining two copies of the same semi-structured data, where each copy is organized in a different format. One copy is in a first-format that may be convenient for storage, but inefficient for query processing. For example, the first-format may be a textual format that needs to be parsed every time a query needs to access individual data items within a semi-structured object. The database system intelligently loads semi-structured first-format data into volatile memory and, while doing so, converts the semi-structured first-format data to a second-format. Because the data in volatile memory is in the second-format, processing queries against the second-format data both allows disk I/0 to be avoided, and increases the efficiency of the queries themselves. For example, the parsing that may be necessary to run a query against a cached copy of the first-format data is avoided.

Abstract: Techniques are described herein for storing and processing codes included in dictionary-encoded data. In an embodiment, for each respective code of a plurality of codes in the dictionary-encoded data: a plurality of bits from a first portion of the respective code is contiguously stored. One or more bits from a second portion of the respective code is stored in one or more slices. Each respective slice of the one or more slices stores a bit from the one or more bits with a corresponding bit position in the respective code. In another embodiment, a bit-vector is generated based on at least one slice by loading each respective bit of the plurality of bits into different respective partitions in a register at a bit position corresponding to the at least one slice.

Abstract: A method, apparatus, and system for OZIP, a data compression and decompression codec, is provided. OZIP utilizes a fixed size static dictionary, which may be generated from a random sampling of input data to be compressed. Compression by direct token encoding to the static dictionary streamlines the encoding and avoids expensive conditional branching, facilitating hardware implementation and high parallelism. By bounding token definition sizes and static dictionary sizes to hardware architecture constraints such as word size or processor cache size, hardware implementation can be made fast and cost effective. For example, decompression may be accelerated by using SIMD instruction processor extensions. A highly granular block mapping in optional stored metadata allows compressed data to be accessed quickly at random, bypassing the processing overhead of dynamic dictionaries. Thus, OZIP can support low latency random data access for highly random workloads, such as for OLTP systems.

Abstract: An apparatus generates a debugger script to output first data corresponding to a symbol name for a breakpoint in a software program compiled as optimized code. A debugger script to output second data corresponding to the symbol name for the breakpoint in the software program compiled as unoptimized code is also generated. The apparatus further compares the first data to the second data, and indicates whether there is a difference between the first data and the second data.