Abstract: A method implemented by at least one processing device, a processing device, and a computer program product are provided for adjusting hash partitions for a hash join operation. An amount of rows within each of an initial number of hash partitions is determined during assignment of respective rows to respective hash partitions. The initial number of hash partitions is adjusted to a final number of hash partitions based on the amount of rows within each of the initial number of hash partitions and an amount of available memory. The adjusting further includes determining the final number of hash partitions based on the amount of rows within each of the initial number of hash partitions and the amount of available memory, and assigning the rows to the final number of hash partitions. The hash join operation is then performed.

Abstract: A computer-implemented method includes receiving a query plan. The method includes identifying a plurality of qualified relevant rows and one or more encoding candidate payload columns. The method includes analyzing the relevant rows in the encoding candidate payload columns to yield a count of distinct contents and a payload column width. The method includes estimating a cost and determining whether the cost is larger than an amount of available memory for on-the-fly encoding all of the plurality of encoding candidate payload columns. The method is responsive to the estimated cost being less than the amount of available memory, by on-the-fly encoding the encoding candidates and responsive to the estimated cost being greater than the amount of available memory by on-the-fly encoding fewer than all of the encoding candidates so as not to exceed the available memory, and leaving alone one or more remaining encoding candidate payload columns unencoded.

Abstract: According to one embodiment of the present invention, a system with a plurality of memories, generates a first filter associated with a first memory, and generates one or more second filters each associated with a corresponding memory. Each second filter is generated with a size at least partially accommodated within the storage capacity of the associated memory. The system determines absence of items from a set by selectively probing the generated filters in an ascending order of data access speed to the associated memories and aborts further probing for an item in response to a probe indicating that the item is absent from the set. Embodiments of the present invention further include a method and computer program product for testing set membership in substantially the same manners described above.

Abstract: According to one embodiment of the present invention, a system with a plurality of memories, generates a first filter associated with a first memory, and generates one or more second filters each associated with a corresponding memory. Each second filter is generated with a size at least partially accommodated within the storage capacity of the associated memory. The system determines absence of items from a set by selectively probing the generated filters in an ascending order of data access speed to the associated memories and aborts further probing for an item in response to a probe indicating that the item is absent from the set. Embodiments of the present invention further include a method and computer program product for testing set membership in substantially the same manners described above.

Abstract: A hybrid approach for performing a join in a database includes: obtaining a distribution of data values in a join column of an inner table; using the distribution, identifying one or more data ranges containing skew; for each data range identified as containing skew, performing, by the processor, the join operation using a skew specific join scheme; and for each data range not identified as containing skew, performing, by the processor, the join operation using a non-skew specific join scheme. One skew specific join scheme involves a compact array table, a highly populated array that represents the range of values that the inner table join column contains. One non-skew specific join scheme involves a compact hash table, an optimized hash table that allows high load factors with a small memory overhead. In combining multiple join techniques, joins may be performed more efficiently for skewed and non-skewed data.

Abstract: Partitioned join operations are performed between a first database object and a second database object by determining an agent group for an agent in response to the agent receiving rows of the second database object to process; partitioning the rows to determine a target hash table for each row and adding the partitioned rows to work performed by the agent group; and distributing the work for the group to agents of the group by assigning to a single agent all the rows associated with a particular hash table to perform a join operation on the assigned rows. Each partition is assigned a first counter value indicating an upper bound of a task id range that is most recently assigned to an agent in the agent group for processing, and a second counter value indicating the highest task id that has been processed for that partition.

Abstract: A first table has more than one key column, and a hash array is built for the table, which includes computing hash indices for respective rows of the first table based on more than one key column. Building the hash array also includes writing in the hash array for each row at a respective location in the hash array, wherein each location is indicated by each row's respective hash index computed from the more than one key column. The hash array is probed responsive to the second table. In the building of the hash array for the first table, the writing in the hash array for each row at a respective location in the hash array writes only one of the key columns as a single key field in the hash array for each row.

Abstract: Embodiments include a method and computer program product for encoding data while it is being processed as part of a query is provided. The method includes receiving a query request and determining a set of values associated with data to be encoded for completing the query request. The method also includes encoding those values such that any subsequent processing operations can be performed on the encoded values to complete the requested query. After performing the subsequent processing operations to complete the requested query, each value is decoded back to its original value.

Abstract: A method includes joining data between at least two data sets. Values of one or more join attributes of each of the data sets is represented in a compressed form, indicated by an encoding scheme. A compression scheme for the one or more join attributes is dynamically selected.

Abstract: A method for building a hash table over a subset of data in a data set includes mapping keys in the data set to values in the data set using multiple parallel computation threads. Each thread scans a subset of the keys and values and partitioning the subset of the keys and values into multiple partitions. A cumulative count for keys and values in each partition is determined. A hash table with space reserved for each partition is formed based on the determined cumulative counts. Each thread selects one or more partitions and inserts keys and values belonging to the selected one or more partitions into the hash table in the reserved space for those partitions.

Abstract: Embodiments include a method, system, and computer program product for encoding data while it is being processed as part of a query is provided. The method includes receiving a query request and determining a set of values associated with data to be encoded for completing the query request. The method also includes encoding those values such that any subsequent processing operations can be performed on the encoded values to complete the requested query. After performing the subsequent processing operations to complete the requested query, each value is decoded back to its original value.

Abstract: Embodiments include a system for encoding data while it is being processed. The system includes a processor, an encoder and a decoder. The processor is configured to process a query request by determining a set of values. The encoder is configured for encoding the set of values, such that a subsequent processing operation can be performed on the encoded values. The processor performs the subsequent processing operations. The decoder is configured for decoding each value back to its value prior to being encoded upon completion of the processor completing the requested query.

Abstract: A database table is partitioned with column-major layout by limiting partitioning one or more columns containing join-column values for a join operator. Join operations are executed for joining the partitioned columns.

Abstract: A hardware accelerator includes a bitmap processor that processes a bitmap structure for multiple list population. A population count processor processes population counts for data and aggregates the population counts. The bitmap data structure includes bitmap bit fields interleaved with aggregated population count fields.

Abstract: Embodiments include a method, system, and computer program product for creating an array table. In one embodiment the method includes identifying keys associated with values in a database and identifying bits common between the plurality of keys using logical functions and removing the common bits to form condensed keys. The method also includes modulating the condensed keys using identified common bits to create transformed keys and populating the plurality of array tables using the transformed keys and associated values.

Abstract: Embodiments of the present invention provide a database processing system for efficient partitioning of a database table with column-major layout for executing one or more join operations. One embodiment comprises a method for partitioning a database table with column-major layout, partitioning only the join-columns by limiting the partitions by size and number, executing one or more join operations for joining the partitioned columns, and optionally de-partitioning the join result to the original order by sequentially writing and randomly reading table values using P cursors.

Abstract: Collisions in hash tables are reduced by removing each empty bucket from a hash table and compacting the non-empty buckets, generating a map of the hash table indicating a status of the buckets of the hash table, and accessing data in the hash table by applying a hash key to the generated map to determine a corresponding bucket containing the data.

Abstract: Embodiments include a method, system, and computer program product for creating an array table. In one embodiment the method includes identifying keys associated with values in a database and identifying bits common between the plurality of keys using logical functions and removing the common hits to form condensed keys. The method also includes modulating the condensed keys using identified common bits to create transformed keys and populating the plurality of array tables using the transformed keys and associated values.

Abstract: A hybrid approach for performing a join in a database includes: obtaining a distribution of data values in a join column of an inner table; using the distribution, identifying one or more data ranges containing skew; for each data range identified as containing skew, performing, by the processor, the join operation using a skew specific join scheme; and for each data range not identified as containing skew, performing, by the processor, the join operation using a non-skew specific join scheme. One skew specific join scheme involves a compact array table, a highly populated array that represents the range of values that the inner table join column contains. One non-skew specific join scheme involves a compact hash table, an optimized hash table that allows high load factors with a small memory overhead. In combining multiple join techniques, joins may be performed more efficiently for skewed and non-skewed data.

Abstract: A hybrid approach for performing a join in a database includes: obtaining a distribution of data values in a join column of an inner table; using the distribution, identifying one or more data ranges containing skew; for each data range identified as containing skew, performing, by the processor, the join operation using a skew specific join scheme; and for each data range not identified as containing skew, performing, by the processor, the join operation using a non-skew specific join scheme. One skew specific join scheme involves a compact array table, a highly populated array that represents the range of values that the inner table join column contains. One non-skew specific join scheme involves a compact hash table, an optimized hash table that allows high load factors with a small memory overhead. In combining multiple join techniques, joins may be performed more efficiently for skewed and non-skewed data.