Abstract: A first table has more than one key column, and a hash array is built for the table, which includes computing hash indices for respective rows of the first table based on more than one key column. Building the hash array also includes writing in the hash array for each row at a respective location in the hash array, wherein each location is indicated by each row's respective hash index computed from the more than one key column. The hash array is probed responsive to the second table. In the building of the hash array for the first table, the writing in the hash array for each row at a respective location in the hash array writes only one of the key columns as a single key field in the hash array for each row.

Abstract: A method creating a compact hash table includes a thread executing using a processor for building a linear probe hash table based on allocating space for an array of keys. Payloads are inserted into the linear probe hash table. Keys from the array of keys are inserted into the linear probe hash table with linear probing. A linear probing bitmap is generated based on occupied locations in the linear probe hash table. A compact hash table is created based on the linear probe hash table and the linear probing bitmap.

Abstract: A computer program product provides efficient multiple aggregation distinct processing. The computer program product including a computer readable storage medium having program instructions embodied therewith. The program instructions are executable by a processor to cause the processor to convert a query with multiple aggregation distincts into a two-stage group-by process using a nest operator. The two-stage group-by process including further program instructions to cause the processor to: perform a first stage group-by process including the nest operator taking a single input row, and concatenating a grouping key with a measure value for each aggregation distinct that forms multiple inputs to a single group-by table, and perform a second stage group-by process including the nest operator bringing together entries for each original group.

Abstract: A method implemented by at least one processing device, a processing device, and a computer program product are provided for adjusting hash partitions for a hash join operation. An amount of rows within each of an initial number of hash partitions is determined during assignment of respective rows to respective hash partitions. The initial number of hash partitions is adjusted to a final number of hash partitions based on the amount of rows within each of the initial number of hash partitions and an amount of available memory. The adjusting further includes determining the final number of hash partitions based on the amount of rows within each of the initial number of hash partitions and the amount of available memory, and assigning the rows to the final number of hash partitions. The hash join operation is then performed.

Abstract: To manage memory in a multi-processing system, a memory budget is assigned to each of a number of agents within the multi-processing system. A portion of memory is allocated to each the agents within the memory budget. Metrics are collected for each agent during processing of data by the agents; the metrics include an amount of data processed and an amount of memory used for each agent. Memory efficiency is determined for each agent based on the collected metrics and another memory budget is determined based on the memory efficiency. The portion of the memory is reallocated to the agents within the other memory budget in response to data stored in the memory relative to the assigned memory budget meeting a criterion.

Abstract: Embodiments include a method, system, and computer program product for encoding data while it is being processed as part of a query is provided. The method includes receiving a query request and determining a set of values associated with data to be encoded for completing the query request. The method also includes encoding those values such that any subsequent processing operations can be performed on the encoded values to complete the requested query. After performing the subsequent processing operations to complete the requested query, each value is decoded back to its original value.

Abstract: A system for sorting in a column organized relational database by sorting only key columns in the column organized relational database to reduce memory and CPU usage. The sort may be performed using an in memory sort. After sorting the key columns, the system orders a payload column according to an order indicated by the key columns. The payload column is associated with the key columns in the column organized relational database. The system assesses memory capacity prior to ordering the payload column. Prior to ordering a plurality of payload columns, the system refrains from automatically concatenating the plurality of payload columns, and refrains from automatically de-concatenating any concatenated payload columns after the ordering the plurality of payload columns. The system determines an amount of required memory necessary to order the payload column, and detects an amount of available memory accessible to order the payload column.

Abstract: A system for sorting in a column organized relational database by sorting only key columns in the column organized relational database to reduce memory and CPU usage. The sort may be performed using an in memory sort. After sorting the key columns, the system orders a payload column according to an order indicated by the key columns. The payload column is associated with the key columns in the column organized relational database. The system assesses memory capacity prior to ordering the payload column. Prior to ordering a plurality of payload columns, the system refrains from automatically concatenating the plurality of payload columns, and refrains from automatically de-concatenating any concatenated payload columns after the ordering the plurality of payload columns. The system determines an amount of required memory necessary to order the payload column, and detects an amount of available memory accessible to order the payload column.

Abstract: A first table has more than one key column, and a hash array is built for the table, which includes computing hash indices for respective rows of the first table based on more than one key column. Building the hash array also includes writing in the hash array for each row at a respective location in the hash array, wherein each location is indicated by each row's respective hash index computed from the more than one key column. The hash array is probed responsive to the second table. In the building of the hash array for the first table, the writing in the hash array for each row at a respective location in the hash array writes only one of the key columns as a single key field in the hash array for each row.

Abstract: A hybrid approach for performing a join in a database includes: obtaining a distribution of data values in a join column of an inner table; using the distribution, identifying one or more data ranges containing skew; for each data range identified as containing skew, performing, by the processor, the join operation using a skew specific join scheme; and for each data range not identified as containing skew, performing, by the processor, the join operation using a non-skew specific join scheme. One skew specific join scheme involves a compact array table, a highly populated array that represents the range of values that the inner table join column contains. One non-skew specific join scheme involves a compact hash table, an optimized hash table that allows high load factors with a small memory overhead. In combining multiple join techniques, joins may be performed more efficiently for skewed and non-skewed data.

Abstract: A method, a system, and a computer program product are provided. A filter is created for each portion of a data set. The filter indicates which one or more characteristics are present among each of the portions. Each of the one or more characteristics comprises one or more groups defined by the data grouping operation. The filters for the portions of the data sets are transferred to one or more filter processors and combined within the one or more filter processors to indicate characteristics of data residing across multiple processing elements to produce a result for a data grouping operation, utilizing transfers based on a combined filter result. In various embodiments, the filter may be a Bloom filter.

Abstract: A method, a system, and a computer program product are provided. A filter is created for each portion of a data set. The filter indicates which one or more characteristics are present among each of the portions. Each of the one or more characteristics comprises one or more groups defined by the data grouping operation. The filters for the portions of the data sets are transferred to one or more filter processors and combined within the one or more filter processors to indicate characteristics of data residing across multiple processing elements to produce a result for a data grouping operation, utilizing transfers based on a combined filter result. In various embodiments, the filter may be a Bloom filter.

Abstract: A first table has more than one key column, and a hash array is built for the table, which includes computing hash indices for respective rows of the first table based on more than one key column. Building the hash array also includes writing in the hash array for each row at a respective location in the hash array, wherein each location is indicated by each row's respective hash index computed from the more than one key column. The hash array is probed responsive to the second table. In the building of the hash array for the first table, the writing in the hash array for each row at a respective location in the hash array writes only one of the key columns as a single key field in the hash array for each row.

Abstract: A method creating a compact hash table includes a thread executing using a processor for building a linear probe hash table based on allocating space for an array of keys. Payloads are inserted into the linear probe hash table. Keys from the array of keys are inserted into the linear probe hash table with linear probing. A linear probing bitmap is generated based on occupied locations in the linear probe hash table. A compact hash table is created based on the linear probe hash table and the linear probing bitmap.

Abstract: A method for creating a compact hash table includes a thread executing using a processor for scanning a subset of keys and values, and for each key, performing a hash operation and inserting hashed keys into a bitmap structure. Cumulative population counts of keys and values are determined within the bitmap. The method repeats scanning the subset of the keys and values. The keys and values are inserted into a compacted array using the cumulative population counts.

Abstract: A method for building a hash table over a subset of data in a data set includes partitioning a subset of keys and values in the data set into multiple partitions. A hash table is formed with space reserved for each partition based on cumulative counts for a number of keys and values in each partition of the multiple partitions. Each thread selects one or more partitions and inserts keys and values belonging to the selected one or more partitions into the hash table in the reserved space for those partitions. A compact hash table is created that includes a bitmap and a compacted army.

Abstract: A database table is partitioned with column-major layout by limiting partitioning one or more columns containing join-column values for a join operator. Join operations are executed for joining the partitioned columns.

Abstract: A method includes joining data between at least two data sets. Values of one or more join attributes of each of the data sets is represented in a compressed form, indicated by an encoding scheme. A compression scheme for the one or more join attributes is dynamically selected.

Abstract: A computer-implemented method includes receiving a query plan. The method includes identifying a plurality of qualified relevant rows and one or more encoding candidate payload columns. The method includes analyzing the relevant rows in the encoding candidate payload columns to yield a count of distinct contents and a payload column width. The method includes estimating a cost and determining whether the cost is larger than an amount of available memory for on-the-fly encoding all of the plurality of encoding candidate payload columns. The method is responsive to the estimated cost being less than the amount of available memory, by on-the-fly encoding the encoding candidates and responsive to the estimated cost being greater than the amount of available memory by on-the-fly encoding fewer than all of the encoding candidates so as not to exceed the available memory, and leaving alone one or more remaining encoding candidate payload columns unencoded.

Abstract: A computer-implemented method includes receiving a query plan. The method includes identifying a plurality of qualified relevant rows and one or more encoding candidate payload columns. The method includes analyzing the relevant rows in the encoding candidate payload columns to yield a count of distinct contents and a payload column width. The method includes estimating a cost and determining whether the cost is larger than an amount of available memory for on-the-fly encoding all of the plurality of encoding candidate payload columns. The method is responsive to the estimated cost being less than the amount of available memory, by on-the-fly encoding the encoding candidates and responsive to the estimated cost being greater than the amount of available memory by on-the-fly encoding fewer than all of the encoding candidates so as not to exceed the available memory, and leaving alone one or more remaining encoding candidate payload columns unencoded.