Abstract: A method includes performing, by a data structure processor, concurrent read and write operations into a hierarchical data structure. Writers acquire latches on the hierarchical data structure elements that the latches modify. The hierarchical data structure elements are directly accessed by readers without acquiring latches. A modify operation is executed by a writer for one or more levels of the hierarchical data structure. When removed portions of the hierarchical data structure are no longer referenced, tracking is performed by use of a combination of a global state value and a copied local state value. The global state value transitions through a non-repeating sequence of values. No longer referenced portions of the hierarchical data structure are tagged with the current global state value.

Abstract: A computer-implemented method includes receiving a query plan. The method includes identifying a plurality of qualified relevant rows and one or more encoding candidate payload columns. The method includes analyzing the relevant rows in the encoding candidate payload columns to yield a count of distinct contents and a payload column width. The method includes estimating a cost and determining whether the cost is larger than an amount of available memory for on-the-fly encoding all of the plurality of encoding candidate payload columns. The method is responsive to the estimated cost being less than the amount of available memory, by on-the-fly encoding the encoding candidates and responsive to the estimated cost being greater than the amount of available memory by on-the-fly encoding fewer than all of the encoding candidates so as not to exceed the available memory, and leaving alone one or more remaining encoding candidate payload columns unencoded.

Abstract: A first request may be received to update a first set of values. The first set of values may be stored at a first location within a first data page of a database. The first location may be read-only. In response to the receiving of the first request, a first set of records may be inserted into a second data page. The first set of records may include the update of the first set of values. In response to the inserting, a forward pointer may be stored in the first data page that points to the first set of records on the second data page. One or more committed values may be identified on the second data page. In response to the identifying, the one or more committed values may be merged from the second data page to a third data page.

Abstract: One embodiment provides for monotonic transactions in a multi-master database with loosely coupled nodes including performing, by a processor, a write transaction protocol including: a client device issuing write transactions at any of the nodes, recording a write transaction locally at a node that issues a write transaction and asynchronously replicating the write transaction to at least one other node, and waiting for reception of an acknowledgment from at least a quorum of the nodes before returning a response to the client device. The quorum is any set of the nodes that includes a special node and at least one other node.

Abstract: To manage memory in a multi-processing system, a memory budget is assigned to each of a number of agents within the multi-processing system. A portion of memory is allocated to each the agents within the memory budget. Metrics are collected for each agent during processing of data by the agents; the metrics include an amount of data processed and an amount of memory used for each agent. Memory efficiency is determined for each agent based on the collected metrics and another memory budget is determined based on the memory efficiency. The portion of the memory is reallocated to the agents within the other memory budget in response to data stored in the memory relative to the assigned memory budget meeting a criterion.

Abstract: A computer-implemented method includes receiving a query plan. The method includes identifying a plurality of qualified relevant rows and one or more encoding candidate payload columns. The method includes analyzing the relevant rows in the encoding candidate payload columns to yield a count of distinct contents and a payload column width. The method includes estimating a cost and determining whether the cost is larger than an amount of available memory for on-the-fly encoding all of the plurality of encoding candidate payload columns. The method is responsive to the estimated cost being less than the amount of available memory, by on-the-fly encoding the encoding candidates and responsive to the estimated cost being greater than the amount of available memory by on-the-fly encoding fewer than all of the encoding candidates so as not to exceed the available memory, and leaving alone one or more remaining encoding candidate payload columns unencoded.

Abstract: A computer-implemented method includes receiving a query plan. The method includes identifying a plurality of qualified relevant rows and one or more encoding candidate payload columns. The method includes analyzing the relevant rows in the encoding candidate payload columns to yield a count of distinct contents and a payload column width. The method includes estimating a cost and determining whether the cost is larger than an amount of available memory for on-the-fly encoding all of the plurality of encoding candidate payload columns. The method is responsive to the estimated cost being less than the amount of available memory, by on-the-fly encoding the encoding candidates and responsive to the estimated cost being greater than the amount of available memory by on-the-fly encoding fewer than all of the encoding candidates so as not to exceed the available memory, and leaving alone one or more remaining encoding candidate payload columns unencoded.

Abstract: The embodiments described herein relate to accessing a plurality of data elements. A page of column data is compressed and stored in a format that includes a collection of data elements. A tuple map is stored, and the collection of data elements is indexed via the tuple map. A query is processed based on the compressed page by identifying a set of tuple identifiers mapping to stored data in support of the query. Each tuple identifier corresponds to a location of a respective tuple of the compressed page.

Abstract: One embodiment provides a method comprising adjusting a runtime of a dataflow processing environment to operate on multiple batches of objects. The method further comprises pre-allocating one or more vectors of objects, and processing the multiple batches one at a time. The one or more vectors of objects are re-used during processing of each batch.

Abstract: One embodiment provides a method for efficient representation, access and modification of variable length data objects including converting, by a processor, all object updates into appends. The processor divides each object update into bounded size tokens. The tokens are stored normalized as an anchor and a linked list of fragments, the anchor having a pointer to a linked list of fragments. A fragment comprises a contiguous list of tokens that correspond to the appends.

Abstract: One embodiment provides a method for transparent partial object instantiation for object oriented applications including analyzing, by a processor, application code to determine a list of objects. The processor determines a list of accessed fields for each of the objects based on performing analysis for each function in the application code for accessed fields and other functions invoked.

Abstract: Embodiments relate to payload storage format for storing data in support of an aggregation function. As an input is subject to aggregation, the input is evaluated to ascertain a payload format for the aggregation. It is understood that there is more than one payload format. An evaluation of the aggregation key is a factor in the initial payload format. If the key is an addition to an existing aggregation, the evaluation considers changing the format of the payload to address processing and/or memory efficiency for the aggregation. The evaluation and the format change takes place dynamically so that the aggregation may continue.

Abstract: A computer-implemented method includes receiving a query plan. The method includes identifying a plurality of qualified relevant rows and one or more encoding candidate payload columns. The method includes analyzing the relevant rows in the encoding candidate payload columns to yield a count of distinct contents and a payload column width. The method includes estimating a cost and determining whether the cost is larger than an amount of available memory for on-the-fly encoding all of the plurality of encoding candidate payload columns. The method is responsive to the estimated cost being less than the amount of available memory, by on-the-fly encoding the encoding candidates and responsive to the estimated cost being greater than the amount of available memory by on-the-fly encoding fewer than all of the encoding candidates so as not to exceed the available memory, and leaving alone one or more remaining encoding candidate payload columns unencoded.

Abstract: A method for creating a compact hash table includes a thread executing using a processor for scanning a subset of keys and values, and for each key, performing a hash operation and inserting hashed keys into a bitmap structure. Cumulative population counts of keys and values are determined within the bitmap. The method repeats scanning the subset of the keys and values. The keys and values are inserted into a compacted array using the cumulative population counts.

Abstract: A method for building a hash table over a subset of data in a data set includes partitioning a subset of keys and values in the data set into multiple partitions. A hash table is formed with space reserved for each partition based on cumulative counts for a number of keys and values in each partition of the multiple partitions. Each thread selects one or more partitions and inserts keys and values belonging to the selected one or more partitions into the hash table in the reserved space for those partitions. A compact hash table is created that includes a bitmap and a compacted army.

Abstract: A method of orchestrated shuffling of data in a non-uniform memory access device that includes a plurality of processing nodes that are connected by interconnects. The method includes running an application on a plurality of threads executing on the plurality of processing nodes. Data to be shuffled is identified from source threads running on source processing nodes among the processing nodes to target threads executing on target processing nodes among the processing nodes. The method further includes generating a plan for orchestrating the shuffling of the data among the all of the memory devices associated with the threads and for simultaneously transmitting data over different interconnects to a plurality of different target processing nodes from a plurality of different source processing nodes. The data is shuffled among all of the memory devices based on the plan.

Abstract: A method, a system, and a computer program product are provided. A filter is created for each portion of a data set. The filter indicates which one or more characteristics are present among each of the portions. Each of the one or more characteristics comprises one or more groups defined by the data grouping operation. The filters for the portions of the data sets are transferred to one or more filter processors and combined within the one or more filter processors to indicate characteristics of data residing across multiple processing elements to produce a result for a data grouping operation, utilizing transfers based on a combined filter result. In various embodiments, the filter may be a Bloom filter.

Abstract: According to one embodiment of the present invention, a method of transferring filters for operations in a distributed system comprises generating at each source node a first filter of a first size. The first filter indicates elements of objects that need not be processed for an operation. Each source node generates one more second filters of a second size less than the first size. Each generated second filter is based on a prior generated filter with each location representing a plurality of locations from the prior generated filter. A second filter is transferred from each source node to each of destination node. The elements of the objects to remove for the operation are determined based on the transferred second filters. Embodiments of the present invention further include a system and computer program product for transferring filters in a distributed system in substantially the same manners described above.

Abstract: Embodiments relate to aggregation operations. More specifically, embodiments relate to translating a query involving a distinct aggregate(s) into a query that does not involve a distinct aggregate. The translation involves a transformation effectively rewriting the original query containing the distinct aggregate including removal of the distinct aggregate from the query. The translated query enables the query to perform a non-distinct aggregation that effectively provides the results of the removed distinct aggregate(s).

Abstract: According to one embodiment of the present invention, a method of transferring filters for operations in a distributed system comprises generating at each source node a first filter of a first size. The first filter indicates elements of objects that need not be processed for an operation. Each source node generates one more second filters of a second size less than the first size. Each generated second filter is based on a prior generated filter with each location representing a plurality of locations from the prior generated filter. A second filter is transferred from each source node to each of destination node. The elements of the objects to remove for the operation are determined based on the transferred second filters. Embodiments of the present invention further include a system and computer program product for transferring filters in a distributed system in substantially the same manners described above.