Abstract: A join operation between split data tables includes providing reduction data from first partitions to each partition among second partitions. The reduction data serves to identify actual values in one of the second partitions that also occur in one of the first partitions. Global IDs are assigned. Translation lists including the global IDs are sent to the first partitions. Each first partition and each second partition create globalized lists which can then be combined to generate respective first and second compiled lists. The join operation can then be conducted on the first and second compiled lists.

Abstract: In some aspects, a method, computer program product, and system are provided for query optimization. In some implementations, there may be provided a method including detecting a select query as a candidate for optimization, the select query including an order by, an offset value, and a limit value; determining, in response to the detecting, a modified offset value as the difference between the offset value and a size of a smaller dataset; and performing the select query to form a result set, the select query executed based on at least the modified offset value. Related systems, methods, and articles of manufacture are also described.

Abstract: Innovations in performing sort operations for dictionary-compressed values of columns in a column-store database using value identifiers (“IDs”) are described. For example, a database system includes a data store and an execution engine. The data store stores values at positions of a column A dictionary maps distinct values to corresponding value IDs. An inverted index stores, for each of the corresponding value IDs, a list of those of the positions that contain the associated distinct value. The execution engine processes a request to sort values at an input set of the positions and identify an output set of the positions for sorted values. In particular, the execution engine iterates through positions stored in the lists of the inverted index. For a given position, the execution engine checks if the given position is one of the input set and, if so, adds the given position to the output set.

Abstract: A computer implemented method for updating a secondary database includes receiving transaction log information from a primary database system at a secondary database system. The transaction log information comprises a first marker identifying a beginning of an operation comprising a plurality of sub-operations and a second marker identifying an end of the operation. The transaction log information is sequentially parsed, a first marker is identified, and a lock state is created after replaying the first marker to ensure proper serialization of the plurality of sub-operations. The lock state is terminated after replaying the second marker. Related apparatus, systems, techniques and articles are also described.

Abstract: A computer implemented method for manipulating data comprises receiving log transaction information from a primary database system at a secondary database system and parsing the log transaction information to identify data objects that have been modified on the primary database system. The secondary database system determines whether the data objects are present in main memory on the secondary database system. If the data objects are not in the main memory, they are copied from persistence data volumes to the main memory. The data objects are modified by replaying the log transactions on the data objects in the main memory. Then, the data objects in the persistence data volumes are modified. Related apparatus, systems, techniques and articles are also described.

Abstract: In some aspects, a method, computer program product, and system are provided for query optimization. In some implementations, there may be provided a method including detecting a select query as a candidate for optimization, the select query including an order by, an offset value, and a limit value; determining, in response to the detecting, a modified offset value as the difference between the offset value and a size of a smaller dataset; and performing the select query to form a result set, the select query executed based on at least the modified offset value. Related systems, methods, and articles of manufacture are also described.

Abstract: Partitioning of source partitions of a table of a database to target partitions is initiated. Thereafter, a transition partition specification is specified that identifies the source partitions and the target partitions. Data is then moved (e.g., asynchronously moved, etc.) from the source partitions to the target partitions. Concurrently with the moving of the data, operates are handled using the transition partition specification. Subsequently, the source partitions are dropped when all of the data has been moved to the target partitions and there are no open transactions accessing the source partitions. Related apparatus, systems, techniques and articles are also described.

Abstract: A computer implemented method for manipulating data comprises receiving log transaction information from a primary database system at a secondary database system and parsing the log transaction information to identify data objects that have been modified on the primary database system. The secondary database system determines whether the data objects are present in main memory on the secondary database system. If the data objects are not in the main memory, they are copied from persistence data volumes to the main memory. The data objects are modified by replaying the log transactions on the data objects in the main memory. Then, the data objects in the persistence data volumes are modified. Related apparatus, systems, techniques and articles are also described.

Abstract: Innovations in performing sort operations for dictionary-compressed values of columns in a column-store database using value identifiers (“IDs”) are described. For example, a database system includes a data store and an execution engine. The data store stores values at positions of a column A dictionary maps distinct values to corresponding value IDs. An inverted index stores, for each of the corresponding value IDs, a list of those of the positions that contain the associated distinct value. The execution engine processes a request to sort values at an input set of the positions and identify an output set of the positions for sorted values. In particular, the execution engine iterates through positions stored in the lists of the inverted index. For a given position, the execution engine checks if the given position is one of the input set and, if so, adds the given position to the output set.

Abstract: A join operation between split data tables includes providing reduction data from first partitions to each partition among second partitions. The reduction data serves to identify actual values in one of the second partitions that also occur in one of the first partitions. Global IDs are assigned. Translation lists including the global IDs are sent to the first partitions. Each first partition and each second partition create globalized lists which can then be combined to generate respective first and second compiled lists. The join operation can then be conducted on the first and second compiled lists.

Abstract: A join operation between split data tables includes providing value IDs. For each of the value IDs, a unique global ID may be associated with the value ID when the actual value represented by the value ID occurs among actual values comprising the second attribute of the second partition. For each identified unique global ID, the identified unique global ID may be paired with a document ID of a data record contained in a second partition stored at the second server in which the actual value in the data record is represented by the value ID associated with the identified unique global ID.

Abstract: A join operation between split data tables includes providing value IDs. For each of the value IDs, a unique global ID may be associated with the value ID when the actual value represented by the value ID occurs among actual values comprising the second attribute of the second partition. For each identified unique global ID, the identified unique global ID may be paired with a document ID of a data record contained in a second partition stored at the second server in which the actual value in the data record is represented by the value ID associated with the identified unique global ID.

Abstract: A join operation between split data tables includes providing reduction data from first partitions to each partition among second partitions. The reduction data serves to identify actual values in one of the second partitions that also occur in one of the first partitions. Global IDs are assigned. Translation lists including the global IDs are sent to the first partitions. Each first partition and each second partition create globalized lists which can then be combined to generate respective first and second compiled lists. The join operation can then be conducted on the first and second compiled lists.

Abstract: Methods and apparatus, including computer program products, for selection of rows and values from indexes with updates. In general, rows of an index may be associated with validity flags that indicate whether a row has been updated with an update inserted in a delta index; one scheme for value identifiers may be used for an index and another scheme for one or more delta indexes where all of the indexes are, to at least some extent, compressed according to dictionary-based compression; and multiple delta indexes may be used in alternation such that one delta index may accept updates while another is being updated. The delta indexes may also have validity flags and all updates, such as modifications of values, deletion of records, and inserting of new records may be handled as updates accepted by one or more delta indexes.

Abstract: Partitioning of source partitions of a table of a database to target partitions is initiated. Thereafter, a transition partition specification is specified that identifies the source partitions and the target partitions. Data is then moved (e.g., asynchronously moved, etc.) from the source partitions to the target partitions. Concurrently with the moving of the data, operates are handled using the transition partition specification. Subsequently, the source partitions are dropped when all of the data has been moved to the target partitions and there are no open transactions accessing the source partitions. Related apparatus, systems, techniques and articles are also described.

Abstract: A join operation between split data tables includes providing reduction data from first partitions to each partition among second partitions. The reduction data serves to identify actual values in one of the second partitions that also occur in one of the first partitions. Global IDs are assigned. Translation lists including the global IDs are sent to the first partitions. Each first partition and each second partition create globalized lists which can then be combined to generate respective first and second compiled lists. The join operation can then be conducted on the first and second compiled lists.

Abstract: An insertion of a record into a table that includes a primary key column and a second column that includes a global uniqueness constraint across all of a plurality of data partitions across which the table is split is initiated without checking that a value of the record in the second column is globally unique by contacting other partitions the one partition to which the record is to be added to. The insertion can be processed, at least in part by implementing a write lock on the one partition but without implementing a read lock on the other partitions. The write lock on the one partition can be released after the insertion is completed, after which the validity of the insertion can be verified, for example by examining the other parts and a delta partition corresponding to the table. The insertion can be undone if the insertion was not valid.

Abstract: A join operation between split data tables includes providing reduction data from first partitions to each partition among second partitions. The reduction data serves to identify actual values in one of the second partitions that also occur in one of the first partitions. Global IDs are assigned. Translation lists including the global IDs are sent to the first partitions. Each first partition and each second partition create globalized lists which can then be combined to generate respective first and second compiled lists. The join operation can then be conducted on the first and second compiled lists.

Abstract: A search query for a collection of electronic documents is parsed to identify one or more terms and such identified terms are associated with one or more languages (i.e., spoken languages such as English, German, Spanish, etc.). A terms inverted index and a language inverted index are accessed to identify documents responsive to the query. Related apparatus, systems, techniques and articles are also described.

Abstract: Methods and apparatus, including computer program products, for selection of rows and values from indexes with updates. In general, rows of an index may be associated with validity flags that indicate whether a row has been updated with an update inserted in a delta index; one scheme for value identifiers may be used for an index and another scheme for one or more delta indexes where all of the indexes are, to at least some extent, compressed according to dictionary-based compression; and multiple delta indexes may be used in alternation such that one delta index may accept updates while another is being updated. The delta indexes may also have validity flags and all updates, such as modifications of values, deletion of records, and inserting of new records may be handled as updates accepted by one or more delta indexes.