Abstract: A multi-level storage architecture and method of performing a partial merge are disclosed. A main store is partitioned into a passive main part and an active main part, the active main part being empty at a start of the partial merge, the passive main part storing encoded data records of the main store that are not subject to the partial merge. A values index corresponding to a sorted dictionary of the passive main part is set to a cardinality of n. The data records of a second level storage structure is merged into the active main part, the active main part having a dictionary that starts with a value of n+1, such that the merging into the active main part continues an encoding scheme according to the values index of the passive main part.

Abstract: In one embodiment, the present invention includes a computer-implemented method comprising storing data in an application using an application custom data type and application custom data structure. The data is stored in a database using the application custom data type and the application custom data structure. In one embodiment, a request is sent to access the data from the application to the database. The data is retrieved from the database in response to the request in the application custom data type and the application custom data structure. In one embodiment, the data is sent from the database to a shared memory in the application custom data type and the application custom data structure and the data is retrieved by the application from the shared memory in the application custom data type and the application custom data structure.

Abstract: Methods and apparatus, including computer program products, for block compression of tables with repeated values. In general, value identifiers representing a compressed column of data may be sorted to render repeated values contiguous, and block dictionaries may be generated. A block dictionary may be generated for each block of value identifiers. Each block dictionary may include a list of block identifiers, where each block identifier is associated with a value identifier and there is a block identifier for each unique value in a block. Blocks may have standard sizes and block dictionaries may be reused for multiple blocks.

Abstract: A pattern can be identified in at least part of a query whose definition is received in a query request. The identified pattern can be matched with a set of pre-defined patterns, each of which has associated therewith at least one pre-compiled query execution sub-component of a plurality of pre-compiled query execution sub-components retained in a library. A plan for executing the query can be generated, for example by incorporating the pre-compiled query execution sub-component associated with the matched pattern into the plan based on a pseudo code representation of the plan derived from the definition.

Abstract: A procedural pattern in a received query execution plan can be matched to a stored pattern for which an equivalent declarative operator has been pre-defined. The query execution plan can describe a query for accessing data. A hybrid execution plan can be generated by replacing the procedural pattern with the equivalent declarative operator. A hybrid execution plan processing cost can be assigned to execution of the hybrid execution plan and a query execution plan processing cost can be assigned to execution of the query execution plan. The assigning can include evaluating a cost model for the hybrid execution plan and the query execution plan. The query can be executed using the hybrid execution plan if the hybrid execution plan processing cost is less than the query execution plan processing cost or the query execution plan if the hybrid execution plan processing cost is greater than the query execution plan processing cost. Related systems, methods, and articles of manufacture are disclosed.

Abstract: In one embodiment, the present invention includes a computer-implemented method comprising storing data in an application using an application custom data type and application custom data structure. The data is stored in a database using the application custom data type and the application custom data structure. In one embodiment, a request is sent to access the data from the application to the database. The data is retrieved from the database in response to the request in the application custom data type and the application custom data structure. In one embodiment, the data is sent from the database to a shared memory in the application custom data type and the application custom data structure and the data is retrieved by the application from the shared memory in the application custom data type and the application custom data structure.

Abstract: A system, medium, and method including modifying a graphical view representation of a first table of a set of data, the first table having at least a first column subject to changes in value; modifying a graphical view representation of a second table of a second set of data; determining a query operation of the view representations of the first table and second table; aggregating values of the view representation of the result of the query operation having a same key; determining, based on the aggregated values, values for the first column of the view representation of the first table; and providing an output of the values for the first column of the view representation of the first table, the output values including changed values.

Abstract: A system and method of logless atomic data movement. An internal transaction is started within a multi-level storage architecture, the internal transaction to merge data from the first level storage structure to the second level storage structure. Committed data is read from a first level storage structure of the multi-level storage architecture as specified by the internal transaction. The committed data from the first level storage structure is inserted into a second level storage structure in a bulk insertion process, and the committed data is marked as being deleted from the first level storage. The internal transaction is then committed to the multi-level storage architecture when the committed data has been inserted into the second level storage structure.

Abstract: Deleting a data record from the second level storage or main store is disclosed. A look-up is performed for the data record in the first level storage, where the data record is defined by a row identifier. If the row identifier is found in the first level storage, a look-up is performed for an updated row identifier representing an update of the data record in the second level storage and the main store, the update of the data record being defined by an updated row identifier. If the updated row identifier is found in the second level storage, an undo log is generated from the first level storage to invalidate a row identifier of the row identifier. A flag is generated representing an invalid updated row identifier, and a redo log is generated to restore the data record in the first level storage.

Abstract: In one embodiment, the present invention includes a computer-implemented method comprising storing data in an application using an application custom data type and application custom data structure. The data is stored in a database using the application custom data type and the application custom data structure. In one embodiment, a request is sent to access the data from the application to the database. The data is retrieved from the database in response to the request in the application custom data type and the application custom data structure. In one embodiment, the data is sent from the database to a shared memory in the application custom data type and the application custom data structure and the data is retrieved by the application from the shared memory in the application custom data type and the application custom data structure.

Abstract: A computer-implemented system and method for performing distinct operations on multiple tables of shared memory of parallel computing environments are disclosed. A distinct operation is executed on each table of a plurality of tables, each distinct operation eliminating duplicate data from each table, the executing creating a hierarchy of table pairs and distinct results, the distinct results comprising a reduced row set for each table. Duplicates on each reduced row set are detected to complete the distinct operation on the plurality of tables.

Abstract: Deleting a data record from the second level storage or main store is disclosed. A look-up is performed for the data record in the first level storage, where the data record is defined by a row identifier. If the row identifier is found in the first level storage, a look-up is performed for an updated row identifier representing an update of the data record in the second level storage and the main store, the update of the data record being defined by an updated row identifier. If the updated row identifier is found in the second level storage, an undo log is generated from the first level storage to invalidate a row identifier of the row identifier. A flag is generated representing an invalid updated row identifier, and a redo log is generated to restore the data record in the first level storage.

Abstract: A system and method of logless atomic data movement. An internal transaction is started within a multi-level storage architecture, the internal transaction to merge data from the first level storage structure to the second level storage structure. Committed data is read from a first level storage structure of the multi-level storage architecture as specified by the internal transaction. The committed data from the first level storage structure is inserted into a second level storage structure in a bulk insertion process, and the committed data is marked as being deleted from the first level storage. The internal transaction is then committed to the multi-level storage architecture when the committed data has been inserted into the second level storage structure.

Abstract: In one embodiment, the present invention includes a computer-implemented method comprising storing data in an application using an application custom data type and application custom data structure. The data is stored in a database using the application custom data type and the application custom data structure. In one embodiment, a request is sent to access the data from the application to the database. The data is retrieved from the database in response to the request in the application custom data type and the application custom data structure. In one embodiment, the data is sent from the database to a shared memory in the application custom data type and the application custom data structure and the data is retrieved by the application from the shared memory in the application custom data type and the application custom data structure.

Abstract: Partitioning of a source table of a database to a target table is initiated. Thereafter, a replay table is generated that is populated with triggers for database operations performed on the source table for subsequent replay for the target partitions. Data is later moved (e.g., asynchronously moved, etc.) from the source table to the target table. The database operations are replayed on the target table T subsequent to the moving of the data using the replay table. In addition, the source table is dropped when all of the data has been moved to the target table and there are no operations requiring replay. Related apparatus, systems, techniques and articles are also described.

Abstract: Innovations for adaptive compression and decompression for dictionaries of a column-store database can reduce the amount of memory used for columns of the database, allowing a system to keep column data in memory for more columns, while delays for access operations remain acceptable. For example, dictionary compression variants use different compression techniques and implementation options. Some dictionary compression variants provide more aggressive compression (reduced memory consumption) but result in slower run-time performance. Other dictionary compression variants provide less aggressive compression (higher memory consumption) but support faster run-time performance. As another example, a compression manager can automatically select a dictionary compression variant for a given column in a column-store database.

Abstract: In one embodiment, the present invention includes a computer-implemented method comprising storing data in an application using an application custom data type and application custom data structure. The data is stored in a database using the application custom data type and the application custom data structure. In one embodiment, a request is sent to access the data from the application to the database. The data is retrieved from the database in response to the request in the application custom data type and the application custom data structure. In one embodiment, the data is sent from the database to a shared memory in the application custom data type and the application custom data structure and the data is retrieved by the application from the shared memory in the application custom data type and the application custom data structure.

Abstract: Methods and apparatus, including computer program products, are provided for providing for processing calculation plans. In one aspect, there is provided a computer-implemented method. The method may include generating a calculation plan including a plurality of nodes; determining whether at least one of the nodes includes a function node; and compiling the function node into executable code to enable execution of the plurality of nodes including the function node at the database. Related apparatus, systems, methods, and articles are also described.

Abstract: The present invention includes a database with a semantic layer for performing standard SQL and complex operations custom to one or more application. In one embodiment the present invention includes receiving a data request from an application in a database, the data request specifying standard SQL operations and custom application operations to be performed for said application. The data request is processed in a semantic layer of the database. The semantic layer processes the standard SQL operations and the custom application operations and generates a logical execution plan comprising the standard SQL operations and the custom application operations. The logical execution plan is converted into a physical execution plan, which is executed by the database to generate a data result. The data result may be sent back to the application.

Abstract: A pattern can be identified in at least part of a query whose definition is received in a query request. The identified pattern can be matched with a set of pre-defined patterns, each of which has associated therewith at least one pre-compiled query execution sub-component of a plurality of pre-compiled query execution sub-components retained in a library. A plan for executing the query can be generated, for example by incorporating the pre-compiled query execution sub-component associated with the matched pattern into the plan based on a pseudo code representation of the plan derived from the definition.