Abstract: Innovations for adaptive compression and decompression for dictionaries of a column-store database can reduce the amount of memory used for columns of the database, allowing a system to keep column data in memory for more columns, while delays for access operations remain acceptable. For example, dictionary compression variants use different compression techniques and implementation options, Some dictionary compression variants provide more aggressive compression (reduced memory consumption) but result in slower run-time performance. Other dictionary compression variants provide less aggressive compression (higher memory consumption) but support faster run-time performance. As another example, a compression manager can automatically select a dictionary compression variant for a given column in a column-store database.

Abstract: Innovations for adaptive compression and decompression for dictionaries of a column-store database can reduce the amount of memory used for columns of the database, allowing a system to keep column data in memory for more columns, while delays for access operations remain acceptable. For example, dictionary compression variants use different compression techniques and implementation options. Some dictionary compression variants provide more aggressive compression (reduced memory consumption) but result in slower run-time performance. Other dictionary compression variants provide less aggressive compression (higher memory consumption) but support faster run-time performance. As another example, a compression manager can automatically select a dictionary compression variant for a given column in a column-store database.

Abstract: A procedural pattern in a received query execution plan can be matched to a stored pattern for which an equivalent declarative operator has been pre-defined. The query execution plan can describe a query for accessing data. A hybrid execution plan can be generated by replacing the procedural pattern with the equivalent declarative operator. A hybrid execution plan processing cost can be assigned to execution of the hybrid execution plan and a query execution plan processing cost can be assigned to execution of the query execution plan. The assigning can include evaluating a cost model for the hybrid execution plan and the query execution plan. The query can be executed using the hybrid execution plan if the hybrid execution plan processing cost is less than the query execution plan processing cost or the query execution plan if the hybrid execution plan processing cost is greater than the query execution plan processing cost. Related systems, methods, and articles of manufacture are disclosed.

Abstract: The multi-level storage system and method of deleting first level storage structure pages or records without record locks. The method includes determining whether a record to be deleted from the first level storage structure has any uncommitted write operation, and if the record has an uncommitted write operation, the record is kept in the first level storage structure. Record-moved version information is added to the record to designate the record being moved from the first level storage structure to the second level storage structure. Data change operations are executed for the record based on the record-moved version information without waiting until the record's movement from the first level storage structure to the second level storage structure finishes.

Abstract: Partitioning of a source table of a database to a target table is initiated. Thereafter, a replay table is generated that is populated with triggers for database operations performed on the source table for subsequent replay for the target partitions. Data is later moved (e.g., asynchronously moved, etc.) from the source table to the target table. The database operations are replayed on the target table T subsequent to the moving of the data using the replay table. In addition, the source table is dropped when all of the data has been moved to the target table and there are no operations requiring replay. Related apparatus, systems, techniques and articles are also described.

Abstract: Methods and apparatus, including computer program products, for compression of tables based on occurrence of values. In general, a number representing an amount of occurrences of a frequently occurring value in a group of adjacent rows of a column is generated, a vector representing whether the frequently occurring value exists in a row of the column is generated, and the number and the vector are stored to enable searches of the data represented by the number and the vector. The vector may omit a portion representing the group of adjacent rows. The values may be dictionary-based compression values representing business data such as business objects. The compression may be performed in-memory, in parallel, to improve memory utilization, network bandwidth consumption, and processing performance.

Abstract: The present invention includes a database with a semantic layer for performing standard SQL and complex operations custom to one or more application. In one embodiment the present invention includes receiving a data request from an application in a database, the data request specifying standard SQL operations and custom application operations to be performed for said application. The data request is processed in a semantic layer of the database. The semantic layer processes the standard SQL operations and the custom application operations and generates a logical execution plan comprising the standard SQL operations and the custom application operations. The logical execution plan is converted into a physical execution plan, which is executed by the database to generate a data result. The data result may be sent back to the application.

Abstract: Deleting a data record from the second level storage or main store is disclosed. A look-up is performed for the data record in the first level storage, where the data record is defined by a row identifier. If the row identifier is found in the first level storage, a look-up is performed for an updated row identifier representing an update of the data record in the second level storage and the main store, the update of the data record being defined by an updated row identifier. If the updated row identifier is found in the second level storage, an undo log is generated from the first level storage to invalidate a row identifier of the row identifier. A flag is generated representing an invalid updated row identifier, and a redo log is generated to restore the data record in the first level storage.

Abstract: In one embodiment, the present invention includes a computer-implemented method comprising storing data in an application using an application custom data type and application custom data structure. The data is stored in a database using the application custom data type and the application custom data structure. In one embodiment, a request is sent to access the data from the application to the database. The data is retrieved from the database in response to the request in the application custom data type and the application custom data structure. In one embodiment, the data is sent from the database to a shared memory in the application custom data type and the application custom data structure and the data is retrieved by the application from the shared memory in the application custom data type and the application custom data structure.

Abstract: Deleting a data record from the second level storage or main store is disclosed. A look-up is performed for the data record in the first level storage, where the data record is defined by a row identifier. If the row identifier is found in the first level storage, a look-up is performed for an updated row identifier representing an update of the data record in the second level storage and the main store, the update of the data record being defined by an updated row identifier. If the updated row identifier is found in the second level storage, an undo log is generated from the first level storage to invalidate a row identifier of the row identifier. A flag is generated representing an invalid updated row identifier, and a redo log is generated to restore the data record in the first level storage.

Abstract: In one embodiment, the present invention includes a computer-implemented method comprising storing data in an application using an application custom data type and application custom data structure. The data is stored in a database using the application custom data type and the application custom data structure. In one embodiment, a request is sent to access the data from the application to the database. The data is retrieved from the database in response to the request in the application custom data type and the application custom data structure. In one embodiment, the data is sent from the database to a shared memory in the application custom data type and the application custom data structure and the data is retrieved by the application from the shared memory in the application custom data type and the application custom data structure.

Abstract: A system and method of logless atomic data movement. An internal transaction is started within a multi-level storage architecture, the internal transaction to merge data from the first level storage structure to the second level storage structure. Committed data is read from a first level storage structure of the multi-level storage architecture as specified by the internal transaction. The committed data from the first level storage structure is inserted into a second level storage structure in a bulk insertion process, and the committed data is marked as being deleted from the first level storage. The internal transaction is then committed to the multi-level storage architecture when the committed data has been inserted into the second level storage structure.

Abstract: Technologies are provided for scaling applications via application scenarios. Portions of application resource content stored in a database are associated with application scenarios. The portions of the resource content associated with an application scenario are replicated to one or more computing nodes associated with the application scenario. Requests associated with the application scenario are transmitted to the one or more one or more computing nodes, where they are processed using the replicated portions of the resource content associated with the application scenario.

Abstract: A system and method of logless atomic data movement. An internal transaction is started within a multi-level storage architecture, the internal transaction to merge data from the first level storage structure to the second level storage structure. Committed data is read from a first level storage structure of the multi-level storage architecture as specified by the internal transaction. The committed data from the first level storage structure is inserted into a second level storage structure in a bulk insertion process, and the committed data is marked as being deleted from the first level storage. The internal transaction is then committed to the multi-level storage architecture when the committed data has been inserted into the second level storage structure.

Abstract: Methods and apparatus, including computer program products, for block compression of tables with repeated values. In general, value identifiers representing a compressed column of data may be sorted to render repeated values contiguous, and block dictionaries may be generated. A block dictionary may be generated for each block of value identifiers. Each block dictionary may include a list of block identifiers, where each block identifier is associated with a value identifier and there is a block identifier for each unique value in a block. Blocks may have standard sizes and block dictionaries may be reused for multiple blocks.

Abstract: A multi-level storage architecture and method of performing a partial merge are disclosed. A main store is partitioned into a passive main part and an active main part, the active main part being empty at a start of the partial merge, the passive main part storing encoded data records of the main store that are not subject to the partial merge. A values index corresponding to a sorted dictionary of the passive main part is set to a cardinality of n. The the data records of a second level storage structure is merged into the active main part, the active main part having a dictionary that starts with a value of n+1, such that the merging into the active main part continues an encoding scheme according to the values index of the passive main part.

Abstract: A system and method of query processing in a multi-level storage system having a unified table architecture. A query is received by a common query execution engine connected with the unified table architecture, the query specifying a data record. The common query execution engine performs a look-up for the data record based on the query at the first level storage structure. If the data record is not present at the first level storage structure, the common query execution engine performs separate look-ups in each of the second level storage structure and the main store.

Abstract: A pattern can be identified in at least part of a query whose definition is received in a query request. The identified pattern can be matched with a set of pre-defined patterns, each of which has associated therewith at least one pre-compiled query execution sub-component of a plurality of pre-compiled query execution sub-components retained in a library. A plan for executing the query can be generated, for example by incorporating the pre-compiled query execution sub-component associated with the matched pattern into the plan based on a pseudo code representation of the plan derived from the definition.

Abstract: A procedural pattern in a received query execution plan can be matched to a stored pattern for which an equivalent declarative operator has been pre-defined. The query execution plan can describe a query for accessing data. A hybrid execution plan can be generated by replacing the procedural pattern with the equivalent declarative operator. A hybrid execution plan processing cost can be assigned to execution of the hybrid execution plan and a query execution plan processing cost can be assigned to execution of the query execution plan. The assigning can include evaluating a cost model for the hybrid execution plan and the query execution plan. The query can be executed using the hybrid execution plan if the hybrid execution plan processing cost is less than the query execution plan processing cost or the query execution plan if the hybrid execution plan processing cost is greater than the query execution plan processing cost. Related systems, methods, and articles of manufacture are disclosed.

Abstract: A system and method of query processing in a multi-level storage system having a unified table architecture. A query is received by a common query execution engine connected with the unified table architecture, the query specifying a data record. The common query execution engine performs a look-up for the data record based on the query at the first level storage structure. If the data record is not present at the first level storage structure, the common query execution engine performs separate look-ups in each of the second level storage structure and the main store.