Abstract: A data storage and retrieval system for a computer memory including a memory slice formed of segments and adapted to contain one or more documents and a checkpoint adapted to persist the memory slice. The checkpoint includes a document vector containing a document pointer corresponding to a document. The document pointer including a segment identifier identifying a logical segment of the memory slice and an offset value defining a relative memory location of the first document within the identified segment. There are checkpoint memory blocks, each storing a copy of a corresponding segment of the memory slice. The segment identifier of the document pointer identifies a checkpoint memory block and the offset value of the document pointer defines a relative location of the document within the checkpoint memory block.

Abstract: Technologies are described for facilitating database system recovery in a distributed database environment having a master node and at least one slave node. The master node maintains an in-memory reference that includes transaction identifiers and their corresponding commit identifiers. When a slave node requests that a commit identifier be assigned to a transaction, the transaction and commit identifiers for the transaction are added to the in-memory reference. The commit identifier is sent by the master node to the slave node. The slave node writes a log entry to a commit log that includes the transaction identifier and the assigned commit identifier. If the database system is interrupted before the slave node writes the commit log entry, the slave node can request the commit identifier for the transaction from the master node, which retrieves the identifier from the in-memory reference.

Abstract: Provided are systems and methods for implementing a database notification and statistics framework. In one example, the method may include receiving a message published by a database, the message including information about a database event performed by the database, determining a publication type of the message from among a plurality of publication types based on attributes included in the message, identifying one or more database components that have subscribed to the identified publication type of the message, and transferring the information about the database event included in the message to the one or more identified database components that have subscribed to the publication type of the message.

Abstract: Provided are systems and methods for paging data into main memory from checkpoint data stored on disk. In one example, the method may include one or more of receiving a request for a database record in main memory, determining whether the database record has been previously stored in the main memory, in response to determining that the database record has been previously stored in the main memory, identifying a slice where the database record was stored from among a plurality of slices included in the main memory, and paging content of the identified slice including a copy of the requested database record into the main memory from a snapshot captured of content included in the identified slice and previously stored on disk. Accordingly, documents can be paged into main memory on-demand from snapshots of slice content rather than paging an entire partition of content.

Abstract: Methods, systems, and computer-readable storage media for determining that a transaction of a plurality of transactions performed in at least a portion of a system includes a delete operation, the plurality of transactions being managed by a secondary transaction manager and including a subset of all transactions performed in the system, in response to the delete operation, inserting a clean-up entry in the secondary transaction manager, attaching the clean-up entry to a subsequent transaction in order to determine and assign a time to the cleanup-entry that is used to subsequently trigger garbage collection, and selectively comparing the time to a most-recently-reported minimum read timestamp that is periodically reported to the secondary transaction manager from a primary transaction manager of the system, wherein the clean-up entry is executed in response to determining that the time is less than the most-recently-reported minimum read timestamp.

Abstract: A data storage and retrieval system for a computer memory including a memory slice formed of segments and adapted to contain one or more documents and a checkpoint adapted to persist the memory slice. The checkpoint includes a document vector containing a document pointer corresponding to a document. The document pointer including a segment identifier identifying a logical segment of the memory slice and an offset value defining a relative memory location of the first document within the identified segment. There are checkpoint memory blocks, each storing a copy of a corresponding segment of the memory slice. The segment identifier of the document pointer identifies a checkpoint memory block and the offset value of the document pointer defines a relative location of the document within the checkpoint memory block.

Abstract: A data storage and retrieval system for a computer memory including a memory slice formed of segments and adapted to contain one or more documents and a checkpoint adapted to persist the memory slice. The checkpoint includes a document vector containing a document pointer corresponding to a document. The document pointer including a segment identifier identifying a logical segment of the memory slice and an offset value defining a relative memory location of the first document within the identified segment. There are checkpoint memory blocks, each storing a copy of a corresponding segment of the memory slice. The segment identifier of the document pointer identifies a checkpoint memory block and the offset value of the document pointer defines a relative location of the document within the checkpoint memory block.

Abstract: A data storage and retrieval system for a computer memory configured according to a columnar document store adapted to contain one or more documents. The columnar document store includes a dictionary containing one or more identifiers, each of the identifiers being associated in the dictionary with a value ID. The document store further includes a data structure having fields and containing at least a first column. The first column is associated with a first identifier of the one or more identifiers, and the first identifier is associated with a first set of values in a first set of documents which contain the first identifier. The first column includes a value ID of the first identifier, one or more document IDs, each identifying a document in the first set of documents, and the first set of values.

Abstract: A new unique value to be added to a column of a database table can be written to an in-memory array representing a dictionary of the column, and space can be allocated in at least one dictionary block on at least one page maintained on a persistent storage without writing the new unique value to the at least one dictionary block. A pending materialization bit for the at least one page can be set to identify the at least one page for deferred materialization.

Abstract: A data storage and retrieval system for a computer memory configured according to a document store adapted to contain documents. The document store includes a document data structure containing a header and a document having one or more members. The document store further includes a chain of one or more differential data structures, each having a header and at least one updated member. The updated member has a value which is updated relative to a value in a corresponding member having a same identifier in the document data structure or in a preceding differential data structure in the chain. A header of each of the differential data structures includes a pointer to the document data structure or the preceding differential data structure in the chain, and a flag indicating that each of the differential data structures contains less than all of the members of the document data structure.

Abstract: A database system can receive, from a client, a query specifying at least one database operation implicating data within a first database and within a second database. The first database can have a first transaction manager and the second database can have a second transaction manager. Thereafter, a plurality of transactions are initiated that are responsive to the query to access the data within the first database and the second database. The first transaction manager and the second transaction manager together coordinate execution of the plurality of transactions. All changes specified by the at least one database operation are persisted in persistence of the first database including changes implicating data stored within the second database. Data responsive to the query can then optionally be provided to the client. Related apparatus, systems, techniques and articles are also described.

Abstract: A system for storage, access, and/or pruning of documents of a document store is provided. In some implementations, the system performs operations comprising analyzing a plurality of parameters of a plurality of documents stored among a plurality of slices in a database and generating, based on the analyzing, statistics about the plurality of parameters including a range of document identifiers present in corresponding slices from the plurality of slices. The operations can further comprise pruning, based on the statistics, one or more slices of the plurality of slices, the pruning triggered in response to a query related to at least a portion of the plurality of stored documents. The operations can further comprise responding to the query based on slices that are not pruned. One or more of the operations can be implemented by a database server. Related systems, methods, and articles of manufacture are also described.

Abstract: A method can include: generating a query execution plan for a query including a plurality of operations that operate on data from a relational database and data from a non-structured query language (NoSQL) database, the generating comprising optimizing the query by: identifying a first operation that operate on data from the relational database and an intermediate result output by a second operation, the second operation outputting the intermediate result by operating on the data from the NoSQL database; and determining a fixed datatype for the intermediate result, an indication of the fixed datatype being included in the query execution plan; and executing, based on the query execution plan, the query, the executing of the query comprising converting the intermediate result to the fixed datatype, the converting enabling the first operation to operate on the intermediate result output by the second operation along with the data from the relational database.

Abstract: Technologies are described for facilitating database system recovery in a distributed database environment having a master node and at least one slave node. The master node maintains an in-memory reference that includes transaction identifiers and their corresponding commit identifiers. When a slave node requests that a commit identifier be assigned to a transaction, the transaction and commit identifiers for the transaction are added to the in-memory reference. The commit identifier is sent by the master node to the slave node. The slave node writes a log entry to a commit log that includes the transaction identifier and the assigned commit identifier. If the database system is interrupted before the slave node writes the commit log entry, the slave node can request the commit identifier for the transaction from the master node, which retrieves the identifier from the in-memory reference.

Abstract: Provided are systems and methods to acquire a manifest data file and a binary copy of a document store slice log having a first filename and comprising a plurality of entries, each of the plurality of entries associated with a respective database transaction and a commit identifier, and the manifest data file indicating one or more data partitions and one or more slices of each of the one or more data partitions, acquire a partition log for one of the indicated one or more data partitions, the generated partition log comprising a log entry associated with a transaction to create a slice and including a second filename associated with the slice, sort the plurality of entries of the document store slice log based on their associated commit identifiers, change the commit identifiers of each of the plurality of entries to a same commit identifier, and store the sorted and changed entries in a second document store slice log having the second filename.

Abstract: Provided are systems and methods for storing checkpoints of an in-memory storage device. In one example, the method may include detecting database transactions performed on a slice of an in-memory storage and storing a record of the database transactions in a log file, receiving a request to persist the slice of in-memory storage, the request comprising a timing indicator corresponding to a timing of database transactions, trimming the log file based on the timing indicator to remove a record of one or more database transactions from the log file that occurred prior to the timing indicator and to preserve a record of one or more database transactions that occurred after the timing indicator, and persisting, to disk, data from the slice of in-memory storage and the trimmed log file preserving the record of the one or more database transactions that occurred after the timing indicator.

Abstract: A system for encoding and/or decoding documents for a document store is provided. In some implementations, the system performs operations comprising receiving a query related to at least a portion of a plurality of stored documents and retrieving one or more documents from the plurality of stored documents based on the query. The operations can further include generating a client-side dictionary including a plurality of definitions corresponding to a plurality of parameters (e.g., reference values for corresponding parameters) from the plurality of stored documents. The operations can also include providing the client-side dictionary and the one or more documents in response to the query. One or more of the operations can be implemented by a database server. Related systems, methods, and articles of manufacture are also described.

Abstract: Technologies are described for facilitating database system recovery in a distributed database environment having a master node and at least one slave node. The master node maintains an in-memory reference that includes transaction identifiers and their corresponding commit identifiers. When a slave node requests that a commit identifier be assigned to a transaction, the transaction and commit identifiers for the transaction are added to the in-memory reference. The commit identifier is sent by the master node to the slave node. The slave node writes a log entry to a commit log that includes the transaction identifier and the assigned commit identifier. If the database system is interrupted before the slave node writes the commit log entry, the slave node can request the commit identifier for the transaction from the master node, which retrieves the identifier from the in-memory reference.

Abstract: Disclosed herein are system, method, and computer program product embodiments for altering the data type of a column in a database. An embodiment operates by converting an original dictionary associated with a column into a new dictionary. The new dictionary stores the values of the original dictionary using a different data type. An index vector containing the keys of the original dictionary is then updated to contain the associated keys of the new dictionary. Because the size of the original dictionary is often substantially smaller than the number of rows in the associated column, this dictionary conversion decreases the computation cost to the database system of altering the data type of the column and reduces or even minimizes database downtime for users.

Abstract: A database system can receive, from a client, a query specifying at least one database operation implicating data within a first database and within a second database. The first database can have a first transaction manager and the second database can have a second transaction manager. Thereafter, a plurality of transactions are initiated that are responsive to the query to access the data within the first database and the second database. The first transaction manager and the second transaction manager together coordinate execution of the plurality of transactions. All changes specified by the at least one database operation are persisted in persistence of the first database including changes implicating data stored within the second database. Data responsive to the query can then optionally be provided to the client. Related apparatus, systems, techniques and articles are also described.