Abstract: Described herein embodiments of a query processing system. The query processing system optimizes execution by generating an optimized slot-based execution (SBE) plan for executing the query. The query processing system optimizes query execution by generating a logical representation of the query, optimizing the logical representation of the query, and translating the optimized logical representation of the query into an SBE plan. The query processing system then executes the SBE plan to generate the query results.

Abstract: A system comprising a pipeline is provided comprising a first stage configured to create a plurality of BSON documents from input data, a block processing stage configured to format the plurality of BSON documents into one or more blocks and to provide an access interface to the one or more blocks, a query stage configured to execute a query to the one or more blocks using the access interface to generate an output array responsive to the query; and a decomposition stage configured to transform the output array into one or more slots of scalar data values.

Abstract: Provided is a system for processing semi-structured data, comprising a tokenization module configured to convert JSON objects into token sequences by recursively traversing keys and values (e.g., in depth-first order) and generating sequences of key tokens, value tokens, and grammatical tokens, wherein key tokens are distinguished from value tokens through a transformation that wraps original key strings in a special token format. The system includes a positional encoding module configured to generate hierarchical position embeddings using a PDA that maintains a stack reflecting parsing state, wherein the positional encoding module computes position embeddings by summing embeddings of stack symbols present at each sequence position. A transformer architecture comprising multiple layers of self-attention mechanisms is configured to process combined token and position embeddings.

Abstract: According to some aspects, provided are systems and methods that implement end-to-end encryption, and provide implementation configured to secure information during execution of queries on an encrypted data source. Various embodiments include multiple encrypted multi-map data structures and associated encryption schemes configured to securely read, write, and delete information while supporting any one or more of the following features: snapshot security, multiple client support, efficient execution under concurrent operation, and resilience to client failures. In various embodiments, addressable multi-map data structures enable concurrent access, and allow correct operation under polynomial time constraints.

Abstract: Systems and methods are configured to identify and resolve user-based issues, architecture issues, and data model issues that are introduced into distributed database deployments. Even in the context of a pre-built distributed database system, “bad” user behavior can have huge impacts on database performance. Various embodiments provide for analysis and resolution of architecture issues, as well schema issues, and can be configured to identify and resolve such problems in the context of dynamic schema database systems. Unlike relational database counterparts, dynamic schema databases do not enforce static database architecture or defined schemas. In such environments, database data and database formatting can be mostly unconstrained. If used properly, this flexibility enables easy configuration and execution. However, when improperly designed, small flaws magnify in significance as the database grows.

Abstract: A system and computer implemented method for execution of aggregation expressions on a distributed non-relational database system is provided. According to one aspect, an aggregation operation may be provided that permits more complex operations using separate collections. For instance, it may be desirable to create a report from one collection using information grouped according to information stored in another collection. Such a capability may be provided within other conventional database systems, however, in a non-relational database system such as NoSQL, the system is not capable of performing server-side joins, such a capability may not be performed without denormalizing the attributes into each object that references it, or by performing application-level joins which is not efficient and leads to unnecessarily complex code within the application that interfaces with the NoSQL database system.

Abstract: Described herein embodiments of a query processing system. The query processing system optimizes execution by generating an optimized slot-based execution (SBE) plan for executing the query. The query processing system optimizes query execution by generating a logical representation of the query, optimizing the logical representation of the query, and translating the optimized logical representation of the query into an SBE plan. The query processing system then executes the SBE plan to generate the query results.

Abstract: Described herein embodiments of a query processing system. The query processing system optimizes execution by generating an optimized slot-based execution (SBE) plan for executing the query. The query processing system optimizes query execution by generating a logical representation of the query, optimizing the logical representation of the query, and translating the optimized logical representation of the query into an SBE plan. The query processing system then executes the SBE plan to generate the query results.

Abstract: A database system comprising a database having a dynamic schema and at least one processor configured to: generate at least one column store index associated with the database, at least by: extracting keys from at least one document in the database, and sorting the keys; store the at least one column store index at least by storing the keys in the database; and access the at least one column store index based on at least one access request.

Abstract: According to aspects of the disclosure, there is provided a scalable cloud distributed database system for hosting data of multiple database tenants. In some embodiments, the database may be serverless. The serverless database may be configured to automatically and dynamically match resources to workload demands for tenants of the database. Databases described herein may include replica sets hosting multiple tenants. Tenants may be migrated from one replica set to another replica set based on usage. Usage of tenants may be throttled during migration between replica sets. Tenants with lower usage may be selected for migration between replica sets. During migration, files containing both a document and a history of updates to the document may be migrated. Databases described herein may include multiple storage tiers having different read and/or write speeds.

Abstract: According to some aspects, provided are systems and methods that implement end-to-end encryption, and provide implementation configured to secure information during execution of queries on an encrypted data source. Various embodiments include multiple encrypted multi-map data structures and associated encryption schemes configured to securely read, write, and delete information while supporting any one or more of the following features: snapshot security, multiple client support, efficient execution under concurrent operation, and resilience to client failures. In various embodiments, addressable multi-map data structures enable concurrent access, and allow correct operation under polynomial time constraints.

Abstract: According to some aspects, provided are systems and methods that implement end-to-end encryption, and provide implementation configured to secure information during execution of queries on an encrypted data source. Various embodiments include multiple encrypted multi-map data structures and associated encryption schemes configured to securely read, write, and delete information while supporting any one or more of the following features: snapshot security, multiple client support, efficient execution under concurrent operation, and resilience to client failures. In various embodiments, addressable multi-map data structures enable concurrent access, and allow correct operation under polynomial time constraints.

Abstract: According to some aspects, provided are systems and methods that implement end-to-end encryption, and provide implementation configured to secure information during execution of queries on an encrypted data source. Various embodiments include multiple encrypted multi-map data structures and associated encryption schemes configured to securely read, write, and delete information while supporting any one or more of the following features: snapshot security, multiple client support, efficient execution under concurrent operation, and resilience to client failures. In various embodiments, addressable multi-map data structures enable concurrent access, and allow correct operation under polynomial time constraints.

Abstract: In some embodiments, an auto-index system may be provided to determine one or more candidate indexes to be automatically created in a database system. The auto-index system may determine if one or more criteria associated with creating the one or more candidate indexes are met. In response to determining that the one or more criteria associated with creating the one or more candidate indexes are met, the auto-index system may automatically create the one or more candidate indexes, for example, by an automation agent. The one or more criteria may also be associated with respective performance metrics of the database system.

Abstract: Described herein are database systems that execute range queries on encrypted data. Provided is a numerical encoding scheme provides a set of order-preserving functions that maps all numerical data types supported by the database systems to a compact set of positive integers representative of the range query. Stored encrypted data may also be represented as positive integers. The positive integers from the range query may then be used to execute a set of equality queries on the stored encrypted data. Also provided is a range hypergraph supporting range queries to be executed on the encrypted data with high-throughput and without high amounts of storage overhead. Additionally, a hypergraph-friendly compaction protocol may be performed with padded inputs, in order to reduce leakage for range queries.

Abstract: Systems and methods are provided for implementing updatable private set intersection in distributed database architectures (e.g., the well-known MongoDB). The systems and methods can include a new specification language referred to for convenience as “MPPL,” for specifying general multi-party computation protocols. This architecture incorporates private set intersection, accessible via command line operators or other operators formatted according to a native query language. Other embodiments detail systems and methods for integrating multi-party database operations via new server node(s) added to a distributed database system (e.g., MongoDB cluster) that manage communication between parties holding private data.

Abstract: Systems and methods described herein may improve data migration between databases that store data under different schemas. Such systems and methods may be used to transform data from a first schema (e.g., tabular, such as in relational databases) to second schema (e.g., document-based, such as in some non-relational databases). In some embodiments, data transformation for database migration may be performed using at least one rule (e.g., specified by a user) specifying a logical arrangement under the second schema for transforming a grouping of source data under the first schema. For example, tables under a tabular schema may be transformed into new documents and/or fields within documents under a document-based schema in a manner specified by the transform rule(s). In some embodiments, transforming source data in this manner permits organizing the migrated data efficiently to reduce computing resources needed to migrate the data between databases and/or for future data access operations.

Abstract: A system and method for linearizable leader read optimizations in Raft are provided. According to one aspect, leader leases enable serving linearizable reads locally at the leaseholding leader without the cost and latency of communication with the followers. By leveraging the benefits of Raft log guarantees, the novel leader lease protocol of some embodiments simplifies complexity of lease management implementation and improves write and read availability during leader transitions.

Abstract: A system is provided for storing, in a database, a plurality of timeseries represented by a plurality of respective documents events in a columnar format. The system further is adapted compress at least one of the values within the plurality of documents. According to some embodiments, the system stores the compressed values as a Simple-8b block and calculates the optimal Simple-8b selector. According to some embodiments, the system is adapted to determine a secondary index based on values within the bucket.

Abstract: A volume hiding structured encryption system and method is provided. According to some embodiments, the system leverages the principle that a STE scheme can leak cumulative information about the query volumes of the data stored in multi-map format, while still hiding the volumes of client queries at query time. According to various examples, the system implements encryption schemes that have smaller storage and better query complexity than the current state-of-the-art, for some input distributions. According to one example, the STE schemes are adapted to the input data structure in order to improve efficiency over known approaches. Further examples includes schemes that are configured to partition a multi-map into smaller multi-maps, and use different allocation functions for different efficiency trade-offs: (i) random allocation, (ii) cuckoo-hashing allocation, and (iii) Garbled Bloom Filter allocation. In one example, the GBF construct enables a stash-less encryption scheme that hides query volumes.