Abstract: In some embodiments, a database cluster to cluster synchronization system may include multiple replicators coupled to a source database cluster and a destination database cluster, where the source and destination cluster may be shard clusters. Each of the multiple replicators may correspond to a respective subset of the source database cluster and configured to monitor changes of data on the respective subset of the source database cluster and translate the changes of data to one or more database operations to be performed on the destination cluster. The changes of data on the source database may be contained in respective change streams associated with each of the replicators.

Abstract: In various aspects of the present disclosure, systems and methods are described to identify and resolve structured queries so they execute consistently and accurately against any data architecture, and for example, dynamic or unstructured database stores. According to one embodiment, a dynamic schema data system implements a query dialect that is configured to expose underlying flexible schemas of the dynamic schema data system, any structured data, unstructured or partially structured data, and expressive querying native to the dynamic schema system in a language that is compatible with structured queries, and for example, compatible with SQL-92. In further embodiments, the query dialect is configured to enable consistency with existing dynamic schema database query semantics (e.g., the known MongoDB database and associated query semantics).

Abstract: A method for managing data synchronization for a database having a dynamic schema, the method comprising: generating, at a first client device, a first changeset that is representative of an operation on a data object in a database; transmitting a synchronization request for synchronizing the database with one or more changesets generated by a second client device; receiving a second changeset that is representative of an operation performed by the second client device on the data object, wherein the database is shared between the first client device and the second client device; and merging, at the first client device, the first changeset and the second changeset to update the data object, wherein the merging is performed based on data received from a plurality of the one or more servers operating in parallel to each other.

Abstract: Systems and methods are provided for managing read requests in a database system. The same read request is communicated to multiple nodes to reduce long tail latency. If the read request is communicated to two nodes and the first node is experiencing a communication failure, the read request is serviced by the second node. Once a response is received from the second node, the read request to the first node can be canceled.

Abstract: According to at least one aspect, a distributed database system is provided. In some implementations, the distributed system comprises an online database, an archive database, and a data processing entity adapted to receive a query of a single logical database, the single logical database being stored across the online database and offline database. In some examples, the data processing entity archives data from the online database to the offline database responsive to one or more archive rules.

Abstract: A database system comprising a database having a dynamic schema and comprising a plurality of data storage nodes; and at least one processor configured to, using an encryption process: manage access to plaintext data stored in the plurality of data storage nodes by users employing at least one client-controlled resource in a client access layer; restrict access to the plaintext data by other users, wherein the other users include users with system administration privileges for the database and administrators of processing resources hosting the database; and manage access to encrypted copies of the plaintext data by the users with system administration privileges for the database such that the system administration privileges do not enable access to plaintext versions of the encrypted copies. A method for managing data security for a database. A database system with a dynamic schema architecture, a client access layer, and an operational database layer.

Abstract: In some implementations, events measured at various points in time may be organized in a data structure that defines an event represented by a document. In particular, events can be organized in columns of documents referred to as buckets. These buckets may be indexed using B-trees by addressing metadata values or value ranges. Buckets may be defined by periods of time. Documents may also be geoindexed and stored in one or more locations in a distributed computer network. One or more secondary indexes may be created based on time and/or metadata values within documents.

Abstract: Described herein are embodiments of a database search system. In the architecture of the database search system, the database search system is decoupled from the management components of a distributed database for which the database search system executes queries. The decoupled architecture of the database search system allows the database search system to utilize its own processing and storage hardware that is separate from that of the management components of the distributed database system. The decoupled architecture thus allows for processing and storage optimizations for searching that lead to improved availability and query execution performance by the database search system.

Abstract: Systems and computerized methods for processing data in a data stream prior to landing the data in a data sink is provided. The system may comprise at least one processor operatively connected to a memory, the at least one processor, when executing, being configured to receive data relating to a data source and data sink, wherein the data source is a boundless data source; establish, based on the received data relating to the data source and data sink, a connection between the data source and the data sink; receive event data from the data source; process the event data on an event-by-event basis; and land the processed event data into the data sink. By performing operations on data directly from the data stream, the system and computerized methods provided herein may provide real-time or near real-time data processing as event data is received from various data sources.

Abstract: Described herein are tiered synchronization systems and methods suitable for use in non-relational database systems distributed across multiple remote locations, which may provide access to some or all functionality of a cloud-tier server at a mid-tier server. For example, the mid-tier server may be locally accessible (e.g., over LAN) to clients independent of any remote connection to the cloud-tier server (e.g., over the Internet). In some embodiments, a mid-tier server may be configured to perform flexible sync functionality, such as flexible partitioning and/or permissions, whether independently and/or in cooperation with the cloud-tier server performing the same or similar functionality. In some embodiments, systems described herein may provide flexible and consistent local data access to non-relational data in locations where remote network connections are intermittent, such as due to weather conditions, relatively frequent network outages, and/or due to the location regularly changing (e.g.

Abstract: Systems and computerized methods for processing data in a data stream prior to landing the data in a data sink is provided. The system may comprise at least one processor operatively connected to a memory, the at least one processor, when executing, being configured to receive data relating to a data source and data sink, wherein the data source is a boundless data source; establish, based on the received data relating to the data source and data sink, a connection between the data source and the data sink; receive event data from the data source; process the event data on an event-by-event basis; and land the processed event data into the data sink. By performing operations on data directly from the data stream, the system and computerized methods provided herein may provide real-time or near real-time data processing as event data is received from various data sources.

Abstract: Described herein embodiments of a query processing system. The query processing system optimizes execution by generating an optimized slot-based execution (SBE) plan for executing the query. The query processing system optimizes query execution by generating a logical representation of the query, optimizing the logical representation of the query, and translating the optimized logical representation of the query into an SBE plan. The query processing system then executes the SBE plan to generate the query results.

Abstract: Described herein are tiered synchronization systems and methods suitable for use in non-relational database systems distributed across multiple remote locations, which may provide access to some or all functionality of a cloud-tier server at a mid-tier server. For example, the mid-tier server may be locally accessible (e.g., over LAN) to clients independent of any remote connection to the cloud-tier server (e.g., over the Internet). In some embodiments, a mid-tier server may be configured to perform flexible sync functionality, such as flexible partitioning and/or permissions, whether independently and/or in cooperation with the cloud-tier server performing the same or similar functionality. In some embodiments, systems described herein may provide flexible and consistent local data access to non-relational data in locations where remote network connections are intermittent, such as due to weather conditions, relatively frequent network outages, and/or due to the location regularly changing (e.g.

Abstract: Described herein are embodiments of a database search system. In the architecture of the database search system, the database search system is decoupled from the management components of a distributed database for which the database search system executes queries. The decoupled architecture of the database search system allows the database search system to utilize its own processing and storage hardware that is separate from that of the management components of the distributed database system. The decoupled architecture thus allows for processing and storage optimizations for searching that lead to improved availability and query execution performance by the database search system.

Abstract: A database system may comprise one or more nodes, where each node is embedded with functionality of each of a shard server configured for storing, retrieving, managing, and/or updating data; a shard routing process; and metadata management. A single node running these functionalities allows a database system to provide improved sharding functionality. Nodes may run on a same hardware profile. Database systems described herein may provide enhanced scalability and may appropriately scale without any input on the part of users.

Abstract: Described herein embodiments of a query processing system. The query processing system optimizes execution by generating an optimized slot-based execution (SBE) plan for executing the query. The query processing system optimizes query execution by generating a logical representation of the query, optimizing the logical representation of the query, and translating the optimized logical representation of the query into an SBE plan. The query processing system then executes the SBE plan to generate the query results.

Abstract: Systems and computerized methods for processing data in a data stream prior to landing the data in a data sink is provided. The system may comprise at least one processor operatively connected to a memory, the at least one processor, when executing, being configured to receive data relating to a data source and data sink, wherein the data source is a boundless data source; establish, based on the received data relating to the data source and data sink, a connection between the data source and the data sink; receive event data from the data source; process the event data on an event-by-event basis; and land the processed event data into the data sink. By performing operations on data directly from the data stream, the system and computerized methods provided herein may provide real-time or near real-time data processing as event data is received from various data sources.

Abstract: In some implementations, events measured at various points in time may be organized in a data structure that defines an event represented by a document. In particular, events can be organized in columns of documents referred to as buckets. These buckets may be indexed using B-trees by addressing metadata values or value ranges. Buckets may be defined by periods of time. Documents may also be geoindexed and stored in one or more locations in a distributed computer network. One or more secondary indexes may be created based on time and/or metadata values within documents.

Abstract: Systems and methods described herein may improve data migration between databases that store data under different schemas. Such systems and methods may be used to transform data from a first schema (e.g., tabular, such as in relational databases) to second schema (e.g., document-based, such as in some non-relational databases). In some embodiments, data transformation for database migration may be performed using at least one rule (e.g., specified by a user) specifying a logical arrangement under the second schema for transforming a grouping of source data under the first schema. For example, tables under a tabular schema may be transformed into new documents and/or fields within documents under a document-based schema in a manner specified by the transform rule(s). In some embodiments, transforming source data in this manner permits organizing the migrated data efficiently to reduce computing resources needed to migrate the data between databases and/or for future data access operations.

Abstract: Systems and methods described herein may improve data migration between databases that store data under different schemas. Such systems and methods may be used to transform data from a first schema (e.g., tabular, such as in relational databases) to second schema (e.g., document-based, such as in some non-relational databases). In some embodiments, data transformation for database migration may be performed using at least one rule (e.g., specified by a user) specifying a logical arrangement under the second schema for transforming a grouping of source data under the first schema. For example, tables under a tabular schema may be transformed into new documents and/or fields within documents under a document-based schema in a manner specified by the transform rule(s). In some embodiments, transforming source data in this manner permits organizing the migrated data efficiently to reduce computing resources needed to migrate the data between databases and/or for future data access operations.