Abstract: A system and computer implemented method for execution of aggregation expressions on a distributed non-relational database system is provided. According to one aspect, an aggregation operation may be provided that permits more complex operations using separate collections. For instance, it may be desirable to create a report from one collection using information grouped according to information stored in another collection. Such a capability may be provided within other conventional database systems, however, in a non-relational database system such as NoSQL, the system is not capable of performing server-side joins, such a capability may not be performed without denormalizing the attributes into each object that references it, or by performing application-level joins which is not efficient and leads to unnecessarily complex code within the application that interfaces with the NoSQL database system.

Abstract: A system and method for determining consensus within a distributed database are provided. According to one aspect, a protocol is provided that reduces or eliminates heartbeat communication between nodes of a replica set. Nodes may communicate liveness information using existing database commands and metadata associated with the database commands. According to another aspect, improved systems and methods are provided for detection of node failures and election of a new primary node.

Abstract: According to one aspect, methods and systems are provided for selectively employing storage engines in a distributed database environment. The methods and systems can include a processor configured to execute a plurality of system components, wherein the system components comprise an operation prediction component configured to determine an expected set of operations to be performed on a portion of the database; a data format selection component configured to select, based on at least one characteristic of the expected set of operations, a data format for the portion of the database; and at least one storage engine for writing the portion of the database in the selected data format.

Abstract: An automation system is provided to automate any administrative task in a distributed database, such that the end user can input a goal state (e.g., create database with a five node architecture) and the automation system generates and executes a plan to achieve the goal state without further user input. According to another aspect, bringing existing database systems into automated management can be as complex as designing the database itself. According to some embodiments, the automation system is configured to analyze existing database systems, capture and/or install monitoring components within the existing database, and generate execution pathways to integrate existing database systems into automation control systems. Based on the current state information, the automation system is configured to generate an installation pathway of one or more intermediate states to transition the existing system from no automation to a goal state having active automation agents distributed throughout the database.

Abstract: The methods and systems can include a database management component configured to manage database instances, the database management component also configured to receive a first data request operation on the distributed database, an execution component configured to process the first data request operation including at least one write request on at least one database instance managed by the database management component, and a fault prediction component configured to detect a potential page fault responsive to a target data of the write request, wherein the execution component is further configured to suspend execution of the first data request operation, request access a physical storage to read the target data into active memory, and re-execute the first data request operation after a period of time for suspending the first data request operation.

Abstract: Systems and methods for managing a database back end as a service are described. In some aspects, the described systems and methods provide for a cloud-based resource for servicing a request for data from an application to a remote database and/or a service. In some aspects, the described systems and methods provide for a cloud-based application client for requesting data from a remote database and/or a service.

Abstract: Systems and methods for managing a database back end as a service are described. In some aspects, the described systems and methods provide for a cloud-based resource for servicing a request for data from an application to a remote database and/or a service. In some aspects, the described systems and methods provide for a cloud-based application client for requesting data from a remote database and/or a service.

Abstract: In various embodiments, the system includes augmented database drivers that are configured to automatically retry execution of write operations if a failure is encountered. In some embodiments, a database daemon is configured to manage the database functionality for a respective database node (e.g., primary or secondary node). Retrying execution of write operations allows the drivers to automatically retry certain write operations a threshold number of times if, for example, a network error is encountered, or if a healthy primary node is not available.

Abstract: Systems and methods are provided to enable control and placement of data repositories. In some embodiments, the system segments data into zones. A website, for example, may need to segment data according to location. In this example, a zone may be created for North America and another zone may be created for Europe. Data related to operations executed in North America, for example, can be placed in the North America zone and data related to transactions in Europe can be placed in the Europe zone. According to some embodiments, the system may use zones to accommodate a range of deployment scenarios.

Abstract: A system and computer implemented method for execution of aggregation expressions on a distributed non-relational database system is provided. According to an aspect, an operation is provided to perform the aggregation operation. An indication of at least one foreign collection of documents is accepted. An indication of at least one variable for at least one local field of at least one local collection of documents is accepted. An indication of a pipeline for execution on the at least one local collection of documents and the at least one foreign collection of documents is accepted. An operation comparing the local field of the at least one local collection of documents and the foreign field of the at least one foreign collection of documents is performed. Responsive to the operation, data from the at least one local collection of documents and the at least one foreign collection of documents is aggregated.

Abstract: Systems and methods are provided to enable control and placement of data repositories. In some embodiments, the system segments data into zones. A website, for example, may need to segment data according to location. In this example, a zone may be created for North America and another zone may be created for Europe. Data related to operations executed in North America, for example, can be placed in the North America zone and data related to transactions in Europe can be placed in the Europe zone. According to some embodiments, the system may use zones to accommodate a range of deployment scenarios.

Abstract: Database systems and methods that implement a data aggregation framework are provided. The framework can be configured to optimize aggregate operations over non-relational distributed databases, including, for example, data access, data retrieval, data writes, indexing, etc. Various embodiments are configured to aggregate multiple operations and/or commands, where the results (e.g., database documents and computations) captured from the distributed database are transformed as they pass through an aggregation operation. The aggregation operation can be defined as a pipeline which enables the results from a first operation to be redirected into the input of a subsequent operation, which output can be redirected into further subsequent operations. Computations may also be executed at each stage of the pipeline, where each result at each stage can be evaluated by the computation to return a result. Execution of the pipeline can be optimized based on data dependencies and re-ordering of the pipeline operations.

Abstract: An automation system is provided to automate any administrative task in a distributed database, such that the end user can input a goal state (e.g., create database with a five node architecture) and the automation system generates and executes a plan to achieve the goal state without further user input. According to another aspect, bringing existing database systems into automated management can be as complex as designing the database itself. According to some embodiments, the automation system is configured to analyze existing database systems, capture and/or install monitoring components within the existing database, and generate execution pathways to integrate existing database systems into automation control systems. Based on the current state information, the automation system is configured to generate an installation pathway of one or more intermediate states to transition the existing system from no automation to a goal state having active automation agents distributed throughout the database.

Abstract: Systems and methods are provided for managing asynchronous replication in a distributed database environment, while providing for scaling of the distributed database. A cluster of nodes can be assigned roles for managing partitions of data within the database and processing database requests. In one embodiment, each cluster includes a node with a primary role to process write operations and mange permit asynchronous replication of the operations to at least one secondary node. Each cluster or set of nodes can host one or more partitions of database data, and grouping of the clusters can be implemented a shard cluster that hosts the data of the distributed database. Each shard can be configured to manage the size of any hosted partitions, splitting database partitions, migrated partitions, and managing expansion of shard clusters to encompass new systems.

Abstract: An automation system is provided to automate any administrative task in a distributed database, such that the end user can input a goal state (e.g., create database with a five node architecture) and the automation system generates and executes a plan to achieve the goal state without further user input. According to another aspect, bringing existing database systems into automated management can be as complex as designing the database itself. According to some embodiments, the automation system is configured to analyze existing database systems, capture and/or install monitoring components within the existing database, and generate execution pathways to integrate existing database systems into automation control systems. Based on the current state information, the automation system is configured to generate an installation pathway of one or more intermediate states to transition the existing system from no automation to a goal state having active automation agents distributed throughout the database.

Abstract: Database systems and methods that implement a data aggregation framework are provided. The framework can be configured to optimize aggregate operations over non-relational distributed databases, including, for example, data access, data retrieval, data writes, indexing, etc. Various embodiments are configured to aggregate multiple operations and/or commands, where the results (e.g., database documents and computations) captured from the distributed database are transformed as they pass through an aggregation operation. The aggregation operation can be defined as a pipeline which enables the results from a first operation to be redirected into the input of a subsequent operation, which output can be redirected into further subsequent operations. Computations may also be executed at each stage of the pipeline, where each result at each stage can be evaluated by the computation to return a result. Execution of the pipeline can be optimized based on data dependencies and re-ordering of the pipeline operations.

Abstract: One aspect relates to a system that integrates a user interface of a device with a cloud based database system without use of a database application. A user may access a cloud based database system without the complexity of installing additional software or managing connections to the cloud based database system. The system may establish a communication link between the user interface and intelligent agents running on the database system in order to allow a user(s) to carry out various database monitoring and verification activities.

Abstract: According to at least one aspect, a database system is provided. The database system includes at least one processor configured to receive a restore request to restore a portion of a dataset to a previous state and, responsive to receipt of the restore request, identify at least one snapshot from a plurality of snapshots of at least some data in the dataset to read based on the restore request and write a portion of the data in the identified at least one snapshot to the dataset to restore the portion of the dataset to the previous state.

Abstract: According to some aspects, a database system is provided. The database system may include at least one processor configured to execute a plurality of system components. The plurality of system components may include an interface component configured to receive a read commit command and provide a result, a snapshot component configured to generate a plurality of snapshots of data stored in a first data storage node of a plurality of data storage nodes and identify a committed snapshot representative of data that has been replicated on a majority of the plurality of data storage nodes, and a command processing component configured to read the committed snapshot responsive to receipt of the read commit command and generate the result.

Abstract: According to one aspect, methods and systems are provided for creating partial indexes in a distributed database environment. The database includes an index engine configured to receive at least one index field, a criteria field, and a criteria condition, wherein the criteria field is not included in the at least one index field; and generate an index comprising the at least one index field from at least one record of the plurality of records and a pointer to the at least one record of the plurality of records, wherein the criteria field of the at least one record of the plurality of records satisfies the criteria condition. The database further includes a query engine configured to receive a search query containing the at least one index field; and search the index for the at least one index field.