Abstract: An automation system is provided to automate any administrative task in a distributed database, such that the end user can input a goal state (e.g., create database with a five node architecture) and the automation system generates and executes a plan to achieve the goal state without further user input. According to another aspect, bringing existing database systems into automated management can be as complex as designing the database itself. According to some embodiments, the automation system is configured to analyze existing database systems, capture and/or install monitoring components within the existing database, and generate execution pathways to integrate existing database systems into automation control systems. Based on the current state information, the automation system is configured to generate an installation pathway of one or more intermediate states to transition the existing system from no automation to a goal state having active automation agents distributed throughout the database.

Abstract: Database systems and methods that implement a data aggregation framework are provided. The framework can be configured to optimize aggregate operations over non-relational distributed databases, including, for example, data access, data retrieval, data writes, indexing, etc. Various embodiments are configured to aggregate multiple operations and/or commands, where the results (e.g., database documents and computations) captured from the distributed database are transformed as they pass through an aggregation operation. The aggregation operation can be defined as a pipeline which enables the results from a first operation to be redirected into the input of a subsequent operation, which output can be redirected into further subsequent operations. Computations may also be executed at each stage of the pipeline, where each result at each stage can be evaluated by the computation to return a result. Execution of the pipeline can be optimized based on data dependencies and re-ordering of the pipeline operations.

Abstract: According to at least one aspect, a database system is provided. The database system includes at least one processor configured to receive a restore request to restore a portion of a dataset to a previous state and, responsive to receipt of the restore request, identify at least one snapshot from a plurality of snapshots of at least some data in the dataset to read based on the restore request and write a portion of the data in the identified at least one snapshot to the dataset to restore the portion of the dataset to the previous state.

Abstract: One aspect relates to a system that integrates a user interface of a device with a cloud based database system without use of a database application. A user may access a cloud based database system without the complexity of installing additional software or managing connections to the cloud based database system. The system may establish a communication link between the user interface and intelligent agents running on the database system in order to allow a user(s) to carry out various database monitoring and verification activities.

Abstract: According to some aspects, a database system is provided. The database system may include at least one processor configured to execute a plurality of system components. The plurality of system components may include an interface component configured to receive a read commit command and provide a result, a snapshot component configured to generate a plurality of snapshots of data stored in a first data storage node of a plurality of data storage nodes and identify a committed snapshot representative of data that has been replicated on a majority of the plurality of data storage nodes, and a command processing component configured to read the committed snapshot responsive to receipt of the read commit command and generate the result.

Abstract: According to some aspects, a database system comprising a processor configured to execute a plurality of system components is provided. The plurality of system components may include an interface component configured to receive a write commit command and provide a write commit confirmation, a snapshot component configured to generate a plurality of snapshots of data stored in a data storage node of a plurality of data storage nodes and identify a committed snapshot representative of data that has been replicated on a majority of the plurality of data storage nodes, and a command processing component configured to modify a data element based on the write commit command, determine whether the majority of the plurality of storage nodes have replicated the modification using the committed snapshot, and generate the write commit confirmation responsive to a determination that the majority of the plurality of data storage nodes have replicated the modification.

Abstract: According to one aspect, methods and systems are provided for creating partial indexes in a distributed database environment. The database includes an index engine configured to receive at least one index field, a criteria field, and a criteria condition, wherein the criteria field is not included in the at least one index field; and generate an index comprising the at least one index field from at least one record of the plurality of records and a pointer to the at least one record of the plurality of records, wherein the criteria field of the at least one record of the plurality of records satisfies the criteria condition. The database further includes a query engine configured to receive a search query containing the at least one index field; and search the index for the at least one index field.

Abstract: Various aspects provide for implementation of a cloud service for running, monitoring, and maintaining cloud distributed database deployments and in particular examples, provides cloud based services to run, monitor and maintain deployments of the known MongoDB database. Various embodiments provide services, interfaces, and manage provisioning of dedicated servers for the distributed database instances (e.g., MongoDB instances). Further aspects, including providing a database as a cloud service that eliminates the design challenges associated with many distributed database implementations, while allowing the client's input on configuration choices in building the database. In some implementations, clients can simply identity a number of database nodes, capability of the nodes, and within minutes have a fully functioning, scalable, replicated, and secure distributed database in the cloud.

Abstract: According to one aspect, provided is a horizontally scaled database architecture. Partition a database enables efficient distribution of data across a number of systems reducing processing costs associated with multiple machines. According to some aspects, the partitioned database can be manages as a single source interface to handle client requests. Further, it is realized that by identifying and testing key properties, horizontal scaling architectures can be implemented and operated with minimal overhead. In one embodiment, databases can be partitioned in an order preserving manner such that the overhead associated with moving the data for a given partition can be minimized during management of the data and/or database. In one embodiment, splits and migrations operations prioritize zero cost partitions, thereby, reducing computational burden associated with managing a partitioned database.

Abstract: An automation system is provided to automate any administrative task in a distributed database, such that the end user can input a goal state (e.g., create database with a five node architecture) and the automation system generates and executes a plan to achieve the goal state without further user input. According to another aspect, bringing existing database systems into automated management can be as complex as designing the database itself. According to some embodiments, the automation system is configured to analyze existing database systems, capture and/or install monitoring components within the existing database, and generate execution pathways to integrate existing database systems into automation control systems. Based on the current state information, the automation system is configured to generate an installation pathway of one or more intermediate states to transition the existing system from no automation to a goal state having active automation agents distributed throughout the database.

Abstract: Systems and methods are provided for managing asynchronous replication in a distributed database environment, while providing for scaling of the distributed database. A cluster of nodes can be assigned roles for managing partitions of data within the database and processing database requests. In one embodiment, each cluster includes a node with a primary role to process write operations and mange asynchronous replication of the operations to at least one secondary node. Each cluster or set of nodes can host one or more partitions of database data. Collectively, the cluster or set of nodes define a shard cluster that hosts all the data of the distributed database. Each shard cluster, individual nodes, or sets of nodes can be configured to manage the size of any hosted partitions, splitting database partitions, migrating partitions, and/or managing expansion of shard clusters to encompass new systems.

Abstract: According to one aspect, provided are methods and systems for minimizing lock contention in a distributed database environment. The methods and systems can include a database management component configured to manage database instances, the database management component also configured to receive a first data request operation on the distributed database, an execution component configured to process the first data request operation including at least one write request on at least one database instance managed by the database management component, and a fault prediction component configured to detect a potential page fault responsive to a target data of the write request, wherein the execution component is further configured to suspend execution of the first data request operation, request access a physical storage to read the target data into active memory, and re-execute the first data request operation after a period of time for suspending the first data request operation.

Abstract: Database systems and methods that implement a data aggregation framework are provided. The framework can be configured to optimize aggregate operations over non-relational distributed databases, including, for example, data access, data retrieval, data writes, indexing, etc. Various embodiments are configured to aggregate multiple operations and/or commands, where the results (e.g., database documents and computations) captured from the distributed database are transformed as they pass through an aggregation operation. The aggregation operation can be defined as a pipeline which enables the results from a first operation to be redirected into the input of a subsequent operation, which output can be redirected into further subsequent operations. Computations may also be executed at each stage of the pipeline, where each result at each stage can be evaluated by the computation to return a result. Execution of the pipeline can be optimized based on data dependencies and re-ordering of the pipeline operations.

Abstract: Various aspects provide for implementation of a cloud service for running, monitoring, and maintaining cloud distributed database deployments and in particular examples, provides cloud based services to run, monitor and maintain deployments of the known MongoDB database. Various embodiments provide services, interfaces, and manage provisioning of dedicated servers for the distributed database instances (e.g., MongoDB instances). Further aspects, including providing a database as a cloud service that eliminates the design challenges associated with many distributed database implementations, while allowing the client's input on configuration choices in building the database. In some implementations, clients can simply identity a number of database nodes, capability of the nodes, and within minutes have a fully functioning, scalable, replicated, and secure distributed database in the cloud.

Abstract: A system and computer implemented method for execution of aggregation expressions on a distributed non-relational database system is provided. According to one aspect, an aggregation operation may be provided that permits more complex operations using separate collections. For instance, it may be desirable to create a report from one collection using information grouped according to information stored in another collection. Such a capability may be provided within other conventional database systems, however, in a non-relational database system such as NoSQL, the system is not capable of performing server-side joins, such a capability may not be performed without denormalizing the attributes into each object that references it, or by performing application-level joins which is not efficient and leads to unnecessarily complex code within the application that interfaces with the NoSQL database system.

Abstract: Various aspects provide for implementation of a cloud service for running, monitoring, and maintaining cloud distributed database deployments and in particular examples, provides cloud based services to run, monitor and maintain deployments of the known MongoDB database. Various embodiments provide services, interfaces, and manage provisioning of dedicated servers for the distributed database instances (e.g., MongoDB instances). Further aspects, including providing a database as a cloud service that eliminates the design challenges associated with many distributed database implementations, while allowing the client's input on configuration choices in building the database. In some implementations, clients can simply identity a number of database nodes, capability of the nodes, and within minutes have a fully functioning, scalable, replicated, and secure distributed database in the cloud.

Abstract: One aspect relates to generating a unifying virtual schema in a non-relational database. A database is analyzed either in its entirety, or in part by using a sample of the database as a representative portion of the database or database portion as a whole. Common attributes within the database may be identified, ranked and displayed to a user through one or more of various visualizations. In other examples, the user may further refine the way the data is displayed and how much is displayed by interacting with a user interface. Another aspect associates visualizations of common attributes and respective data with executable query language. In one embodiment, selections within the UI generate executable queries for the user.

Abstract: Various aspects provide for implementation of a cloud service for running, monitoring, and maintaining cloud distributed database deployments and in particular examples, provides cloud based services to run, monitor and maintain deployments of the known MongoDB database. Various embodiments provide services, interfaces, and manage provisioning of dedicated servers for the distributed database instances (e.g., MongoDB instances). Further aspects, including providing a database as a cloud service that eliminates the design challenges associated with many distributed database implementations, while allowing the client's input on configuration choices in building the database. In some implementations, clients can simply identity a number of database nodes, capability of the nodes, and within minutes have a fully functioning, scalable, replicated, and secure distributed database in the cloud.

Abstract: Aspects of the present invention are directed to system and methods for optimizing identification of locations within a search area using hash values. A hash value represents location information in a single dimension format. Computing points around some location includes calculating an identification boundary that surrounds the location of interest based on the location's hash value. The identification boundary is expanded until it exceeds a search area defined by the location and a distance. Points around the location can be identified based on having associated hash values that fall within the identification boundary. Hashing operations let a system reduce the geometric work (i.e. searching inside boundaries) and processing required, by computing straightforward operations on hash quantities (e.g. searching a linear range of geohashes), instead of, for example, point to point comparisons.

Abstract: A system and method for determining consensus within a distributed database are provided. According to one aspect, a protocol is provided that reduces or eliminates heartbeat communication between nodes of a replica set. Nodes may communicate liveness information using existing database commands and metadata associated with the database commands. According to another aspect, improved systems and methods are provided for detection of node failures and election of a new primary node.