Abstract: Methods, systems, and devices for data management are described. For example, techniques for scalable backup solutions for non-relational databases are described. A data management system (DMS) may use different buffer pools for the extraction of data from the non-relational database and the movement (e.g., copying or transferring of data) from the buffers to a remote storage location. A backup process for a collection may involve one or more iterations involving retrieving or selecting a buffer from an empty buffer queue of an agent of the DMS at the non-relational database, filling the buffer with data from the non-relational database to the retrieved buffer, moving the filled buffer to a full buffer queue, moving the data to a remote storage location, and moving the buffer back to the empty buffer queue after moving the data on the buffer to the remote storage location.

Abstract: Methods, systems, and devices for data management are described. For example, techniques for scalable backup solutions for non-relational databases are described. A data management system (DMS) may use different buffer pools for the extraction of data from the non-relational database and the movement (e.g., copying or transferring of data) from the buffers to a remote storage location. A backup process for a collection may involve one or more iterations involving retrieving or selecting a buffer from an empty buffer queue of an agent of the DMS at the non-relational database, filling the buffer with data from the non-relational database to the retrieved buffer, moving the filled buffer to a full buffer queue, moving the data to a remote storage location, and moving the buffer back to the empty buffer queue after moving the data on the buffer to the remote storage location.

Abstract: Methods, systems, and devices for data management are described. For example, techniques for scalable backup solutions for non-relational databases are described. Operation logs (oplogs) may capture changes that occur at a non-relational database. A data management system (DMS) may use multiple queues and local disk memory of the host of the non-relational database to streamline the movement of oplogs from the non-relational database to a remote storage environment accessible to the DMS. Oplogs may be parsed into a multitenant queue, moved from the multitenant queue to collection-specific queues, written from the collection-specific queues to local disk memory of the host, and moved from the local disk memory of the host to the remote storage environment. Oplogs from multiple collections may be moved from the collection-specific queues to local disk memory of the host and from the local disk memory to the remote storage environment in parallel, reducing latency.

Abstract: Methods, systems, and devices for data management are described. For example, techniques for scalable backup solutions for non-relational databases are described. A data management system (DMS) may use different buffer pools for the extraction of data from the non-relational database and the movement (e.g., copying or transferring of data) from the buffers to a remote storage location. A backup process for a collection may involve one or more iterations involving retrieving or selecting a buffer from an empty buffer queue of an agent of the DMS at the non-relational database, filling the buffer with data from the non-relational database to the retrieved buffer, moving the filled buffer to a full buffer queue, moving the data to a remote storage location, and moving the buffer back to the empty buffer queue after moving the data on the buffer to the remote storage location.