Abstract: The subject technology generates a dataset based at least in part on a set of files. The subject technology generates, utilizing a machine learning model, a set of labels corresponding to the dataset. The subject technology filters the dataset using a set of conditions to generate at least a subset of the dataset. The subject technology generates a virtual object based at least in part on the subset of the dataset and the set of labels, where the virtual object corresponds to a selection of data from the dataset. The subject technology trains a second machine learning model using the virtual object and at least the subset of the dataset, where training the second machine learning model includes utilizing streaming file input/output (I/O), the streaming file I/O providing access to at least the subset of the dataset during training.

Abstract: A method, system, and computer program product for providing, via a provisioning engine, a scalable set of indexed key-value pairs enabled to store objects in a data storage environment; wherein the data representing the objects is enabled to be spread across arrays in the data storage environment; wherein additional arrays are enabled to be added to the data storage environment and included in the indexed key-value pairs; wherein the data stored across the arrays may be balanced.

Abstract: A computer implemented method, system, and computer program product for enabling strong consistency of unique objects between zones comprising enabling consistency of an unique data object between in zones by switching between consistency techniques to keep the unique data object consistent between the zones; wherein read and write access is enabled to the unique object in the zones.

Abstract: A computer-executable method, computer program product, and system for managing backups in a distributed data storage system including a first zone, a second zone, and a third zone, the computer-executable method, computer program product, and system comprising processing, at the third zone, a first portion of data of the first zone and a second portion of data of the second zone to create a combined portion of data, and removing the first portion and second portion from the third zone.

Abstract: A computer-executable method, computer program product, and system of managing I/O requests in a distributed data storage system, wherein the distributed data storage system includes a first node and one or more data storage array, the computer-executable method, computer program product, and system comprising receiving an I/O request at the first node enabled to utilize a storage engine to process the I/O request, wherein the storage engine is comprised of two or more layers, analyzing the I/O request using a first layer of the two or more layers to determine whether the I/O request relates to a portion of metadata managed by the first layer of the two or more layers, and processing the I/O request based on the determination.

Abstract: A computer-executable method, computer program product, and system for managing metadata within a distributed data storage system, including a compute node in communication with a data storage array, the computer-executable method, computer program product, and system comprising receiving a data I/O from an application executing within the distributed data storage system, and creating a first storage system within the compute node, wherein the first storage system is enabled to manage metadata related to the data I/O, and processing the data I/O using the first storage system.

Abstract: An object storage system having object metadata query capability. Clients can pose queries to the object system in a high-level, declarative language. A query is evaluated against stored object metadata, and only objects satisfying the query's predicates are returned. The object storage system can evaluate alternative logical and physical query plans using a cost model. Queries can be posed using familiar REST-based APIs. The object storage system includes a distributed key-value store for secondary indexing of object metadata. A user can define secondary indexes considering the most common object access patterns.

Abstract: A computer implemented method, computer program product, and system for providing, via a storage provisioning engine, a scalable objects store enabled to store objects across multiple heterogeneous file arrays; wherein file arrays are enabled to be actively added to the object store without pausing the file arrays; and wherein data representing the objects enabled to be balanced across the heterogeneous file arrays based.

Abstract: A computer implemented method, system, and computer program product for enabling consistency between zones comprising creating a lease agreement between a first zone and at least a second zone; wherein the lease indicates that the first zone is the owner of an object; wherein the lease agreement dictates that a notification is to be sent before a cached object in the first zone is updated if the lease is still valid; wherein the lease indicates the first zone is to send a heartbeat to the second zone within the predetermined period of time if the object has not been changed and the lease is still valid; and setting up heartbeats from the first zone to at least a second zone within a predetermined amount of time; wherein the heartbeat indicates that the lease is still valid and the object has not been changed.

Abstract: In one aspect, a method includes using a file system layer configured to interact with a plurality of volumes and enabling an application to interact with any of the plurality of volumes using a single file system format provided by the file system layer. At least two of the plurality of volumes have different file system formats.

Abstract: A database associated with a primary database cluster may be replicated in a backup database cluster located in a different location in order to provide a highly-available fault tolerant database service. The databases are reconciled through a cluster management module distributed in each database cluster. The cluster management module uses a set of reconciliation data structures to track locally the reconciled states of each database in each database cluster, the operations made locally to the databases in each database cluster, and the author of the operations. The cluster management module in each database cluster engages in a stateless messaging protocol using the set of reconciliation data structures to determine whether or not the databases may be reconciled.

Abstract: In one aspect, a method includes enabling a REST interface to have access to a volume, receiving a request to allow a native file access to the volume and allowing an application to use a native file interface to have access to the volume while preventing the modifications to the volume through the REST interface if the request is received.

Abstract: A method, system, and computer program product for providing, via a provisioning engine, a scalable set of indexed key-value pairs enabled to store objects in a data storage environment; wherein the data representing the objects is enabled to be spread across arrays in the data storage environment; wherein additional arrays are enabled to be added to the data storage environment and included in the indexed key-value pairs; wherein the data stored across the arrays may be balanced.

Abstract: A database associated with a primary database cluster may be replicated in a backup database cluster located in a different location in order to provide a highly-available fault tolerant database service. The databases are reconciled through a cluster management module distributed in each database cluster. The cluster management module uses a set of reconciliation data structures to track locally the reconciled states of each database in each database cluster, the operations made locally to the databases in each database cluster, and the author of the operations. The cluster management module in each database cluster engages in a stateless messaging protocol using the set of reconciliation data structures to determine whether or not the databases may be reconciled.

Abstract: Embodiments are directed to providing database access during database reconfiguration and to maintaining replication connections during database reconfiguration. In an embodiment, a computer system establishes multiple quorum sets of replicas to replicate the data of a data partition. The quorum sets of replicas ensure that at least a minimum number of replicas are operating to commit pending transactions during partition reconfiguration. The computer system determines that a data partition reconfiguration has been initiated and provides access to the data partition's data during reconfiguration of the data partition using at least a quorum of replicas in each of the quorum sets of replicas.

Abstract: Embodiments are directed to providing database access during database reconfiguration and to maintaining replication connections during database reconfiguration. In an embodiment, a computer system establishes multiple quorum sets of replicas to replicate the data of a data partition. The quorum sets of replicas ensure that at least a minimum number of replicas are operating to commit pending transactions during partition reconfiguration. The computer system determines that a data partition reconfiguration has been initiated and provides access to the data partition's data during reconfiguration of the data partition using at least a quorum of replicas in each of the quorum sets of replicas.