Abstract: In some aspects, an apparatus includes a processor and a memory. In some embodiments, the memory includes programmed instructions that, when executed by the processor, cause the apparatus to receive a request to create a bucket. In some embodiments, the memory includes programmed instructions that, when executed by the processor, cause the apparatus to, in response to receiving the request to create the bucket, write, to a first object store, a first mapping that maps an identifier of global metadata to an identifier of a second object store where the bucket is to be created, write, to the second object store, a second mapping that maps the identifier of global metadata to local metadata, and replicate, to the second object store, the first mapping.

Abstract: In accordance with some aspects of the present disclosure, a non-transitory computer readable medium is disclosed. The non-transitory computer readable medium includes instructions when executed by a processor cause the processor to write an object to an object store using an object-based access protocol, look up the object from the object store using a file-based access protocol, and create one or more directories on-demand in the object store.

Abstract: A unified namespace technique provides coherent access to unstructured data across different data access protocols having different logical constructs that are stored and managed on a storage system. A control plane infrastructure operates in connection with storage services to provide support for a vast array of storage platforms including file servers of a file system and object storage servers of an object store. Metadata associated with a data access transaction is processed separately and natively by a protocol stack of a particular storage service according to a particular data access protocol. The processed metadata is stored native to the access protocol in a metadata store associated with the particular storage service and is made available to the protocol stacks of the other storage services. Processed metadata is made available to the protocol stacks via an event notification logging service implemented as a message bus.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a processor with programmed instructions to receive a request to process storage data using a compute application, determine that a first location of the storage data is in a first storage resource in a first cluster of compute, storage, and network resources, determine that a second location of the compute application is in a first compute resource of a second cluster of compute, storage, and network resources, determine whether the first cluster includes a second compute resource that is compatible with the compute application, and respectively, either, based on a first compatibility determination, migrate the compute application from the first compute resource to the second compute resource, or, based on a second compatibility determination, replicate the storage data from the first storage resource to a second storage resource in the second cluster.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a processor and a memory. In some embodiments, the memory includes programmed instructions that, when executed by the processor, cause the apparatus to store a first object and a second object in a first region based on the first object and the second object having a first policy. In some embodiments, the memory includes programmed instructions that, when executed by the processor, cause the apparatus to store a third object in a second region based on the third object having a second policy. In some embodiments, a virtual disk includes the first region and the second region.

Abstract: In accordance with some aspects of the present disclosure, a non-transitory computer readable medium is disclosed. The non-transitory computer readable medium includes instructions when executed by a processor cause the processor to write an object to an object store using an object-based access protocol, look up the object from the object store using a file-based access protocol, and create one or more directories on-demand in the object store.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a processor and a memory. In some embodiments, the memory includes programmed instructions that, when executed by the processor, cause the apparatus to store a first object and a second object in a first region based on the first object and the second object having a first policy. In some embodiments, the memory includes programmed instructions that, when executed by the processor, cause the apparatus to store a third object in a second region based on the third object having a second policy. In some embodiments, a virtual disk includes the first region and the second region.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a processor having programmed instructions to map a plurality of objects to a region and map the region to a plurality of vdisks. The plurality of objects are stored in the plurality of vdisks. The processor has programmed instructions to, responsive to mapping the plurality of objects to the region, identify a policy associated with the region. The policy specifies a policy action in response to satisfying a predetermined condition. The processor has programmed instructions to, responsive to satisfying the predetermined condition, access the plurality of vdisks and perform the policy action on the plurality of objects.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a processor with programmed instructions to receive a request to process storage data using a compute application, determine that a first location of the storage data is in a first storage resource in a first cluster of compute, storage, and network resources, determine that a second location of the compute application is in a first compute resource of a second cluster of compute, storage, and network resources, determine whether the first cluster includes a second compute resource that is compatible with the compute application, and respectively, either, based on a first compatibility determination, migrate the compute application from the first compute resource to the second compute resource, or, based on a second compatibility determination, replicate the storage data from the first storage resource to a second storage resource in the second cluster.

Abstract: Exemplary methods, apparatuses, and systems maintain a plurality of summary data structures corresponding to a plurality of logical file system namespaces representing a plurality of hierarchies of one or more directories having one or more files, each file being stored in the storage system as a plurality of segments in a deduplicated manner. In response to a request to estimate a storage usage by a first of the file system namespace, identify a first of the summary data structures corresponding to the first file system namespace, wherein the first summary data structure stores information summarizing deduplicated segments referenced by one or more files of the first file system namespace. Estimate the storage usage of the first file system namespace based on the first summary data structure and a global summary data structure, wherein the global summary data structure stores information summarizing deduplicated segments referenced by all of the file system namespaces.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a processor having programmed instructions to send an application programming interface (API) write request to a first virtual machine (VM) on a first node to write an object, receive a response to the API write request including a physical disk location of a physical disk to which the object is written, wherein the physical disk is located on a second node, and using the physical disk location, send an API read request to a second VM on the second node to read the object.

Abstract: An illustrative embodiment disclosed herein is an apparatus including a processor having programmed instructions to map a plurality of objects to a region and map the region to a plurality of vdisks. The plurality of objects are stored in the plurality of vdisks. The processor has programmed instructions to, responsive to mapping the plurality of objects to the region, identify a policy associated with the region. The policy specifies a policy action in response to satisfying a predetermined condition. The processor has programmed instructions to, responsive to satisfying the predetermined condition, access the plurality of vdisks and perform the policy action on the plurality of objects.

Abstract: Embodiments are described for repairing the locality of a namespace index on a deduplication storage device. A namespace index is selected from a plurality of namespace indices, each of which is mountable on the storage system. The selected namespace index is traversed and divided into regions, each of which was stored in one or more storage containers. Locality information for each region is generated, including a density metric and a distribution metric. If either metric is below a threshold value for the metric, then the region is enqueued for namespace locality repair. A region can be repaired by reading the region into memory and flushing the memory to co-located containers on the storage system.

Abstract: An indication is received that a data object is to be deleted, wherein the data object comprises data stored in a segment within a container. It is determined no currently alive data object references any segment within the container. The container is placed in a delete-ready but not yet reclaimable state.

Abstract: A method, article of manufacture, and apparatus for scaling an index on a solid state drive in a deduplicated storage system is discussed. The index may be scaled as index hash buckets approach capacity, or as new solid state drives are added to the system. The hash buckets may be mapped to HDD hash buckets, which are stored on a hard drive.

Abstract: According to one embodiment, a request for seeding a predetermined number of small files with a predetermined locality in a storage system is received, each of the files to have a predetermined file size. In response to the request, a plurality of segments and fingerprints of the segments are generated. File trees representing the predetermined number of files respectively are generated based on the fingerprints of the segments, each of the files represented by the segments having the predetermined file size. A namespace representing one or more directories of the files is generated based on the file trees, where each of the directories of files satisfies the predetermined locality. The namespace and segments corresponding to the files of one or more directories are written to a storage device of the storage system.

Abstract: In an embodiment of the present invention, a method can include converting a data-full snapshot having a plurality of user data and corresponding metadata to a dataless snapshot. The dataless snapshot stores the metadata corresponding to the user data. Converting the data-full snapshot to the dataless snapshot includes removing the user data from the data-full snapshot. The metadata can be at least one of a checksum or hash of the corresponding user data.

Abstract: In an embodiment, a method can include loading a first snapshot of data stored on a storage device, the first snapshot being verified. The method can further include capturing a second snapshot of data stored on the store device after waiting an interval of time from creation of the first snapshot. The method can further include generating a list of closed files between the two snapshots by differentiating the first snapshot and the second snapshot. The method can additionally include verifying the second snapshot by comparing the closed files in the list of closed files by in the second snapshot to the closed files in the storage device, which is an active snapshot. The method can also include deleting the first snapshot.

Abstract: An indication is received that a data object is to be deleted, wherein the data object comprises data stored in a segment within a container. It is determined no currently alive data object references any segment within the container. The container is placed in a delete-ready but not yet reclaimable state.

Abstract: A garbage collector of a storage system traverses a namespace of a file system of the storage system to identify segments that are alive in a breadth-first manner. The namespace includes information identifying files that are represented by segments arranged in a plurality of levels in a hierarchy, where an upper level segment includes one or more references to one or more lower level segments, and at least one segment is referenced by multiple files. All live segments of an upper level are identified before any of live segments of a lower level are identified. Upon all live segments of all levels have been identified, the live segments are copied from their original storage locations to a new storage location, and a storage space associated with the original storage locations is reclaimed.