Abstract: At least a portion of data of a tree data structure is serialized into a first set of flat data. At least a portion of a structure of the tree data structure is serialized to reproduce at least the portion of the structure in a second set of flat data. It is determined to access a desired data of the tree data structure from an archive. The second set of flat data is read to reconstitute at least the portion of a structure of the tree data structure. The reconstituted structure of the tree data structure is used to determine a data offset associated with the first set of flat data, wherein the data offset corresponds to the desired data. The desired data is accessed using the data offset associated with the first set of flat data.

Abstract: A method and system for restoring a database are described. An identification of a restoration point of the database is received. Using a processor, metadata of a plurality of backups are analyzed to identify from the plurality of backups a reduced dataset required to restore the database to the restoration point. The reduced dataset is provided for use in restoring the database to the restoration point.

Abstract: A method and system for migrating data are described. A storage capture instance in a first format from a first type of device is received. Processing utilized to back up the storage capture instance to a second format of a snapshot tree is distributed to multiple storage nodes. A request to migrate at least part of the storage capture instance to a second type of device in a third format is received. In response to the request, processing utilized to migrate the at least part of the storage capture instance from the second format to the second type of device in the third format is distributed to the storage nodes. In one aspect, file(s) in a first network attached storage (NAS) device format are migrated to a second NAS device format. In another aspect, a virtual machine (VM) and/or file(s) therein having a first VM format are migrated to a second VM format.

Abstract: A method and system for managing a storage cluster is described. An identification of a plurality of nodes of a storage cluster to be removed from the storage cluster is received. It is determined whether to allow the removal of the plurality of nodes. In response to a determination that the removal of the plurality of nodes is allowed, identifications are made of data migrations to be performed to rebalance the data among nodes remaining in the storage cluster after removal of the plurality of nodes. The data migrations are performed. The plurality of nodes are removed from the storage cluster.

Abstract: Approaches for parallelized data deduplication. An instruction to perform data deduplication on a plurality of files is received. The plurality of files is organized into two or more work sets that each correspond to a subset of the plurality of files. Responsibility for performing each of said two or more work sets is assigned to a set of nodes in a cluster of nodes. The nodes may be physical nodes or virtual nodes. Each node in the set performs data deduplication on a different work set. In performing data deduplication, each node may store metadata describing where shared chunks of data are maintained in a distributed file system. The shared chunks of data are two or more sequences of bytes which appear in two or more of said plurality of files.

Abstract: A virtual machine container file is analyzed to determine which portion of the virtual machine container file corresponds to a virtual machine file system metadata of the virtual machine container file. One or more differences between a first version of a virtual machine container file and a second version of the virtual machine container file are determined at least in part by traversing a snapshot structure associated with the virtual machine container file. The determined one or more differences that corresponds to the virtual machine file system metadata portion of the virtual machine container file are identified based at least in part on the analysis of the virtual machine container file.

Abstract: A method and apparatus for maintaining file system consistency in a distributed system using version vectors is presented. A method generally includes comparing incarnation and transaction identifiers of a current version vector associated with a file with incarnation and transaction identifiers of a last completed version vector associated with the file. Upon determining that a current version vector reflects operations on the file that are either earlier than or the same as the identifiers in the last completed version vector, the node performing one or more file system operations on the file.

Abstract: An identifier associated with a persistent connection virtualization container corresponding to a specified secondary storage system identifier associated with a secondary storage system is requested from a caching service virtualization container. The identifier associated with the persistent connection virtualization container corresponding to the specified secondary storage system identifier is received. One or more commands are provided to the persistent connection virtualization container associated with the secondary storage system.

Abstract: At least a portion of a virtual machine is hosted on at least one node of a first subset of a plurality of nodes of a secondary storage system. The virtual machine comprises a plurality of portions that can be distributed between the plurality of nodes and is configured into a first state of a plurality of states, such that, in the first state, the plurality of portions is distributed between a first subset of the plurality of nodes and each of the first subset of nodes stores a portion of the virtual machine in its corresponding storage device. A node from the second subset of the plurality of nodes to host the virtual machine in a second state of the plurality of states is selected based on at least one of storage, memory or processing resources of one or more nodes of a second subset of the plurality of nodes.

Abstract: A request to write data corresponding to at least a first portion of a file is received. It is determined whether to perform the request either as an in-place write or as an out-of-place write. Performing the in-place write comprises performing a write to a low latency storage device, and performing the out-of-place write comprises performing a write to a higher latency storage device. The request is performed as either the in-place write or the out-of-place write based on the determination. Performing the request as the in-place write includes writing the data to a first location on a storage tier storing the first portion of the file, and performing the request as the out-of-place write includes writing the data to a second location on one of a plurality of storage tiers of a computing node, other than the first location.

Abstract: Techniques for cloning, writing to, and reading from file system metadata. Cloning involves identifying a first set of pointers included in a first root node in a file system metadata tree structure that stores file system metadata n leaf nodes of the tree structure, creating a first copy of the first root node that includes the first set of pointers, creating a second copy of the first root node that includes the first set of pointers, associating the first copy with a first view, and associating the second copy with a second view. Reading generally involves traversing the tree structure towards a target leaf node that contains data to be read. Writing generally involves traversing the tree structure n the same manner, but also creating copies of any nodes to be modified if those nodes are deemed to have a different treeID than a particular root node.

Abstract: A first persistent connection pipe is established from a node of a secondary storage system to an access service associated with a cloud service system. A second persistent connection pipe is established from the node of the secondary storage system to the access service. A communication direction role of the second persistent connection pipe is switched to enable the access service to initiate a communication to the node of the secondary storage system via the second persistent pipe.

Abstract: Techniques are disclosed which allow a secondary storage system to provide data to non-production workloads in conjunction with performing data backup and protection tasks. As disclosed, a secondary storage system exposes backup data stored by the secondary storage system to other workloads, such as test and development applications, data analytics, etc. These non-production workloads can run at the same time the secondary storage system provides backup services to a primary storage system. This consolidation eliminates the need for an enterprise to deploy separate storage clusters for analytics, test and development applications, etc. and eliminates unnecessary copies of data.

Abstract: File access statistics associated with a content file are maintained by a file tier agent of a secondary storage system. The content file is accessible via a primary storage system. A target storage tier corresponding to the file access statistics associated with the content file is determined according to one or more policies. At least a portion of data of the content file is migrated to the determined storage tier while accessibility of the content file via the primary storage system is maintained.

Abstract: Cluster state information is generated in response to a request to establish a connection with a cloud service system. The cluster state information includes a first instance of a security token and host information. The cluster state information is provided to a web browser associated with a user. The web browser associated with the user is redirected to a cloud identity provider. The cloud identity provider is configured to provide to the cloud service system via the web browser associated with the user, the cluster state information that includes the first instance of the security token and the host information. A certificate is requested from the cloud service system. The cluster state information that includes a second instance of the security token is provided to the cloud service system. The cloud service system is configured to establish the connection based on a comparison between the first instance of the security token and the second instance of the security token.

Abstract: Embodiments presented herein disclose techniques for capturing a snapshot of a file system object (e.g., a file or a directory) that is associated with a write journal having outstanding data. A bridge process in a storage server receives a request to capture a snapshot of a file system object. The snapshot is a backup of a state of the file system object in a given point in time. Upon determining that the file system object has one or more outstanding updates recorded in a write journal, the bridge process generates a copy of the write journal. The bridge process captures the snapshot of the file system object. The bridge process also associates the copy of the write journal with the snapshot of the file system object.

Abstract: Application-level data in a storage system are evaluated. For example, a backup analysis tool retrieves a backup object stored in the storage system. The backup analysis tool reconstructs an application object from the backup object. The backup analysis tool accesses the first application object according to a native application format associated with the first object.

Abstract: Embodiments presented herein disclose adaptive techniques for scheduling self-maintenance processes. A load predictor estimates, based on a current state of a distributed storage system, an amount of resources of the system required to perform each of a plurality of self-maintenance processes. A maintenance process scheduler estimates, based on one or more inputs, an amount of resources of the distributed system available to perform one or more of the self-maintenance processes during at least a first time period. The maintenance process scheduler determines a schedule for the one or more of the self-maintenance processes to perform during the first time period, based on the estimated amount of resources required and available.

Abstract: A method and apparatus for dynamically adjusting an ingestion rate for backup operations on a source system. The method generally includes monitoring a resource utilization related to one or more performance metrics of the source system in performing at least a primary workload. Based on the monitored resource utilization, the backup system determines a data ingestion rate for backup operations on the source system. The backup system ingests data from the source system to a backup repository at the determined data ingestion rate.

Abstract: A request associated with restoring a previous version of a linked clone virtual machine is received. One or more changes between a base image of a parent virtual machine and the previous version of the linked clone virtual machine are determined. One or more data blocks corresponding to the base image and one or more data blocks corresponding to the determined changes are provided to a remote system.