Abstract: Maintenance is performed to determine one or more content files associated with a stored backup snapshot having a size that is greater than a threshold size. It is determined that the size of a content file of the one or more content files is greater than the threshold size. In response to determining that the size of the content file is greater than the threshold size, a new tree data structure is generated and a component file metadata structure corresponding to the content file is split into a plurality of component file metadata structures for the content file.

Abstract: Techniques for cloning, writing to, and reading from file system metadata. Cloning involves identifying a first set of pointers included h a first root node in a file system metadata tree structure that stores file system metadata n leaf nodes of the tree structure, creating a first copy of the first root node that includes the first set of pointers, creating a second copy of the first root node that includes the first set of pointers, associating the first copy with a first view, and associating the second copy with a second view. Reading generally involves traversing the tree structure towards a target leaf node that contains data to be read. Writing generally involves traversing the tree structure in the same manner, but also creating copies of any nodes to be modified if those nodes are deemed to have a different treeID than a particular root node.

Abstract: An indication to perform a backup of file system data is received. The file system data includes a content file having a size greater than a threshold size. A plurality of component file metadata structures for the content file are generated. File metadata of the content file split across the plurality of component file metadata structures is stored. The plurality of component file metadata structures are associated with different portions of the content file. A component file metadata structure of the plurality of component file metadata structures stores file metadata corresponding to a portion of the content file. The file metadata corresponding to the portion of the content file includes one or more references to locations of data chunks associated with the portion of the content file.

Abstract: A tree data structure for a file is maintained. The tree data structure is used to capture different versions of the file at different moments of time. An access pattern associated with the file stored on a storage system is determined based at least in part on time-series data associated with the file using the tree data structure that captures different versions of the file at different moments in time. The determined access pattern indicates a period for which the file is likely to be accessed. The file is migrated from a secondary storage tier to a primary storage tier.

Abstract: Disclosed herein are embodiments for managing the placement of virtual machines in a virtual machine network. In an embodiment, a method involves determining whether to separate at least one virtual machine in a set of virtual machines supporting a process and running on a first host computer from other virtual machines in the set. If at least one virtual machine is to be separated, then at least one virtual machine is selected based on a number of memory pages changed. The selected virtual machine is then separated from the other virtual machines in the set.

Abstract: A method and apparatus for maintaining file system consistency in a distributed system using version vectors is presented. A method generally includes comparing incarnation and transaction identifiers of a current version vector associated with a file with incarnation and transaction identifiers of a last completed version vector associated with the file. Upon determining that a current version vector reflects operations on the file that are either earlier than or the same as the identifiers in the last completed version vector, the node performing one or more file system operations on the file.

Abstract: According to one embodiment of the present disclosure, a first set of file system objects included in performing a requested file system operation is identified in response to a request to perform a file system operation. An update intent corresponding to the requested file system operation is inserted into a data structure associated with each identified file system object. Each file system object corresponding to the corresponding data structure is modified as specified by the update intent in that data structure. After modifying the file system object corresponding to the corresponding data structure, the update intent is removed from that data structure.

Abstract: A plurality of virtual internet protocol addresses for a first single network interface card of a node of a storage cluster are provided to a client. A separate connection is established between the client and the node for each of the plurality of virtual internet protocol addresses. The separate connections are utilized together in parallel to transfer data between the client and the node.

Abstract: An inode is associated with an incomplete file system operation to a file system object is determined based on an update intent stored in the inode. It is determined that a set of inodes is associated with the incomplete file system operation based on the update intent stored in the inode. The update intent indicates the set of inodes is associated with the incomplete file system operation. It is determined to complete the incomplete file system operation based on evaluating the set of inodes that is associated with the incomplete file system operation. Evaluating the set of inodes that is associated with the incomplete file system operation includes determining whether the set of inodes that is associated with the incomplete file system operation stores a corresponding update intent.

Abstract: A plurality of portions of a content file are stored. It is determined that the content file has a size that is greater than a threshold size. In response to determining that the content file has the size that is greater than the threshold size, a plurality of component file metadata structures are generated for each of the plurality of portions of the content file. A component file metadata structure of the plurality of component file metadata structures corresponds to one of the portions of the content file. Each of the plurality of component file metadata structures includes corresponding metadata that enables data chunks associated with a corresponding portion of the content file to be located.

Abstract: A request to write data corresponding to at least a first portion of a file is received. It is determined whether to perform the request either as an in-place write or as an out-of-place write. Performing the in-place write comprises performing a write to a low latency storage device, and performing the out-of-place write comprises performing a write to a higher latency storage device. The request is performed as either the in-place write or the out-of-place write based on the determination. Performing the request as the in-place write includes writing the data to a first location on a storage tier storing the first portion of the file, and performing the request as the out-of-place write includes writing the data to a second location on one of a plurality of storage tiers of a computing node, other than the first location.

Abstract: A first fingerprint corresponding to a first chunk associated with a stream of data is generated. It is determined that the first fingerprint matches a second fingerprint of a plurality of fingerprints listed in at least one entry in a deduplication map associated with a plurality of storage systems. A first storage system of the plurality of storage systems is located at a first geographic location and a second storage system of the plurality of storage systems is located at a second geographic location. The first chunk corresponding to the second fingerprint is stored by at least the second storage system. In response to a determination that the first fingerprint matches the second fingerprint, it is determined to store at the first storage system a local copy of the first chunk based in part on one or more deduplication factors.

Abstract: A plurality of virtual internet protocol addresses for a first single network interface card of a node of a storage cluster are provided to a client. A separate connection is established between the client and the node for each of the plurality of virtual internet protocol addresses. The separate connections are utilized together in parallel to transfer data between the client and the node.

Abstract: A size associated with a content file is determined to be greater than a threshold size. Contents of the content file split across a plurality of component files are stored. Metadata, for the content file, is updated to reference a plurality of component file metadata structures for the component files. A node of the metadata is configured to track different sizes of portions of the content file stored in different component files of the plurality of component files. File metadata of the content file is split across the plurality of component file metadata structures and each component file metadata structure of the plurality of component file metadata structures specifies a corresponding structure organizing data components for a corresponding portion of the content file.

Abstract: Embodiments described herein provide techniques for maintaining consistency in a distributed system (e.g., a distributed secondary storage system). According to one embodiment of the present disclosure, a first set of file system objects included in performing the requested file system operation is identified in response to a request to perform a file system operation. An update intent corresponding to the requested file system operation is inserted into an inode associated with each identified file system object. Each file system object corresponding to the inode is modified as specified by the update intent in that inode. After modifying the file system object corresponding to the inode, the update intent is removed from that inode.

Abstract: Embodiments presented herein describe techniques for deduplicating chunks of data across multiple clusters. A process executing in a storage system identifies one or more chunks in an incoming stream of data. For each chunk, a first fingerprint corresponding to the chunk is generated. The process determines whether the first fingerprint matches a second fingerprint listed in a corresponding entry in a deduplication map. Each entry of the deduplication map corresponds to a chunk stored in a location in one of the storage clusters. Upon determining that the first fingerprint matches the second fingerprint, the process writes, to a local persistent storage, a pointer referencing the location in that storage cluster.

Abstract: A tree data structure for a file is maintained. The tree data structure is used to capture different versions of the file at different moments of time. An access pattern associated with the file stored on a storage system is determined based at least in part on time-series data associated with the file using the tree data structure that captures different versions of the file at different moments in time. The determined access pattern indicates a period for which the file is likely to be accessed. The file is migrated from a secondary storage tier to a primary storage tier.

Abstract: An indication to perform a backup of file system data is received. The file system data includes a content file having a size greater than a threshold size. A plurality of component file metadata structures for the content file are generated. File metadata of the content file split across the plurality of component file metadata structures is stored. The plurality of component file metadata structures are associated with different portions of the content file. A component file metadata structure of the plurality of component file metadata structures stores file metadata corresponding to a portion of the content file. The file metadata corresponding to the portion of the content file includes one or more references to locations of data chunks associated with the portion of the content file.

Abstract: File access statistics associated with a content file are maintained by a file tier agent of a secondary storage system. The content file is accessible via a primary storage system. A target storage tier corresponding to the file access statistics associated with the content file is determined according to one or more policies. At least a portion of data of the content file is migrated to the determined storage tier while accessibility of the content file via the primary storage system is maintained.

Abstract: A size associated with a content file is determined to be greater than a threshold size. In response to the determination, file metadata of the content file split and stored across a plurality of component file metadata structures. The file metadata of the content file specifies tree structure organizing data components of the content file and each component file metadata structure of the plurality of component file metadata structures stores a portion of the tree structure. A snapshot tree is updated to reference the plurality of component file metadata structures for the content file.