Abstract: Metadata structures and related locking techniques to improve performance and scalability in a cluster file system are disclosed. In one implementation, a method includes receiving, by a server, a request for a first file, retrieving, by the server, the first file and metadata for a second file that is related to the first file, and sending, by the server in response to the retrieving and in advance of any request for the metadata for the second file, an asynchronous lock request to obtain a lock for accessing at least a portion of the metadata for the second file.

Abstract: In one embodiment of the present invention, a data-backup method includes partitioning a fingerprint namespace among a cluster of backup servers, the fingerprint namespace comprising fingerprints for representing units of data, each backup server of the cluster of backup servers managing units of data having fingerprints corresponding to an assigned partition of the fingerprint namespace. The method further includes receiving backup information from a client computing device for a block of data comprising units of data, the backup information including at least a fingerprint for each of the units of data and client-specific backup information. In addition, the method includes, utilizing the fingerprint for each of the units of data, deduplicating the units of data in parallel at the cluster of backup servers in accordance with the partitioning step, the deduplicating step comprising identifying ones of the units data already stored by the cluster of backup servers.

Abstract: Metadata structures and related locking techniques to improve performance and scalability in a cluster file system are disclosed. In one implementation, a method includes receiving, by a server, a request for a first file, retrieving, by the server, the first file and metadata for a second file that is related to the first file, and sending, by the server in response to the retrieving and in advance of any request for the metadata for the second file, an asynchronous lock request to obtain a lock for accessing at least a portion of the metadata for the second file.

Abstract: A cluster file system is disclosed. A plurality of disk servers, operating as a single distributed disk server layer, are connected to the one or more physical disks. Each disk server stores metadata for each of the files. A plurality of lock servers, having one or more locks for each file and associated file system metadata operates as a single centralized distributed lock server to coordinate access to the files stored by the disk server layer. A plurality of asynchronous file servers, operating as a single distributed file server layer in response to a request for a file from a user program: (i) retrieves the requested file from the disk server layer and at least one lock, associated with the retrieved file, from the lock server, and (ii) retrieves metadata for at least one file that is related to the requested files, and at least one lock, for each file that has had its metadata retrieved.

Abstract: In one embodiment of the present invention, a data-backup method includes partitioning a fingerprint namespace among a cluster of backup servers, the fingerprint namespace comprising fingerprints for representing units of data, each backup server of the cluster of backup servers managing units of data having fingerprints corresponding to an assigned partition of the fingerprint namespace. The method further includes receiving backup information from a client computing device for a block of data comprising units of data, the backup information including at least a fingerprint for each of the units of data and client-specific backup information. In addition, the method includes, utilizing the fingerprint for each of the units of data, deduplicating the units of data in parallel at the cluster of backup servers in accordance with the partitioning step, the deduplicating step comprising identifying ones of the units data already stored by the cluster of backup servers.

Abstract: In one embodiment of the present invention, a data-backup method includes partitioning a fingerprint namespace among a cluster of backup servers, the fingerprint namespace comprising fingerprints for representing units of data, each backup server of the cluster of backup servers managing units of data having fingerprints corresponding to an assigned partition of the fingerprint namespace. The method further includes receiving backup information from a client computing device for a block of data comprising units of data, the backup information including at least a fingerprint for each of the units of data and client-specific backup information. In addition, the method includes, utilizing the fingerprint for each of the units of data, deduplicating the units of data in parallel at the cluster of backup servers in accordance with the partitioning step, the deduplicating step comprising identifying ones of the units data already stored by the cluster of backup servers.

Abstract: A computer-based method includes determining, based at least in part on a quantity of first backup data on a source computer, a data-transfer mode for the first backup data, the data-transfer mode selected from the group consisting of network transfer of the first backup data and physical-media transfer of the first backup data and, responsive to the determining step, transferring the first backup data via the determined data-transfer mode to a server located at a remote destination from the source computer. The physical-media data-transfer mode of the first backup data includes physical transport of physical media to the server. The network-transfer data-transfer mode is accomplished via a network connection and does not include physical transport of physical media.

Abstract: In one embodiment of the present invention, a data-backup method includes partitioning a fingerprint namespace among a cluster of backup servers, the fingerprint namespace comprising fingerprints for representing units of data, each backup server of the cluster of backup servers managing units of data having fingerprints corresponding to an assigned partition of the fingerprint namespace. The method further includes receiving backup information from a client computing device for a block of data comprising units of data, the backup information including at least a fingerprint for each of the units of data and client-specific backup information. In addition, the method includes, utilizing the fingerprint for each of the units of data, deduplicating the units of data in parallel at the cluster of backup servers in accordance with the partitioning step, the deduplicating step comprising identifying ones of the units data already stored by the cluster of backup servers.

Abstract: A computer-based method includes determining, based at least in part on a quantity of first backup data on a source computer, a data-transfer mode for the first backup data, the data-transfer mode selected from the group consisting of network transfer of the first backup data and physical-media transfer of the first backup data and, responsive to the determining step, transferring the first backup data via the determined data-transfer mode to a server located at a remote destination from the source computer. The physical-media data-transfer mode of the first backup data includes physical transport of physical media to the server. The network-transfer data-transfer mode is accomplished via a network connection and does not include physical transport of physical media.

Abstract: A cluster file system is disclosed. A plurality of disk servers, operating as a single distributed disk server layer, are connected to the one or more physical disks. Each disk server stores metadata for each of the files. A plurality of lock servers, having one or more locks for each file and associated file system metadata operates as a single centralized distributed lock server to coordinate access to the files stored by the disk server layer. A plurality of asynchronous file servers, operating as a single distributed file server layer in response to a request for a file from a user program: (i) retrieves the requested file from the disk server layer and at least one lock, associated with the retrieved file, from the lock server, and (ii) retrieves metadata for at least one file that is related to the requested files, and at least one lock, for each file that has had its metadata retrieved.

Abstract: A cluster file system is disclosed. A plurality of disk servers, operating as a single distributed disk server layer, are connected to the one or more physical disks. Each disk server stores metadata for each of the files. A plurality of lock servers, having one or more locks for each file and associated file system metadata operates as a single centralized distributed lock server to coordinate access to the files stored by the disk server layer. A plurality of asynchronous file servers, operating as a single distributed file server layer in response to a request for a file from a user program: (i) retrieves the requested file from the disk server layer and at least one lock, associated with the retrieved file, from the lock server, and (ii) retrieves metadata for at least one file that is related to the requested files, and at least one lock, for each file that has had its metadata retrieved.

Abstract: A distributed file system is disclosed. A plurality of disk servers, operating as a single distributed disk server layer, are connected to the one or more physical disks. Each disk server stores meta-data for each of the files. A plurality of lock servers, having one or more locks for each file operates as a single distributed lock server to coordinate access to the files stored by the disk server layer. A plurality of asynchronous file servers, operating as a single distributed file server layer in response to a request for a file from a user program: (i) retrieves the requested file from the disk server layer and at least one lock, associated with the retrieved file, from the lock server, and (ii) retrieves meta-data for at least one file that is related to the requested files, and at least one lock, for each file that has had its meta-data retrieved.