Abstract: Methods, systems, and computer programs are presented for deduplicating data. One method includes an operation for receiving a data block having a logical address. The storage system includes a logical address mapping index for mapping logical addresses to block numbers, a block index for mapping block numbers to physical locations, and a fingerprint index for mapping fingerprints of data blocks to block numbers. Additionally, the method includes an operation for inline processing the data block. Inline processing the data block includes determining a fingerprint of the data block, examining the fingerprint index to determine if the fingerprint is already mapped to an existing data block in the storage system, if the fingerprint is already mapped then adding a mapping of the logical address to the existing data block in the logical address mapping index, and if the fingerprint is not already mapped then creating the corresponding entries in the indices.

Abstract: A storage administrator may maintain location information in separate layers. A data storage system may identify the location of particular data by identifying the virtual location of data, such as the logical extent to which the data belongs. Object stores may maintain mappings of virtual locations to physical locations, such as mappings of extent identifiers to virtual storage objects and mappings of virtual storage objects to storage unit locations. When particular data is relocated to a new location, a storage administrator may update mappings used to translate virtual locations to physical locations, such as an extent-object mapping or an object-storage unit mapping. References to the virtual locations, such as references to logical extent identifiers, may not be updated in response to the relocation of data.

Abstract: Methods, systems, and computer programs are presented for deduplicating data. One method includes an operation for receiving a data block having a logical address. The storage system includes a logical address mapping index for mapping logical addresses to block numbers, a block index for mapping block numbers to physical locations, and a fingerprint index for mapping fingerprints of data blocks to block numbers. Additionally, the method includes an operation for inline processing the data block. Inline processing the data block includes determining a fingerprint of the data block, examining the fingerprint index to determine if the fingerprint is already mapped to an existing data block in the storage system, if the fingerprint is already mapped then adding a mapping of the logical address to the existing data block in the logical address mapping index, and if the fingerprint is not already mapped then creating the corresponding entries in the indices.

Abstract: A computer-implemented process receives a request to utilize one or more virtual data center (VDC) resources at a virtual data center and determines a particular service level applicable to request. Based on the particular service level and mapping information that indicates associations between VDC resource utilization policies and service levels, the process determines a particular VDC resource utilization policy corresponding to the request and causes completion of the request according to the particular VDC resource utilization policy. Another process determines that a resource utilization performance is incompatible with a requested service level and selects a new resource utilization based in part on the resource utilization performance information and mapping information. The process causes data distributed according to a prior resource utilization policy to be distributed according to the new resource utilization policy in one or more resources at a virtual data center.

Abstract: A computer-implemented process receives a request to utilize one or more virtual data center (VDC) resources at a virtual data center and determines a particular service level applicable to request. Based on the particular service level and mapping information that indicates associations between VDC resource utilization policies and service levels, the process determines a particular VDC resource utilization policy corresponding to the request and causes completion of the request according to the particular VDC resource utilization policy. Another process determines that a resource utilization performance is incompatible with a requested service level and selects a new resource utilization based in part on the resource utilization performance information and mapping information. The process causes data distributed according to a prior resource utilization policy to be distributed according to the new resource utilization policy in one or more resources at a virtual data center.

Abstract: A computer-implemented process receives a request to utilize one or more virtual data center (VDC) resources at a virtual data center and determines a particular service level applicable to request. Based on the particular service level and mapping information that indicates associations between VDC resource utilization policies and service levels, the process determines a particular VDC resource utilization policy corresponding to the request and causes completion of the request according to the particular VDC resource utilization policy. Another process determines that a resource utilization performance is incompatible with a requested service level and selects a new resource utilization based in part on the resource utilization performance information and mapping information. The process causes data distributed according to a prior resource utilization policy to be distributed according to the new resource utilization policy in one or more resources at a virtual data center.

Abstract: A storage administrator may maintain location information in separate layers. A data storage system may identify the location of particular data by identifying the virtual location of data, such as the logical extent to which the data belongs. Object stores may maintain mappings of virtual locations to physical locations, such as mappings of extent identifiers to virtual storage objects and mappings of virtual storage objects to storage unit locations. When particular data is relocated to a new location, a storage administrator may update mappings used to translate virtual locations to physical locations, such as an extent-object mapping or an object-storage unit mapping. References to the virtual locations, such as references to logical extent identifiers, may not be updated in response to the relocation of data.

Abstract: A computer-implemented process receives a request to utilize one or more virtual data center (VDC) resources at a virtual data center and determines a particular service level applicable to request. Based on the particular service level and mapping information that indicates associations between VDC resource utilization policies and service levels, the process determines a particular VDC resource utilization policy corresponding to the request and causes completion of the request according to the particular VDC resource utilization policy. Another process determines that a resource utilization performance is incompatible with a requested service level and selects a new resource utilization based in part on the resource utilization performance information and mapping information. The process causes data distributed according to a prior resource utilization policy to be distributed according to the new resource utilization policy in one or more resources at a virtual data center.

Abstract: A storage system, such as a file server, receives a request to perform a write operation that affects a data block. In response, the storage system writes to a storage device the data block together with context information which uniquely identifies the write operation with respect to the data block. When the data block is subsequently read from the storage device together with the context information, the context information that was read with the data block is used to determine whether a previous write of the data block was lost.

Abstract: The present invention provides techniques for managing ownership (i.e., control) of one or more memory buffer (mbuf) data structures within a network subsystem and a storage subsystem of a storage operating system implemented in a storage system. When the storage system receives data to be written to a storage medium, the network subsystem stores the received data in one or more variable-length chains of mbufs. Unlike conventional approaches, the received data is not subsequently copied out of the mbufs into fixed-sized data buffers for use by the storage subsystem. Instead, the storage subsystem can directly manipulate the received data stored in the mbufs. By eliminating the steps of copying data out of the mbufs and into fixed-sized data buffers, the invention reduces the amount of time and system resources consumed by the storage system when writing blocks of received data to disk storage.

Abstract: A destination storage server, which may be a filer, mirrors a volume managed by a source storage server, which may also be a filer. According to an embodiment of the invention, changes made to the source volume are logged and persistently stored on a data container, such as a file, on the destination volume. The source storage server is coupled to clients that make data access requests to the volume. When an access request is made by a client, the request is written to a log on the source storage server. At the same time, the request is written to a data container on a volume managed by the destination storage server. Each source storage server coupled to the destination storage server has its own file on the volume.

Abstract: A system and method are provided to check consistency of an aggregate capable of supporting flexible volumes. The method includes identifying an inode having a flexible volume type present in the aggregate; determining whether the inode is identified in a metadata directory of the aggregate; and performing consistency check on the flexible volume associated with the inode.

Abstract: In one embodiment, the present invention provides a method for mirroring data representing a file system. The data is stored on a primary storage server and is mirrored on a secondary storage server. In the method the file system is mirrored without comparing blocks used to that represent the file system at the primary storage server, and the secondary storage server.

Abstract: A synchronous data mirroring technique includes a log forwarding process and a consistency point (CP) process. During log forwarding, a source storage server receives write requests from clients, creates a log entry for each request in nonvolatile memory, and transmits each log entry to a destination storage server at a mirror site. The destination storage server writes each log entry to a file. If a primary volume becomes inaccessible due to a failure, the file is used to produce an updated mirror volume. The CP process includes updating a primary volume based on the write requests, and updating the mirror volume to reflect the updated primary volume. The mirror volume is updated by transmitting CP data from the source storage server to the destination storage server, and using the CP data to update the mirror volume through a network administration layer and a storage layer in the destination storage server.

Abstract: A source storage server receives various write requests from a set of clients, logs each request in local memory, and forwards each log entry to a destination storage server at a secondary site. At a consistency point, the source storage server saves data, modified per the requests, to a first set of mass storage devices, and also initiates a synchronization phase during which process the modified data is mirrored at the secondary site. The destination storage server uses data in the received log entries to mirror at least a portion of the modified data in a second set of mass storage devices located at the secondary site, such that said portion of the modified data does not have to be sent from the source storage server to the destination storage server at the consistency point.

Abstract: A storage system, such as a file server, receives a request to perform a write operation that affects a data block. In response, the storage system writes to a storage device the data block together with context information which uniquely identifies the write operation with respect to the data block. When the data block is subsequently read from the storage device together with the context information, the context information that was read with the data block is used to determine whether a previous write of the data block was lost.