Abstract: Embodiments relate to off-loading aspects of storage virtualization to storage hardware and modifying software to take advantage of hardware virtualization features. A co-design of hardware and software allows a filesystem to provide files such that indirection overhead normally needed to access the content of files can be bypassed while still managing the files as filesystem objects. A storage device manages and exposes a virtual volume which is used to store the content of a file. Virtual volumes can be initialized or populated so that virtual blocks therein align with device storage blocks. A virtual volume can be initialized and populated by parsing a virtual disk file to access virtual disk metadata, which is then used to determine and set features of the virtual volume.

Abstract: An invention is disclosed for maintaining out-of-band metadata for data. In embodiments of the invention, an upper layer of a storage stack determines that the metadata of a lower layer of that storage stack may have become out of sync. In response, the upper layer may issue a series of commands to the lower layer based on the metadata. In other embodiments of the invention, an offload-copy operation on data may also transfer out-of-band metadata so that it is applied to the data at the destination.

Abstract: Embodiments relate to off-loading aspects of storage virtualization to storage hardware and modifying software to take advantage of hardware virtualization features. A co-design of hardware and software allows a filesystem to provide files such that indirection overhead normally needed to access the content of files can be bypassed while still managing the files as filesystem objects. A storage device manages and exposes a virtual volume which is used to store the content of a file. Virtual volumes can be initialized or populated so that virtual blocks therein align with device storage blocks. A virtual volume can be initialized and populated by parsing a virtual disk file to access virtual disk metadata, which is then used to determine and set features of the virtual volume.

Abstract: A thinly provisioned storage system detects whether physical storage capacity is available when there is a request to allocate storage capacity, prior to data being written to the storage system. In particular, at the time when the file system allocates storage, such as when creating a file or performing an extending write (append) operation, allocating storage to an unallocated region of a sparse file, defragmenting a file, and the like, a storage system can verify that actual physical storage capacity is available. Thus, if there is insufficient actual physical capacity at the time when a storage allocation is attempted, then an error message can be sent and remedial action can be taken.

Abstract: A thinly provisioned storage system detects whether physical storage capacity is available when there is a request to allocate storage capacity, prior to data being written to the storage system. In particular, at the time when the file system allocates storage, such as when creating a file or performing an extending write (append) operation, allocating storage to an unallocated region of a sparse file, defragmenting a file, and the like, a storage system can verify that actual physical storage capacity is available. Thus, if there is insufficient actual physical capacity at the time when a storage allocation is attempted, then an error message can be sent and remedial action can be taken.

Abstract: A thinly provisioned storage system detects whether physical storage capacity is available when there is a request to allocate storage capacity, prior to data being written to the storage system. In particular, at the time when the file system allocates storage, such as when creating a file or performing an extending write (append) operation, allocating storage to an unallocated region of a sparse file, defragmenting a file, and the like, a storage system can verify that actual physical storage capacity is available. Thus, if there is insufficient actual physical capacity at the time when a storage allocation is attempted, then an error message can be sent and remedial action can be taken.

Abstract: A thinly provisioned storage system detects whether physical storage capacity is available when there is a request to allocate storage capacity, prior to data being written to the storage system. In particular, at the time when the file system allocates storage, such as when creating a file or performing an extending write (append) operation, allocating storage to an unallocated region of a sparse file, defragmenting a file, and the like, a storage system can verify that actual physical storage capacity is available. Thus, if there is insufficient actual physical capacity at the time when a storage allocation is attempted, then an error message can be sent and remedial action can be taken.

Abstract: An invention is disclosed for maintaining out-of-band metadata for data. In embodiments of the invention, an upper layer of a storage stack determines that the metadata of a lower layer of that storage stack may have become out of sync. In response, the upper layer may issue a series of commands to the lower layer based on the metadata. In other embodiments of the invention, an offload-copy operation on data may also transfer out-of-band metadata so that it is applied to the data at the destination.

Abstract: An invention is disclosed for maintaining out-of-band metadata for data. In embodiments of the invention, an upper layer of a storage stack determines that the metadata of a lower layer of that storage stack may have become out of sync. In response, the upper layer may issue a series of commands to the lower layer based on the metadata. In other embodiments of the invention, an offload-copy operation on data may also transfer out-of-band metadata so that it is applied to the data at the destination.

Abstract: In one embodiment, a host application may manage a data set maintained at a storage device using a token. A processor 220 of a host computer executing a host application may obtain a token representing a data set. The processor 220 may read a data set result based on the data set into a memory local to the host application. The data set result may be a data set copy, a data set digest, or a data set transformation.

Abstract: A model in which filter drivers are managed to receive callbacks for I/O requests in which the filter drivers have registered an interest. Per-volume instances of filter drivers register with a filter manager for pre-callbacks (for I/O to the file system) and post-callbacks (for I/O from the file system), and identify which I/O requests (e.g., create, read, write) they are registering to receive callbacks. The filter manager orders the instances for callbacks. When an I/O request is received, the filter manager converts the I/O request to callback data and calls the interested filters in the callback order, whereby the filter instances can process the I/O data. As the request returns from the file system, filters desiring post callbacks are called in the reverse order. Efficient context management for the filters and other functions, such as non-reentrant file I/O, are also provided by the model.

Abstract: Systems and methods that create a virtual name space of Universal Naming Convention (UNC) over a local file system (e.g., drive letter based names), via employing a UNC provider and filter arrangement that changes a name in the file object to reparse it to a volume where the local file exists. A name in the file object is changed to a local name (e.g., morphed name); subsequently reparsed and forwarded to a Name Space Owner component (NSO). As such, a request can originate from a UNC stack and reparsed on the local stack. The name space is presented to an application in UNC format, while the underlying file write occur in the file system.

Abstract: Aspects of the subject matter described herein relate to antivirus protection and transactions. In aspects, a filter detects that a file is participating in a transaction and then may cause the file to be scanned together with any changes that have made to the file during the transaction. After a file is scanned, a cache entry may be updated to indicate that the file is clean. The cache entry may be used subsequently for like-type states. For example, if the file was scanned inside a transaction, the cache entry may be used later in the transaction. If the file was scanned outside a transaction, the cache entry may be used later for requests pertaining to files not in a transaction. Cache entries may be discarded when they are invalid or no longer useful.

Abstract: Aspects of the subject matter described herein relate to file system filters and transactions. In aspects, a filter may enlist to receive notification of events associated with a transaction. Afterwards, the filter may receive notification of a transaction event for which it has enlisted. In response to receiving notification of the transaction the filter may perform an action as appropriate. Aspects of the subject matter described herein may be applied to stacked and managed filters.

Abstract: File system metadata regarding states of a file system affected by transactions is tracked consistently even in the face of dirty shutdowns which might cause rollbacks in transactions which have already been reflected in the metadata. In order to only request time- and resource-heavy rebuilding of metadata for metadata which may have been affected by rollbacks, reliability information is tracked regarding metadata items. When a metadata item is affected by a transaction which may not complete properly in the case of a problematic shutdown or other event, that metadata item's reliability information indicates that it may not be reliable in case of such a problematic (“dirty” or“abnormal”) event. In addition to flag information indicating unreliability, timestamp information tracking a time of the command which has made a metadata item unreliable is also maintained.

Abstract: A method and system for ordering software modules in a guaranteed order for execution. Unique values are statically assigned to software modules (e.g., filter drivers) when fully developed. Each module's assigned value determines its relative position to other modules in a stack or other arrangement, fixing the execution order for any set of filter drivers. Static values may comprise floating-point numbers, whereby each new software module may be assigned a number that enables positioning it between any two existing software modules. For example, filter drivers may be generally classified and assigned values in a range according to type. Drivers of the same type may be ordered within their general range to guarantee one possible ordering. A filter manager architecture is described, in which filter drivers register with a manager for relevant file system I/O operations. The manager calls appropriately registered filter drivers in an order based on their assigned numbers.

Abstract: Object rundown protection that scales with the number of processors in a shared-memory computer system is disclosed. Prior to object rundown, a cache-aware reference count data structure is used to prevent cache-pinging that would otherwise result from data sharing across processors in a multiprocessor computer system. In this data structure, a counter of positive references and negative dereferences, aligned on a particular cache line, is maintained for each processor. When an object is to be destroyed, a rundown wait process is begun, during which new references on the object are prohibited, and the total number of outstanding references is added to an on-stack global counter. Destruction is delayed until the global reference count is reduced to zero. In an implementation on non-uniform memory access multiprocessor machines, each processor's reference count is additionally allocated in a region of main memory that is physically close to that processor.

Abstract: A system and method for object rundown protection that scales with the number of processors in a shared-memory computer system is disclosed. In an embodiment of the present invention, prior to object rundown, a cache-aware reference count data structure is used to prevent cache-pinging that would otherwise result from data sharing across processors in a multiprocessor computer system. In this data structure, a counter of positive references and negative dereferences, aligned on a particular cache line, is maintained for each processor. When an object is to be destroyed, a rundown wait process is begun, during which new references on the object are prohibited, and the total number of outstanding references is added to an on-stack global counter. Destruction is delayed until the global reference count is reduced to zero.

Abstract: A model in which filter drivers are managed to receive callbacks for I/O requests in which the filter drivers have registered an interest. Per-volume instances of filter drivers register with a filter manager for pre-callbacks (for I/O to the file system) and post-callbacks (for I/O from the file system), and identify which I/O requests (e.g., create, read, write) they are registering to receive callbacks. The filter manager orders the instances for callbacks. When an I/O request is received, the filter manager converts the I/O request to callback data and calls the interested filters in the callback order, whereby the filter instances can process the I/O data. As the request returns from the file system, filters desiring post callbacks are called in the reverse order. Efficient context management for the filters and other functions, such as non-reentrant file I/O, are also provided by the model.

Abstract: A method and system for caching remote objects locally. A request to access an object is received. A determination is made as to whether the object is cached. If the object is cached and the request is not to create a new object, modify an existing object, or open a directory, the request is directed to a local file system. Otherwise, the request is directed to a remote file system. Information about which objects are requested may be used to copy objects stored remotely to a local server as specified by a caching policy.