Abstract: A high tier storage area stores a stub file and a lower tier cloud storage area stores the file corresponding to the stub file. When a client apparatus requests segments of the file from the high tier storage area, reference is made to the stub file to determine a predicted non-sequential pattern of requests to the segments by the client apparatus. The high tier storage area follows the predicted non-sequential pattern of requests to retrieve the segments of the file from the cloud prior to the client apparatus actually requesting the segments. As such, the file may be efficiently provided to the client apparatus while also efficiently storing the file on the lower tier cloud storage area.

Abstract: A NAS server runs a native filesystem application that manages the filesystem and also includes a virtualization component that allows third-party data processing applications to run on the NAS server substantially as though running on a separate server while allowing the applications to access filesystem information without use of the network and without the overhead of network protocols and related operating system processing. The virtualization component intercepts certain filesystem-related system calls made by the application and directs processing of those filesystem-related system calls (e.g., by the virtualization component or by the native filesystem application), bypassing certain operating system processes including those relating to network protocol processing.

Abstract: A hardware-based file system includes multiple linked sub-modules that perform functions ancillary to client data handling. Each sub-module is associated with a metadata cache. A doubly-rooted structure is used to store each file system object at successive checkpoints. Metadata is stored within an object and/or as a separate object. Provisions are made for allocating sparse objects. A delayed write feature is used for writing certain objects into non-volatile storage. Checkpoints can be retained as read-only versions of the file system. Modifications to the file system are accomplished without affecting the contents of retained checkpoints. The file system can be reverted to a retained checkpoint. Multiple file servers can be interconnected as a cluster, and each file server stores requests from another file server. Interconnections between file servers can be dynamically modified. A distributed locking mechanism is used to control access to file system objects stored by the file servers.

Abstract: A hardware-based file system includes multiple linked sub-modules that perform functions ancillary to client data handling. Each sub-module is associated with a metadata cache. A doubly-rooted structure is used to store each file system object at successive checkpoints. Metadata is stored within an object and/or as a separate object. Provisions are made for allocating sparse objects. A delayed write feature is used for writing certain objects into non-volatile storage. Checkpoints can be retained as read-only versions of the file system. Modifications to the file system are accomplished without affecting the contents of retained checkpoints. The file system can be reverted to a retained checkpoint. Multiple file servers can be interconnected as a cluster, and each file server stores requests from another file server. Interconnections between file servers can be dynamically modified. A distributed locking mechanism is used to control access to file system objects stored by the file servers.

Abstract: A file cloning mechanism allows for quickly creating copies (clones) of files within a filesystem, such as when a user makes a copy of a file. In exemplary embodiments, a clone of a source object is at least initially represented by a structure containing references to various elements of the source object (e.g., indirect onodes, direct onodes, and data blocks). Both read-only and mutable clones can be created. The source file and the clone initially share such elements and continue to share unmodified elements as changes are made to the source file or mutable clone. None of the user data blocks or the metadata blocks describing the data stream (i.e., the indirect/direct onodes) associated with the source file need to be copied at the time the clone is created. At appropriate times, cloned files may be “de-cloned.

Abstract: An apparatus in one embodiment handles service requests over a network, wherein the network utilizes a protocol. In this aspect, the apparatus includes: a network subsystem for receiving and transmitting network service requests using the network protocol; and a service subsystem, coupled to the network subsystem, for satisfying the network service requests. At least one of the network subsystem and the service subsystem is hardware-implemented; the other of the network subsystem and the service subsystem may optionally be hardware-accelerated. A variety of related embodiments are also provided, including file servers and web servers.

Abstract: A distributed file system and method distributes file system objects across multiple self-contained volumes, where each volume is owned by a unique file system node. Logical links are used to reference a file system object between volumes. Each file system node includes a relocation directory in which is maintained hard links to locally-stored file system objects that are referenced from another file system node using logical links. Various file system operations that involve multiple volumes are performed without having to place a write lock on more than one volume at a time. Various caching schemes allow the various file system nodes to cache file system object data and metadata.

Abstract: Validating a data structure includes (a) maintaining a tracking structure in a memory, the tracking structure including a corresponding status field for each table entry, the status fields having an unmarked state and a marked state; (b) processing the table entries sequentially and tracking the used and free table entries using the tracking structure, such tracking including marking the status fields corresponding to used table entries and marking the status fields corresponding to the next free table entries referenced in the pointer fields of free table entries; and (c) determining validity of the data structure based on the tracking structure.

Abstract: A distributed file system and method distributes file system objects across multiple self-contained volumes, where each volume is owned by a unique file system node. Logical links are used to reference a file system object between volumes. Each file system node includes a relocation directory in which is maintained hard links to locally-stored file system objects that are referenced from another file system node using logical links. Various file system operations that involve multiple volumes are performed without having to place a write lock on more than one volume at a time. Various caching schemes allow the various file system nodes to cache file system object data and metadata.

Abstract: A file system adapter card that may be plugged into a host computer system for providing hardware-based file system accesses outside the purview of a host operating system running on the host computer system. The file system adapter card includes a hardware-implemented or hardware-accelerated file service subsystem and a computer bus that permits a host computer system to communicate directly with the file service subsystem for providing file service requests and receiving file service responses. The file service subsystem includes dedicated hardware that operates outside the immediate control of a host operating system, including specialized circuitry for performing at least one major subsystem function.

Abstract: Mirror relationships between system drives may be detected and/or configured automatically by writing a predetermined data pattern to a predetermined location within a system drive and then checking one or more other system drives for a mirrored copy of the same data pattern. If the data pattern is found in another SD, then it can be assumed that there is a mirror relationship between the two SDs. Failure to find the data pattern in another SD, or finding the data pattern in multiple other SDs, may indicate an error condition. Once mirror relationships are discovered, the mirror relationships can be configured into the file server automatically so that the engineer does not have to manually configure the mirror relationships. The same mechanism can be used to check mirror relationships from time to time, for example, to detect possible problems with the RAID system.

Abstract: Embodiments of the present invention provide for compression of files in a filesystem. In embodiments of the present invention, a file is logically partitioned into chunks and each chunk is compressed, such that a request involving a portion of a file can be satisfied by reading and decompressing only the compressed chunk(s) corresponding to that portion.

Abstract: Embodiments of the present invention provide for compression of files in a filesystem. In embodiments of the present invention, a file is logically partitioned into chunks and each chunk is compressed, such that a request involving a portion of a file can be satisfied by reading and decompressing only the compressed chunk(s) corresponding to that portion.

Abstract: Available storage locations are identified by searching through a free blocks object non-sequentially. The free blocks object may be logically divided into sections, and the sections may be searched non-sequentially. Each section may be logically divided into a number of sub-sections, and the sub-sections of the various sections may be searched non-sequentially. A new section may be added to the free blocks object, and the added section may be searched until a predetermined threshold or condition is reached, at which point non-sequential searching may begin or resume.

Abstract: Embodiments of the present invention allow multiple checkpoints to be taken so that multiple versions of the filesystem, including a working version and at least two checkpoint versions, can be maintained over time. Specifically, at least three “superblock” root structures are used to manage multiple instantiations of the filesystem. The superblocks are preferably stored in fixed locations within the storage system for easy access, although they may alternatively be stored in other ways. The number of superblocks may be fixed or variable.

Abstract: Objects stored in a storage system (such as a file server system) are protected by multiple levels of validation. Each chunk of an object is associated with a chunk validator, and an object validator is computed for the object based on the chunk validators. The object validator is stored in the storage system and may be used at various times to validate the object, for example, upon a startup of the storage system, upon taking a checkpoint or “snapshot” of the status of the storage system, or at other appropriate times.

Abstract: Validating a data structure includes (a) maintaining a tracking structure in a memory, the tracking structure including a corresponding status field for each table entry, the status fields having an unmarked state and a marked state; (b) processing the table entries sequentially and tracking the used and free table entries using the tracking structure, such tracking including marking the status fields corresponding to used table entries and marking the status fields corresponding to the next free table entries referenced in the pointer fields of free table entries; and (c) determining validity of the data structure based on the tracking structure.

Abstract: Checksums for internet protocol transmissions of certain transport layer messages are generated using IPv6 pseudo-headers for both IPv4 and IPv6 internet protocol transmissions. For IPv4 transmissions, IPv4 information is mapped into an IPv6 pseudo-header, and the checksum is computed using the IPv6 pseudo-header. Among other things, embodiments of the present invention allow checksums for both IPv4 and IPv6 internet protocol transmissions such transport layer messages to be computed using a common checksum computer.

Abstract: Mirror relationships between system drives may be detected and/or configured automatically by writing a predetermined data pattern to a predetermined location within a system drive and then checking one or more other system drives for a mirrored copy of the same data pattern. If the data pattern is found in another SD, then it can be assumed that there is a mirror relationship between the two SDs. Failure to find the data pattern in another SD, or finding the data pattern in multiple other SDs, may indicate an error condition. Once mirror relationships are discovered, the mirror relationships can be configured into the file server automatically so that the engineer does not have to manually configure the mirror relationships. The same mechanism can be used to check mirror relationships from time to time, for example, to detect possible problems with the RAID system.

Abstract: A network-attached system, device, and method supports multiple storage tiers. Data may be migrated between storage tiers, for example, based on a data migration policy. A plurality of dedicated storage devices that provide a plurality of storage tiers having different characteristics are managed using a disk control protocol and data is migrated between storage tiers based on predetermined storage criteria.