Abstract: A file system that utilizes Virtual Local Area Network (VLAN) addressing and routing schemes to permit implementing multiple networked file system applications in a single data processing system. In particular, a networked file server or data mover has associated with it a mass storage device such as one or more disk drives, as well as one or more Network Interface Cards (NICs). A network interface to the data mover is associated with VLAN identifiers (IDs) such that a VLAN ID can be assigned to each file system. At the physical layer of the data mover, VLAN information is encapsulated and passed up to various higher protocol layers, such as a network layer and/or a transport layer, eventually to a selected file system as dictated by the VLAN ID. The file system application in effect becomes an addressable element of the VLAN. A single data mover can thus be deployed to service multiple groups of end users, as long as the members of each group have different VLAN identifiers for the multiple file systems.

Abstract: A network file server includes a first set of data processors for receiving requests from clients, and a second set of data processors for accessing read-write file systems. A respective data processor in the second set is assigned to each file system for exclusive management of read and write locks on the file system. Each data processor in the first set can authorize access to file systems directly accessed by more than one data processor in the second set. Processing of a request for access that is authorized is continued by the data processor that is assigned to manage the locks on the file system to be accessed. The exclusivity of lock management eliminates cache coherency problems, and dynamic load balancing can be used to prevent the lock management from becoming a bottleneck to performance. A preferred embodiment includes a cached disk storage subsystem linked to data mover computers.

Abstract: A cached disk array includes a disk storage array, a global cache memory, disk directors coupling the cache memory to the disk storage array, and front-end directors for linking host computers to the cache memory. The front-end directors service storage access requests from the host computers, and the disk directors stage requested data from the disk storage array to the cache memory and write new data to the disk storage. At least one of the front-end directors or disk directors is programmed for block resolution of virtual logical units of the disk storage, and for obtaining, from a storage allocation server, space allocation and mapping information for pre-allocated blocks of the disk storage, and for returning to the storage allocation server requests to commit the pre-allocated blocks of storage once data is first written to the pre-allocated blocks of storage.

Abstract: A primary processor manages metadata of a production dataset and a snapshot copy, while a secondary processor provides concurrent read-write access to the primary dataset. The secondary processor determines when a first write is being made to a data block of the production dataset, and in this case sends a metadata change request to the primary data processor. The primary data processor commits the metadata change to the production dataset and maintains the snapshot copy while the secondary data processor continues to service other read-write requests. The secondary processor logs metadata changes so that the secondary processor may return a “write completed” message before the primary processor commits the metadata change. The primary data processor pre-allocates data storage blocks in such a way that the “write anywhere” method does not result in a gradual degradation in I/O performance.

Abstract: In a data processing system, a first processor pre-allocates data blocks for use in a file system at a later time when a second processor needs data blocks for extending the file system. The second processor selectively maps the logical addresses of the pre-allocated blocks so that when the pre-allocated blocks are used in the file system, the layout of the file system on disk is improved to avoid block scatter and enhance I/O performance. The selected mapping can be done at a program layer between a conventional file system manager and a conventional logical volume layer so that there is no need to modify the data block mapping mechanism of the file system manager or the logical volume layer. The data blocks can be pre-allocated adaptively in accordance with the allocation history of the file system.

Abstract: Accordingly a method and interface allows an attribute data base used by an Information Manager to be quickly populated and accurately maintained. A single Bulk Attribute Retrieval Request triggers the primary storage device to collect object attribute information. The method allows for selective collection of objects and attributes by providing filters and attribute lists in the Requests. The Request may be used to provide an incremental scan with appropriate time stamp filtering. In addition, the size of the results can be controlled by the IM by eliminating attributes that are not of interest to the IM. The Request is advantageously issued over a FileMover interface, which is an HTTP connection, and encoded in XML, allowing the IM to easily customize the Request as desired.

Abstract: An intelligent network client has the capability of accessing a first network server in accordance with a first high-level file access protocol, and responding to a redirection reply from the first network server by accessing a second network server in accordance with a second high-level file access protocol. For example, the intelligent network client can be redirected from a CIFS/DFS server to a NFS server, and from an NFSv4 server to a CIFS server. Once redirected, the intelligent network client performs a directory mounting operation so that a subsequent client access to the same directory goes directly to the second network server. For example, the first network server is a namespace server for translating pathnames in a client-server network namespace into pathnames in a NAS network namespace, and the second network server is a file server in the NAS network namespace.

Abstract: A namespace server translates client requests for access to files referenced by pathnames in a client-server namespace into requests for access to files referenced by pathnames in a NAS network namespace. The namespace server also translates between different file access protocols. If a client supports redirection and is requesting access to a file in a file server that supports the client's redirection, then the namespace server may redirect the client to the NAS network pathname of the file. Otherwise, the namespace server forwards a translated client request to the file server, and returns a reply from the file server to the client. A file server may redirect a redirection-capable client's access back to the namespace server for access to a share, directory, or file that is offline for migration, or for a deletion or name change that would require a change in translation information in the namespace server.

Abstract: For each high-level protocol, a respective mesh of Transmission Control Protocol (TCP) connections is set up for a cluster of server computers for the forwarding of client requests. Each mesh has a respective pair of TCP connections in opposite directions between each pair of server computers in the cluster. The high-level protocols, for example, include the Network File System (NFS) protocol, and the Common Internet File System (CIFS) protocol. Each mesh can be shared among multiple clients because there is no need for maintenance of separate TCP connection state for each client. The server computers may use Remote Procedure Call (RPC) semantics for the forwarding of the client requests, and prior to the forwarding of a client request, a new unique transaction ID can substituted for an original transaction ID in the client request so that forwarded requests have unique transaction IDs.

Abstract: Servers in a storage system store a nested multilayer directory structure, and a global index that is an abstract of the directory structure. The global index identifies respective portions of the directory structure that are stored in respective ones of the servers, and the global index identifies paths through the directory structure linking the respective portions. Upon performing a top-down search of the directory structure in response to a client request and finding that a portion of it is offline, the global index is searched to discover portions of the directory structure that are located below the offline portion. The global index may also identify the respective server storing each of the respective portions of the directory structure, and may indicate whether or not each of the respective portions of the directory structure is known to be offline.

Abstract: A network file server has storage for storing a file system, and a computer programmed for access to the file system in accordance with a file access protocol and in accordance with a storage access protocol. The computer receives a file access request from a network client for access to a file in the file system in accordance with the network file access protocol. The computer decides whether it should finish the file access using the file access protocol or the client should finish the file access using the storage access protocol. Upon deciding that the client should finish the file access using the storage access protocol, the computer returns to the client metadata of the file including metadata specifying addresses of logical blocks of storage allocated to the file, and then the computer responds to storage access requests from the client by performing read-write access to the file.

Abstract: A protocol is provided for allocating file locking tasks between primary and secondary data mover computers in a network file server. When there is frequent read access and infrequent write access to a file, a primary data mover grants read locks to the entire file to secondary data movers, and the secondary data movers grant read locks to clients requesting read access. When write access to the file is needed, the read locks to the entire file are released and the read locks granted to the clients are released or expire or are demoted to non-conflicting byte range locks managed by the primary data mover. Concurrent read and write access to the same file is then managed by the primary data mover.

Abstract: A network file server includes a first set of data processors for receiving requests from clients, and a second set of data processors for accessing read-write file systems. A respective data processor in the second set is assigned to each file system for exclusive management of read and write locks on the file system. Each data processor in the first set can authorize access to file systems directly accessed by more than one data processor in the second set. Processing of a request for access that is assigned to manage the locks on the file system to be accessed. The exclusivity of lock management eliminates cache coherency problems, and dynamic load balancing can be used to prevent the lock management from becoming a bottleneck to performance.

Abstract: A file server includes a plurality of stream server computers linking data storage to a data network, and at least two controller servers for controlling the stream server computers. The controller servers are programmed so that at a given time one of the controller servers is active in controlling the stream server computers, and another of the controller servers is inactive. The inactive controller server is programmed to respond automatically to a failure of the active controller server by becoming active. For example, each of the controller servers has a respective flag for indicating whether or not the controller server is active. Each controller server is programmed so that, upon booting, it will read the flag of the other stream server, and if the flag of the other controller server indicates that the other controller server is active, then the controller server becomes inactive. Otherwise, the stream server assumes an active or inactive state based on a predetermined arbitration method.

Abstract: To reorganize a striped file system, data blocks are sequentially moved from storage locations in original data storage to storage locations in an array including the original data storage and additional data storage. If the new striping has the same precedence order as the original striping, then temporary storage is not needed for the block movement. Otherwise, stripes of the data blocks are sequentially moved to a stripe of temporary storage locations, and moved from the stripe of the temporary locations to storage locations in the array. Once a pivot point is reached, however, there is no need to use the temporary storage. Moreover, there is an increasing separation between the read and write pointers. Once this separation exceeds the stripe length, the file system metadata can be synchronized at decreasing intervals and still permit concurrent read access.

Abstract: A first data mover computer services data access requests from a network client, and a second data mover computer is coupled to the first data mover computer for servicing data access requests from the first data mover computer. The first data mover computer uses a connection-oriented protocol to obtain client context information and to respond to a session setup request from the client by authenticating the client. Then the first data mover computer responds to a file system connection request from the client by forwarding the client context information and the file system connection request to the second data mover computer. Then the first data mover computer maintains a connection between the first data mover computer and the second data mover computer when the client accesses the file system and the first data mover computer passes file access requests from the client to the second data mover computer and returns responses to the file access requests from the second data mover computer to the client.

Abstract: A plurality of data mover computers control access to respective file systems in data storage. A network client serviced by any of the data movers can access each of the file systems. If a data mover receives a client request for access to a file in a file system to which access is controlled by another data mover, then the data mover that received the client request sends a metadata request to the data mover that controls access to the file system. The data mover that controls access to the file system responds by placing a lock on the file and returning metadata of the file. The data mover that received the client request uses the metadata to formulate a data access command that is used to access the file data in the file system over a bypass data path that bypasses the data mover computer that controls access to the file system.

Abstract: There is a performance loss associated with servicing a pipe or stream for a connection oriented process by maintaining a connection between a server thread and a client for a series of messages. As a result of maintaining this connection, there is less balance; some threads work harder than others, causing a loss of performance. To solve this problem, a collector queue combines messages from the connection oriented process with messages from the other concurrent processes. The threads receive messages from the collector queue rather than individual pipes. Any idle thread can pick up a message from the collector queue. The collector queue keeps track of which pipe each message came from so that the reply of the server to each message is directed to the same pipe from which the message came from. Therefore the collector queue ensures thread balance and efficiency in servicing the messages. In the preferred implementation, each entry in the collector queue includes a message pointer and a pipe pointer.

Abstract: A network file server includes a first set of data processors for receiving requests from clients, and a second set of data processors for accessing read-write file systems. A respective data processor in the second set is assigned to each file system for exclusive management of locks on the file system. The file server can detect failure of a failed data processor and automatically recover from the failure. When a failure of a data processor in the first set is detected, a spare data processor is programmed with the logical and physical network addresses of the failed data processor so that the spare data processor assumes the network identity of the failed data processor. When a failure of a data processor in the second set is detected, responsibility for management of the locks on each file system managed by the failed data processor is transferred to an operational data processor.

Abstract: A network file server includes a first set of data processors for receiving requests from clients, and a second set of data processors for accessing read-write file systems. A respective data processor in the second set is assigned to each file system for exclusive management of read and write locks on the file system. Each data processor in the first set can authorize access to file systems directly accessed by more than one data processor in the second set. Processing of a request for access that is authorized is continued by the data processor that is assigned to manage the locks on the file system to be accessed. The exclusivity of lock management eliminates cache coherency problems, and dynamic load balancing can be used to prevent the lock management from becoming a bottleneck to performance. A preferred embodiment includes a cached disk storage subsystem linked to data mover computers.