Abstract: In a data processing system, a first processor pre-allocates data blocks for use in a file system at a later time when a second processor needs data blocks for extending the file system. The second processor selectively maps the logical addresses of the pre-allocated blocks so that when the pre-allocated blocks are used in the file system, the layout of the file system on disk is improved to avoid block scatter and enhance I/O performance. The selected mapping can be done at a program layer between a conventional file system manager and a conventional logical volume layer so that there is no need to modify the data block mapping mechanism of the file system manager or the logical volume layer. The data blocks can be pre-allocated adaptively in accordance with the allocation history of the file system.

Abstract: Accessing data file objects includes providing a file system interface on a client, where the file system interface allows an application running on the client to make file system calls to access the data file objects. A Web Services interface may also be provided on the client, where the Web Services interface allows an application to access file objects using the Web Services at the same time that file objects are being accessed through the file system interface. The Web Services interface may be a SOAP interface and/or a REST interface. Accessing data file objects may also include providing a direct file object interface on the client, where the direct file object interface allows an application to directly access file objects at the same time that file objects are being accessed using the Web Services interface and using the file system interface.

Abstract: A method is used in managing metadata. Data of an object is stored in a magnetic hard disk set in an object addressable data storage system. Metadata for the object is stored in an SSD set in the object addressable data storage system. The metadata includes information necessary for determining the location of the data of the object in the magnetic hard disk set.

Abstract: Accessing data file objects includes providing a file system interface on a client, where the file system interface allows an application running on the client to make file system calls to access the data file objects. A Web Services interface may also be provided on the client, where the Web Services interface allows an application to access file objects using the Web Services at the same time that file objects are being accessed through the file system interface. The Web Services interface may be a SOAP interface and/or a REST interface. Accessing data file objects may also include providing a direct file object interface on the client, where the direct file object interface allows an application to directly access file objects at the same time that file objects are being accessed using the Web Services interface and using the file system interface.

Abstract: A read-write access to a file in a file server includes an initial metadata access step, an asynchronous read or write of file data, and a metadata commit step. In order to parallelize access to a range locking database, an inode and indirect file block mapping database, and a pre-allocation data base, operations are pipelined within the metadata access step and also within the metadata commit step. The pipelined operations within the metadata access step include a first stage of range locking, a second stage of reading block mapping metadata, and a third stage of reading and updating a pre-allocation map for write access. The pipelined operations within the metadata commit step include a first stage of releasing range locks, a second stage of accessing old block metadata for write access, and a third stage of updating metadata and flushing the pre-allocation map for write access.

Abstract: An improved policy enforcement architecture includes several components that improve the performance of policy enforcement in a hierarchical storage environment. A File System Query instruction permits a richer set of queries to be expressed by a policy engine, thereby permitting more complex policies to be implemented with ease. The File System result is generated at the file server, and only files of interest are forwarded to the policy engine. The file system query advantageously may be applied against a pre-generated index having one more characterizing attributes to further reduce the processing required to retrieve policy data. An Index Build instruction, added to a programming interface, allows a policy engine generate the characterizing indices in advance of use. Index maintenance techniques maintain consistency between the index and the file system to ensure that the policy can be enforced using accurate file information.

Abstract: An improved policy enforcement architecture includes several components that improve the performance of policy enforcement in a hierarchical storage environment. A File System Query instruction permits a richer set of queries to be expressed by a policy engine, thereby permitting more complex policies to be implemented with ease. The File System result is generated at the file server, and only files of interest are forwarded to the policy engine. The file system query advantageously may be applied against a pre-generated index having one more characterizing attributes to further reduce the processing required to retrieve policy data. An Index Build instruction, added to a programming interface, allows a policy engine generate the characterizing indices in advance of use. Index maintenance techniques maintain consistency between the index and the file system to ensure that the policy can be enforced using accurate file information.

Abstract: Methods, systems, and computer program products for providing access to shared storage by a plurality of nodes are disclosed. According to one method, at a node of a plurality of nodes sharing access to a disk array, an application input/output (I/O) operation and whether sufficient space in the disk array has been mapped at the node for the I/O operation are detected. In response to detecting that sufficient space in the disk array has not been mapped at the node for the I/O operation, a map for the I/O operation including physical disk access information is requested and obtained from a server. The physical disk access information included within the map for the application I/O operation obtained from the server is used to perform the I/O operation by accessing the disk array without intervention by the server.

Abstract: A primary processor manages metadata of a production dataset and a snapshot copy, while a secondary processor provides concurrent read-write access to the primary dataset. The secondary processor determines when a first write is being made to a data block of the production dataset, and in this case sends a metadata change request to the primary data processor. The primary data processor commits the metadata change to the production dataset and maintains the snapshot copy while the secondary data processor continues to service other read-write requests. The secondary processor logs metadata changes so that the secondary processor may return a “write completed” message before the primary processor commits the metadata change. The primary data processor pre-allocates data storage blocks in such a way that the “write anywhere” method does not result in a gradual degradation in I/O performance.

Abstract: A cached disk array includes a disk storage array, a global cache memory, disk directors coupling the cache memory to the disk storage array, and front-end directors for linking host computers to the cache memory. The front-end directors service storage access requests from the host computers, and the disk directors stage requested data from the disk storage array to the cache memory and write new data to the disk storage. At least one of the front-end directors or disk directors is programmed for block resolution of virtual logical units of the disk storage, and for obtaining, from a storage allocation server, space allocation and mapping information for pre-allocated blocks of the disk storage, and for returning to the storage allocation server requests to commit the pre-allocated blocks of storage once data is first written to the pre-allocated blocks of storage.

Abstract: A file server system has a cluster of server computers that share access to a file system in shared storage. One of the server computers has primary responsibility for management of access to the file system. In order to reduce the possibility of primary server overload when a large number of the clients happen to concurrently access the same file system, most metadata processing operations are offloaded to secondary server computers. This also facilitates recovery from failure of a primary server computer since only a fraction of the ongoing metadata operations of a primary server computer is interrupted by a failure of the primary server computer. For example, a secondary data mover may truncate, delete, create, or rename a file in response to a client request.

Abstract: Tracking storage resources includes providing a table containing storage resources along with capabilities and statuses thereof, updating the table in response to a change of status of a storage resource, updating the table in response to a change in capabilities of a storage resource and, in response to an inquiry for a storage resource having a particular capability, searching the table for a storage resource having the particular capability. Tracking storage resources may also include adding an element to the table in response to a new resource being added to the system. The capabilities may include RAID striping, data deduplication, and green operation. The status may be one of: on-line, off-line, and full.

Abstract: Managing a plurality of files includes providing at least one server having a table with plurality of file identifiers, each of the file identifiers including a pointer to a metadata file object containing a layout storage object that indicates storage parameters for corresponding files and includes accessing a particular one of the files by obtaining information from the at least one server to access the layout storage object corresponding to the particular file to determine synchronous and asynchronous mirrors for the file. Each layout storage object may be provided as a tree and leaves of the tree may map logical addresses of a corresponding file to data storage locations. At least one internal node of the tree may indicate that at least one leaf node thereof is a mirror of an other leaf node thereof.

Abstract: Managing files includes receiving a plurality of policies for a new file, where different policies may be applied according to predetermined conditions, receiving a request to create a new file, selecting an appropriate policy for the new file according to the predetermined conditions, determining if sufficient resources are available according to the appropriate policy, and obtaining file objects to create the file in response to their being sufficient resources according to the appropriate policy. Policies may be received via a user management interface. The predetermined conditions may include an identity of a client creating a file. The appropriate policy may include information regarding mirrors for the file. The appropriate policy may indicate a minimum geographic distance between the mirrors.

Abstract: Managing file objects in a data storage system includes providing a plurality of metadata servers, each metadata server having information for only a subset of file objects in the storage system, each subset for each of the metadata servers being less than a total number of file objects for the system and includes providing at least one metadata location server, where an entity accessing a specific file object of the system determines which of the metadata servers contains data for the specific file object by first accessing the at least one metadata location server. Information for the subset of file objects may include a table having file identifiers and metadata location information. The metadata location information may point to data storage for a metadata object for a file.

Abstract: Accessing data file objects includes providing a file system interface on a client, where the file system interface allows an application running on the client to make file system calls to access the data file objects. A Web Services interface may also be provided on the client, where the Web Services interface allows an application to access file objects using the Web Services at the same time that file objects are being accessed through the file system interface. The Web Services interface may be a SOAP interface and/or a REST interface. Accessing data file objects may also include providing a direct file object interface on the client, where the direct file object interface allows an application to directly access file objects at the same time that file objects are being accessed using the Web Services interface and using the file system interface.

Abstract: To permit multiple unsynchronized processors to update the file-modification time attribute of a file during concurrent asynchronous writes to the file, a primary processor having a clock manages access to metadata of the file. A number of secondary processors service client request for access to the file. Each secondary processor has a timer. When the primary processor grants a range lock upon the file to a secondary, it returns its clock time (m). Upon receipt, the secondary starts a local timer (t). When the secondary modifies the file data, it determines a file-modification time that is a function of the clock time and the timer interval, such as a sum (m+t). When the secondary receives an updated file-modification time (mp) from the primary, if mp>m+t, then the secondary updates the clock time (m) to (mp) and resets its local timer.

Abstract: A cached disk array includes a disk storage array, a global cache memory, disk directors coupling the cache memory to the disk storage array, and front-end directors for linking host computers to the cache memory. The front-end directors service storage access requests from the host computers, and the disk directors stage requested data from the disk storage array to the cache memory and write new data to the disk storage. At least one of the front-end directors or disk directors is programmed for block resolution of virtual logical units of the disk storage, and for obtaining, from a storage allocation server, space allocation and mapping information for pre-allocated blocks of the disk storage, and for returning to the storage allocation server requests to commit the pre-allocated blocks of storage once data is first written to the pre-allocated blocks of storage.

Abstract: A protocol is provided for allocating file locking tasks between primary and secondary data mover computers in a network file server. When there is frequent read access and infrequent write access to a file, a primary data mover grants read locks to the entire file to secondary data movers, and the secondary data movers grant read locks to clients requesting read access. When write access to the file is needed, the read locks to the entire file are released and the read locks granted to the clients are released or expire or are demoted to non-conflicting byte range locks managed by the primary data mover. Concurrent read and write access to the same file is then managed by the primary data mover.

Abstract: In a data processing system, a first processor pre-allocates data blocks for use in a file system at a later time when a second processor needs data blocks for extending the file system. The second processor selectively maps the logical addresses of the pre-allocated blocks so that when the pre-allocated blocks are used in the file system, the layout of the file system on disk is improved to avoid block scatter and enhance I/O performance. The selected mapping can be done at a program layer between a conventional file system manager and a conventional logical volume layer so that there is no need to modify the data block mapping mechanism of the file system manager or the logical volume layer. The data blocks can be pre-allocated adaptively in accordance with the allocation history of the file system.