Abstract: Described are techniques for performing automated discovery processing in a network. A data storage system monitors transmissions sent over the network. The data storage system extracts one or more data elements from one or more messages included in the transmissions and determines at least one application executing on a component in the network in accordance with the one or more data elements.

Abstract: Servers in a storage system store a nested multilayer directory structure, and a global index that is an abstract of the directory structure. The global index identifies respective portions of the directory structure that are stored in respective ones of the servers, and the global index identifies paths through the directory structure linking the respective portions. Upon performing a top-down search of the directory structure in response to a client request and finding that a portion of it is offline, the global index is searched to discover portions of the directory structure that are located below the offline portion. The global index may also identify the respective server storing each of the respective portions of the directory structure, and may indicate whether or not each of the respective portions of the directory structure is known to be offline.

Abstract: An improved policy enforcement architecture includes several components that improve the performance of policy enforcement in a hierarchical storage environment. A File System Query instruction permits a richer set of queries to be expressed by a policy engine, thereby permitting more complex policies to be implemented with ease. The File System result is generated at the file server, and only files of interest are forwarded to the policy engine. The file system query advantageously may be applied against a pre-generated index having one more characterizing attributes to further reduce the processing required to retrieve policy data. An Index Build instruction, added to a programming interface, allows a policy engine generate the characterizing indices in advance of use. Index maintenance techniques maintain consistency between the index and the file system to ensure that the policy can be enforced using accurate file information.

Abstract: A method is used in managing metadata. Data of an object is stored in a magnetic hard disk set in an object addressable data storage system. Metadata for the object is stored in an SSD set in the object addressable data storage system. The metadata includes information necessary for determining the location of the data of the object in the magnetic hard disk set.

Abstract: In a data processing system, a first processor pre-allocates data blocks for use in a file system at a later time when a second processor needs data blocks for extending the file system. The second processor selectively maps the logical addresses of the pre-allocated blocks so that when the pre-allocated blocks are used in the file system, the layout of the file system on disk is improved to avoid block scatter and enhance I/O performance. The selected mapping can be done at a program layer between a conventional file system manager and a conventional logical volume layer so that there is no need to modify the data block mapping mechanism of the file system manager or the logical volume layer. The data blocks can be pre-allocated adaptively in accordance with the allocation history of the file system.

Abstract: A method is used in managing metadata. Data of an object is stored in a magnetic hard disk set in an object addressable data storage system. Metadata for the object is stored in an SSD set in the object addressable data storage system. The metadata includes information necessary for determining the location of the data of the object in the magnetic hard disk set.

Abstract: A read-write access to a file in a file server includes an initial metadata access step, an asynchronous read or write of file data, and a metadata commit step. In order to parallelize access to a range locking database, an inode and indirect file block mapping database, and a pre-allocation data base, operations are pipelined within the metadata access step and also within the metadata commit step. The pipelined operations within the metadata access step include a first stage of range locking, a second stage of reading block mapping metadata, and a third stage of reading and updating a pre-allocation map for write access. The pipelined operations within the metadata commit step include a first stage of releasing range locks, a second stage of accessing old block metadata for write access, and a third stage of updating metadata and flushing the pre-allocation map for write access.

Abstract: An improved policy enforcement architecture includes several components that improve the performance of policy enforcement in a hierarchical storage environment. A File System Query instruction permits a richer set of queries to be expressed by a policy engine, thereby permitting more complex policies to be implemented with ease. The File System result is generated at the file server, and only files of interest are forwarded to the policy engine. The file system query advantageously may be applied against a pre-generated index having one more characterizing attributes to further reduce the processing required to retrieve policy data. An Index Build instruction, added to a programming interface, allows a policy engine generate the characterizing indices in advance of use. Index maintenance techniques maintain consistency between the index and the file system to ensure that the policy can be enforced using accurate file information.

Abstract: Described are techniques for managing data in a data storage system. While the data storage system is operating in a first mode, portions of cached write data not yet destaged to a data storage device are copied from the volatile memory cache to a first area in a flash-based memory. In response to a vault operation event, the data storage system operates in a second mode in which all remaining portions of cached write data from the volatile memory cache not currently included in the first area are copied to a second area of the flash-based memory.

Abstract: An improved policy enforcement architecture includes several components that improve the performance of policy enforcement in a hierarchical storage environment. A File System Query instruction permits a richer set of queries to be expressed by a policy engine, thereby permitting more complex policies to be implemented with ease. The File System result is generated at the file server, and only files of interest are forwarded to the policy engine. The file system query advantageously may be applied against a pre-generated index having one more characterizing attributes to further reduce the processing required to retrieve policy data. An Index Build instruction, added to a programming interface, allows a policy engine generate the characterizing indices in advance of use. Index maintenance techniques maintain consistency between the index and the file system to ensure that the policy can be enforced using accurate file information.

Abstract: Methods, systems, and computer program products for providing access to shared storage by a plurality of nodes are disclosed. According to one method, at a node of a plurality of nodes sharing access to a disk array, an application input/output (I/O) operation and whether sufficient space in the disk array has been mapped at the node for the I/O operation are detected. In response to detecting that sufficient space in the disk array has not been mapped at the node for the I/O operation, a map for the I/O operation including physical disk access information is requested and obtained from a server. The physical disk access information included within the map for the application I/O operation obtained from the server is used to perform the I/O operation by accessing the disk array without intervention by the server.

Abstract: A primary processor manages metadata of a production dataset and a snapshot copy, while a secondary processor provides concurrent read-write access to the primary dataset. The secondary processor determines when a first write is being made to a data block of the production dataset, and in this case sends a metadata change request to the primary data processor. The primary data processor commits the metadata change to the production dataset and maintains the snapshot copy while the secondary data processor continues to service other read-write requests. The secondary processor logs metadata changes so that the secondary processor may return a “write completed” message before the primary processor commits the metadata change. The primary data processor pre-allocates data storage blocks in such a way that the “write anywhere” method does not result in a gradual degradation in I/O performance.

Abstract: A cached disk array includes a disk storage array, a global cache memory, disk directors coupling the cache memory to the disk storage array, and front-end directors for linking host computers to the cache memory. The front-end directors service storage access requests from the host computers, and the disk directors stage requested data from the disk storage array to the cache memory and write new data to the disk storage. At least one of the front-end directors or disk directors is programmed for block resolution of virtual logical units of the disk storage, and for obtaining, from a storage allocation server, space allocation and mapping information for pre-allocated blocks of the disk storage, and for returning to the storage allocation server requests to commit the pre-allocated blocks of storage once data is first written to the pre-allocated blocks of storage.

Abstract: Described are techniques for processing a data storage configuration request for an application A first user level of a plurality of user levels at which said data storage configuration request is made is determined. One or more rule sets defining mappings between different ones of said plurality of user levels are provided. Each of the different user levels is associated with a different level of abstraction with respect to processing performed in the data storage system for implementing the data storage configuration request. One of the rule sets is selected in accordance with the application, a data service requested in said data storage configuration request, and a user grouping including said plurality of user levels. The data storage configuration request is serviced using rules of the selected rule set.

Abstract: Consistent updates are made automatically over a wide-area IP network, concurrently with read-only access to the remote copies. A replication control protocol (RCP) is layered over TCP/IP providing the capability for a remote site to replicate and rebroadcast blocks of the remote copy data to specified groups of destinations, as configured in a routing table. A volume multicast layer over RCP provides for multicasting to specified volume extents of the blocks. The blocks are copied at the logical level, so that it does not matter what physical structure is used for storing the remote copies. Save volumes buffer the remote copy data transmitted between the primary or secondary file system volume and the IP network, in order to ensure independence between the replication process, the IP transport method, and the primary file system being replicated.

Abstract: Described are techniques for processing a data storage configuration request for an application A first user level of a plurality of user levels at which said data storage configuration request is made is determined. One or more rule sets defining mappings between different ones of said plurality of user levels are provided. Each of the different user levels is associated with a different level of abstraction with respect to processing performed in the data storage system for implementing the data storage configuration request. One of the rule sets is selected in accordance with the application, a data service requested in said data storage configuration request, and a user grouping including said plurality of user levels. The data storage configuration request is serviced using rules of the selected rule set.

Abstract: A protocol is provided for allocating file locking tasks between primary and secondary data mover computers in a network file server. When there is frequent read access and infrequent write access to a file, a primary data mover grants read locks to the entire file to secondary data movers, and the secondary data movers grant read locks to clients requesting read access. When write access to the file is needed, the read locks to the entire file are released and the read locks granted to the clients are released or expire or are demoted to non-conflicting byte range locks managed by the primary data mover. Concurrent read and write access to the same file is then managed by the primary data mover.

Abstract: File system cells are linked together to form a meta file system that appears to a user or application program to be a single file system. The meta file system permits concurrent access by multiple processors in a file server wherein each file system cell is managed by a respective one of the processors. The file server responds to a directory access request by returning a file handle containing a file system cell ID and a pointer to a file in the file system cell. The file server responds to a subsequent file access request including the file handle by extracting the file system cell ID and the pointer to the file, searching a routing table for an entry having a file system cell ID matching the file system cell ID extracted from the file handle, and routing the request to the processor managing the file system cell.

Abstract: A file system is migrated from a source file server to a target file server in a data network while permitting clients to have concurrent read/write access to the file system. The target file server issues directory read requests and file read requests to the source file server in accordance with a network file access protocol to transfer the file system from the source file server to the target file server. Concurrent with the transfer of the file system from the source file server to the target file server, the target file server responds to client read/write requests for access to the file system. In a preferred embodiment, the target file server maintains a hierarchy of on-line nodes off-line nodes. The online nodes represent file system objects that have been completely migrated, and the offline nodes representing file system objects that have not been completely migrated.

Abstract: A data storage system receives sets of the revisions such that each set of revisions changes the dataset from one consistent state to another. Each set of revisions is processed in a write-selected phase followed by a read-selected phase. In the write-selected phase, the revisions in each set are written to a buffer and processed to produce a directory of the set of revisions. In the read-selected phase, the revisions are read from the buffer and integrated into the dataset. When one set of revisions is in the read-selected phase, the next set of revisions is in the write-selected phase.