Abstract: Various methods and systems for securing access to hash-based storage systems are disclosed. One method involves receiving information to be stored in a storage system from a storage system client and then generating a key. The key identifies the information to be stored. The value of the key is dependent upon a secret value, which is associated with the storage system. The key is generated, at least in part, by applying a hash algorithm to the information to be stored. The key can then be returned the key to the storage system client. The storage system client can then use the key to retrieve the stored information.

Abstract: Various methods and systems for updating a search engine index based on a file change log are disclosed. One method involves accessing a file, which stores a file change log maintained by a file system. The file change log identifies one or more recently modified files. The method also involves updating an index, based on a recently-modified file identified in the file change log. The index is used to search files maintained by the file system. An application external to the file system accesses the file storing the file change log and updates the index.

Abstract: Systems and methods for repairing a failed component within a data storage subsystem without interruption of service. A client device is coupled to a data storage cluster comprising a plurality of data storage locations addressable as rows and columns in an array. Each column of the array comprising a separate computer interconnected to other columns via a network. After both detecting a failure of a first column in the array and detecting a predetermined time threshold has elapsed, a predetermined reconstruction column coordinates reconstructing data storage locations within the reconstruction column. For a second row in the array, a predetermined coordinating column coordinates the execution of a received memory access requests from the client corresponding to the second row. In this manner, a column coming back online after a failure is able to repair without halting the servicing of read and write requests.

Abstract: A method and mechanism for managing a time-indexed block map in a file system configured to store multiple versions of file data. A file system includes inodes or other structures corresponding to files in the system. Responsive to creating a file, one or more blocks are allocated for use in storing the file data. A block map is maintained for the file and is updated to include an identification of the blocks storing the file data and a timestamp corresponding to the stored data. In response to creating a file, data corresponding block map is maintained. In addition, a new entry is created in the block map which identifies the newly allocated blocks. Further, an entry of the block map is updated to (i) identify the particular blocks to which the write was directed and (ii) include a timestamp corresponding to the time of the write.

Abstract: A system and method for data storage in an array. A system includes a client coupled to a storage subsystem including data storage locations addressable as rows and columns in an array. Each column comprises a separate storage device. Each row includes redundant data. For a given row, a coordinating storage device coordinates the computation and storage of redundant data. The system detects a failure of a storage region and in response, configures an overlay storage device to temporarily overlay the failed region, maintains an association between the overlay device and the failed region, and maintains a record of changes made to the overlay device while the region is in a failed state. In response to detecting that the failed region has been restored, the system uses the association to identify the overlay device and uses the record of changes made to the overlay device to resynchronize the failed region.

Abstract: A method, system, and computer program product are provided to update file change log data to indicate that a change to a file in a file system has occurred. The file change log data are accessible for read access using a file system-independent operating system function. Applications can read, open, close, and seek data in the file change log using file system-independent operating system functions. The file change log data can be read by multiple applications simultaneously, even when data are being written to the file change log. Operations such as writing data to the file change log and renaming or deleting the file change log file itself are prevented when performed by an application rather than the file system. These prohibitions protect the file change log.

Abstract: A system and method for data storage in an array. A system includes a client coupled to a storage subsystem. The storage subsystem comprises data storage locations addressable as rows and columns in an array. Each column comprises a separate storage device. Each row includes redundant data. For a given row, a coordinating storage device receives data from the client, coordinates computation and storage of redundant data, and forwards data to other storage devices. In response to receiving data targeted for storage in a given storage location, a non-volatile, temporary storage device that is associated with the separate storage device that includes the given storage location buffers the received data. The coordinating storage device conveys a write completion message to the client in response to detecting that the data has been buffered in the non-volatile, temporary storage devices. At least two storage devices are coordinating storage devices in separate rows.

Abstract: Information identifying the inode of the parent directory of a file may be stored in that file's inode. A reverse pathname lookup from the file's inode identifier may be performed by reading a parent directory inode identifier of the file's parent directory from the file's inode and using the parent directory inode identifier to generate a pathname for the file. Generating the pathname may involve identifying the filename of the file by searching the parent directory identified by the parent inode identifier for the file's filename. A file's inode may include more than one parent directory inode identifier.

Abstract: A system and method for data storage in an array. A system includes a client coupled to a storage subsystem including data storage locations addressable as rows and columns in an array. Each column comprises a separate storage device. Each row includes redundant data. For a given row, a coordinating storage device coordinates the computation and storage of redundant data. The system detects a failure of a storage region and in response, configures an overlay storage device to temporarily overlay the failed region, maintains an association between the overlay device and the failed region, and maintains a record of changes made to the overlay device while the region is in a failed state. In response to detecting that the failed region has been restored, the system uses the association to identify the overlay device and uses the record of changes made to the overlay device to resynchronize the failed region.

Abstract: A system and method for efficient file content searching within a file system. In one embodiment, the system may include a storage device configured to store data and a file system configured to manage access to the storage device. The file system may be configured to detect a search operation to determine the presence of a given data pattern within a first file stored on the storage device, and to store an indication of the given data pattern and an indication of result data of the search operation in a first record associated with the first file.

Abstract: A file system may save dirty file information indicating portions of files, residing on a storage system, to be modified corresponding data modification requests received by the file system. The file system may update the saved dirty file information to indicate successful modification of the files across redundant storage of the storage system and may track redundancy consistency recovery information on a file-by-file basis or on a file portion basis. A file system may also retrieve saved dirty file information and recover redundancy consistency for portions of files determined, based on the saved dirty file information, not to have redundancy consistency on the storage system. Recovering redundancy consistency may include copying, either directly to via a storage system mechanism, portions of files between redundant storage of the storage system. The redundancy consistency for the saved dirty file information may be recovered prior to recovering redundancy consistency for individual files.

Abstract: A method and system for externally managed file versions. In shared storage environments file versions may be managed utilizing sparse data objects in storage devices. A file version may comprise a chain of one or more sparse data objects, where each object may represent changes compared to the data in other objects of the chain. New, empty sparse data objects may be attached to existing object chains to create new file versions. When providing file version metadata, a metadata server may provide information for the entire object chain. Write permission may be granted only for the front-most object in a chain, while all other objects in the chain may be write-protected. A storage device may expose the presence of holes in sparse data objects to help locate information in the object chain.

Abstract: A system and method for data storage in an array. A system includes a client coupled to a storage subsystem. The storage subsystem comprises data storage locations addressable as rows and columns in an array. Each column comprises a separate storage device. Each row includes redundant data. For a given row, a coordinating storage device receives data from the client, coordinates computation and storage of redundant data, and forwards data to other storage devices. In response to receiving data targeted for storage in a given storage location, a non-volatile, temporary storage device that is associated with the separate storage device that includes the given storage location buffers the received data. The coordinating storage device conveys a write completion message to the client in response to detecting that the data has been buffered in the non-volatile, temporary storage devices. At least two storage devices are coordinating storage devices in separate rows.

Abstract: A file system may selectively perform redundancy consistency recovery and synchronization of mirrors and may store dirty file information in a log or journal that indicates the modification of a file. A file system may update saved dirty file information to indicate successful modification of data across mirrors and may keep track of redundancy consistency recovery information on a file-by-file basis or may track the redundancy consistency of portions of files. A file system may also instruct an underlying storage system not to track data modifications, possibly selectively for each I/O request. During synchronization the file system may synchronize its log and use saved dirty file information to determine files to be synchronized. A file system may selectively use file system logging for individual data modification requests and may expose an API allowing applications to specify a redundancy consistency mechanism.

Abstract: A method for performing file lookups based on tags indicative of file characteristics may involve: a file system storing an association between a tag and a file, where the tag is indicative of a characteristic of the file; the file system storing an additional association between the tag and an additional file that also has the characteristic; the file system searching for files having the tag; and in response to searching, the file system returning identifiers of both files associated with the tag. The returned identifiers may include the files' pathnames or inode identifiers of the files' inodes.

Abstract: A method may involve assigning each of a plurality of freelists and a plurality of hashlists used to implement a cache to one of a plurality of lock groups and acquiring one of a plurality of locks. Objects on each freelist and hashlist that are assigned to the same lock group are allocated from the same one of a plurality of memory allocation pools. Each lock group is associated with a respective one of the plurality of locks. Acquiring the lock locks a freelist and several hashlists included in an associated lock group of the plurality of lock groups.

Abstract: A method may involve: a node included in a cluster looking up a file in a metadata name cache; in response to a miss for the file in the metadata name cache, the node creating a metadata name cache entry for the file in the metadata name cache, where the metadata name cache entry includes a copy of a cookie associated with metadata allocated to the file; in response to a lock on the metadata changing state for the node, the node modifying the cookie associated with the metadata, where the lock changes state for the node in response to a request to a global lock manager for the lock on the metadata; and comparing the copy of the cookie stored in the metadata name cache entry to a current value of the cookie associated with the metadata to determine if the metadata name cache entry is valid.

Abstract: A method may involve assigning each of a plurality of freelists and a plurality of hashlists used to implement a cache to one of a plurality of lock groups and acquiring one of a plurality of locks. Objects on each freelist and hashlist that are assigned to the same lock group are allocated from the same one of a plurality of memory allocation pools. Each lock group is associated with a respective one of the plurality of locks. Acquiring the lock locks a freelist and several hashlists included in an associated lock group of the plurality of lock groups.