Abstract: “A system and associated method restore a file system from incremental backups in the presence of deletion, without restoring deleted files. This is accomplished by logging deletions in a deletion log. Using the deletion log, the system re-deletes the deleted files upon a restore operation to an incremental backup epoch. The system the logs deletions so that the files that were deleted and survived a backup epoch have their records logged. The activity log enables an incremental backup from a prior snapshot, and represents a point-in-time version of the file system. The system can either not restore the deleted files, or, alternatively, it can restore the deleted files and then re-deletes them.

Abstract: In a distributed file system the distributed storage management is made useful to a variety of applications. Multiple quality of service options are provided through locking. Three locking systems are provided. The system offers a locking system designed for sequential consistency with write-back caching, typical of distributed file systems. A second locking system is provided for sequential consistency with no caching for applications that manage their own caches. Finally, a locking system that implements a weaker consistency model with write-back caching, designed for efficient replication and distribution of data is included. Locks for replication are suitable for serving dynamic data on the Internet and other highly-concurrent applications. The selection of the appropriate lock protocol for each file is set using the file metadata. Further, a novel locking system is provided for the lock system implementing a weak consistency model with write back caching.

Abstract: A system and method for managing access to assets in a distributed data storage system includes requesting, from a client computer, a lock from a server computer. The lock is evaluated using a bitwise comparison of the protected access modes and restricted access modes defined by the lock with the protected and restricted modes defined by the currently outstanding locks using an algorithm, such that a potentially large compatibility table is not needed.

Abstract: A data management system for storage area networks (SANs) includes an authentication and directory service that integrates the management of cache coherency and synchronization into an authentication mechanism, referred to herein as a ticket. The data in the system is notionally classified into colors. When a principal computer requests a read or a write access to data of a particular “color”, the service determines whether the principal is authorized for the requested access, and if so, the service issues a ticket to the principal that specifies a data color that the bearer of the ticket is authorized to access using the type of access mechanism specified on the ticket. A hidden “hue” can also be specified on the ticket. The service can variously inform principals to cancel tickets based on color or time of issue and can also variously inform storage devices not to honor tickets of a certain hue or time of issue as appropriate to thereby maintain cache coherency and data consistency.

Abstract: A system, method, and computer program product for a lease-based timeout scheme that addresses fencing's shortcomings. Unlike fencing, this scheme (or protocol) enables an isolated computer to realize it is disconnected from the distributed system and write its dirty data out to storage before its locks are stolen. In accordance with the invention, data consistency during a partition in a distributed system is ensured by establishing a lease based protocol between in the distributed system wherein a client can hold a lease with a server. The lease represents a contract between a client and a server in which the server promises to respect the client for a period of time. The server respects the contract even when it detects a partition between the client and itself.

Abstract: A data storage system randomly determines a start offset at which to write objects to a storage medium. For updated blocks of the object, e.g., for blocks written during copy-on-write as part of a point-in-time snapshot, the updated block is written in the region of the original file or as close thereto as possible to achieve “virtual contiguity”. Subsequent reads of the object read entire region containing both the object and, potentially, “chaff” data other than the object. The “chaff” data is discarded by the I/O system or file system using, e.g., a bit mask, subsequent to the read.

Abstract: A system and associated method restore a file system from incremental backups in the presence of deletion, without restoring deleted files. This is accomplished by logging the deletions in a deletion log. Using the deletion log, the system re-deletes the deleted files upon a restore operation to an incremental backup epoch. The system the logs deletions so that the files that were deleted and survived a backup epoch have their records logged. The activity log enables an incremental backup from a prior snapshot, and represents a point-in-time version of the file system. The system can either not restore the deleted files, or, alternatively, it can restore the deleted files and then re-deletes them.

Abstract: A system and method for managing access to assets in a distributed data storage system includes requesting, from a client computer, a semi-preemptible access lock from a server computer. The semi-preemptible lock, if granted, is held by the client computer as long as the server does not demand it back, with the client computer granting open instances under non-preemptible file locks for the asset to which the locks pertain as long as the client computer holds the semi-preemptible lock. When another client computer requests the semi-preemptible lock, the server can demand the lock from the holding client, which relinquishes the lock if no open instances are protected by the lock. Otherwise, the holding client computer first attempts to downgrade its lock to meet the request, and if compatibility is not achieved thereby, the holding client refuses to relinquish the lock.

Abstract: In a distributed file system the distributed storage management is made useful to a variety of applications. Multiple quality of service options are provided through locking. Three locking systems are provided. The system offers a locking system designed for sequential consistency with write-back caching, typical of distributed file systems. A second locking system is provided for sequential consistency with no caching for applications that manage their own caches. Finally, a locking system that implements a weaker consistency model with write-back caching, designed for efficient replication and distribution of data is included. Locks for replication are suitable for serving dynamic data on the Internet and other highly-concurrent applications. The selection of the appropriate lock protocol for each file is set using the file metadata. Further, a novel locking system is provided for the lock system implementing a weak consistency model with write back caching.

Abstract: A bit-parallel system and method for allocating storage space for data objects using a bitmap. It is determined whether a suffix of free space bits in a previous bitmap word can be used with a prefix of free space bits in a current word that is contiguous to the previous word. If so, this renders a string of free space bits spanning multiple words, and it is determined whether the string represents a sufficiently large number of contiguous blocks (“the target”) to store the data object. If not, it is determined whether sufficient contiguous free space bits in the current word exist to fulfill the target. If the target still can't be achieved, the longest suffix of free space bits in the current word is found for possible use with the prefix of the next contiguous word, and the next word is then retrieved and processed. The algorithm for finding the suffix preferably is undertaken by considering bits in parallel.

Abstract: A bit-parallel system and method for allocating storage space for data objects using a bitmap. It is determined whether a suffix of free space bits in a previous bitmap word can be used with a prefix of free space bits in a current word that is contiguous to the previous word. If so, this renders a string of free space bits spanning multiple words, and it is determined whether the string represents a sufficiently large number of contiguous blocks (“the target”) to store the data object. If not, it is determined whether sufficient contiguous free space bits in the current word exist to fulfill the target. If the target still can't be achieved, the longest suffix of free space bits in the current word is found for possible use with the prefix of the next contiguous word, and the next word is then retrieved and processed. The algorithm for finding the suffix preferably is undertaken by considering bits in parallel.

Abstract: A data storage system randomly determines a start offset at which to write objects to a storage medium. For updated blocks of the object, e.g., for blocks written during copy-on-write as part of a point-in-time snapshot, the updated block is written in the region of the original file or as close thereto as possible to achieve “virtual contiguity”. Subsequent reads of the object read entire region containing both the object and, potentially, “chaff” data other than the object. The “chaff” data is discarded by the I/O system or file system using, e.g., a bit mask, subsequent to the read.

Abstract: A decentralized file system based on a network of remotely encrypted storage devices is disclosed. The file system includes a network to which a network client, a secure remotely encrypted storage device, a key manager, and a lock manager are attached. The system organizes data as files and directories. Files or directories are composed of one or more streams, which logically partition the data associated with the files or directories. The device serves as a repository of the system's data. The key manager controls data access keys while the lock manager handles consistency of the files. A network user may have read or write access to a file. Access is controlled using keys and access lists maintained by the key manager.

Abstract: A method and a system are presented for generating differentially compressed output from binary sources. Given two versions of the same file as input streams, a compact encoding of one of the input streams is generated, by representing it as a set of changes with respect to the other input stream. Algorithms for differencing files requiring time linear in the size of the input and a constant amount of space for execution are presented. In addition, advanced techniques for improving existing differencing algorithms are developed and applied to previous methods. These techniques allow algorithms to increase their efficiency without a loss of compression and to accept arbitrarily large inputs without sacrificing correctness or degrading the compression data rate. The differential compression methods provide a computationally efficient compression technique for applications that generate versioned data.

Abstract: A computing system provides continuous availability of data files that are maintained at a file management system and linked to a database management system (DBMS) through a Datalink data type, even while any particular file is being changed with DBMS append or update operations. When a file is linked, it is designated to be available for read-only operations. A user who wants to perform updates on a file gets a "check-out" copy of the file for updating operations, such that the original file remains linked to the database system while the copy is being updated and remains available to other users. The file management system includes a "check-in" function that receives the updated file, saves the updated file under a new name different from the original, updates the Datalink, generates new metadata for the updated file, and transactionally updates the file with its new metadata. In this way, data files are continuously available to all users through appending and updating actions.

Abstract: The present invention discloses a method, apparatus, and article of manufacture for generating, transmitting, replicating, and rebuilding in-place reconstructible software updates to a file from a source computer to a target computer updates to a file from a source computer to a target computer. The present invention stores the first version of the file and the updates to the first version of the file in the memory of the source computer. The first version is also stored in the memory of the target computer. The updates are then transmitted from the memory of the source computer to the memory of the target computer. The present invention uses these updates at the target computer to build the second version of the file in-place.

Abstract: A network storage device is disclosed for use in a secure array of such devices to support a distributed file system. Each device is an independent repository of remotely encrypted data objects to be accessed by authorized network clients. All encryption is done by the clients, rather than by the devices. In order for the system to revoke access to an object on the device, the object must be re-encrypted. Each storage device has a device owner for controlling access to the device's data. All data requests from the clients and responses to them are authenticated using keys derived from the owner key and hashed message authentication codes.