Abstract: Potentially identical objects (such as files) across multiple computers are located. In one embodiment, a computer generates object information for an object stored on the computer. The object information can be generated in a variety of manners (e.g., based on hashing the object, based on characteristics of the object, and so forth). The object information is then transferred to one or more database server computers, where the object information can be compared to object information from other computers to determine whether the object is potentially identical to another object on one of the other computers.

Abstract: Potentially identical objects (e.g., files) are located across multiple computers based on stochastic partitioning of workload. For each of a plurality of objects stored on a plurality of computers in a network, a portion of object information corresponding to the object is selected. The object information can be generated in a variety of manners (e.g., based on hashing the object, based on characteristics of the object, and so forth). Any of a variety of portions of the object information can be used (e.g., the least significant bits of the object information). A stochastic partitioning process is then used to identify which of the plurality of computers to communicate the object information to for identification of potentially identical objects on the plurality of computers.

Abstract: In a serverless distributed file system, the writer of a file can provide file authentication information to a verifying machine without having to compute a new digital signature every time a written file is closed. Periodically, the writer compiles a list of the hash values of all files that have been written over a recent interval, computes a hash of the list, and signs the hash. This signed list of hash values is known as a manifest, akin to a shipping manifest that enumerates the items in a shipment. The advantage of using a signed manifest is that the writer need only perform a single signature computation in order to authenticate the writes to multiple files, rather than having to compute a separate signature for each file, as it would if a signature were embedded in each file.

Abstract: A file format for a serverless distributed file system is composed of two parts: a primary data stream and a metadata stream. The data stream contains a file that is divided into multiple blocks. Each block is encrypted using a hash of the block as the encryption key. The metadata stream contains a header, a structure for indexing the encrypted blocks in the primary data stream, and some user information. The indexing structure defines leaf nodes for each of the blocks. Each leaf node consists of an access value used for decryption of the associated block and a verification value used to verify the encrypted block independently of other blocks. In one implementation, the access value is formed by hashing the file block and encrypting the resultant hash value using a randomly generated key. The key is then encrypted using the user's key as the encryption key. The verification value is formed by hashing the associated encrypted block using a one-way hash function.

Abstract: A serverless distributed file system manages the storage of files and directories using one or more directory groups. The directories may be managed using Byzantine-fault-tolerant groups, whereas files are managed without using Byzantine-fault-tolerant groups. Additionally, the file system may employ a hierarchical namespace to store files. Furthermore, the directory group may employ a plurality of locks to control access to objects (e.g., files and directories) in each directory.

Abstract: A method and system for storing the data of files having duplicate content, by maintaining a single instance of the data, and providing logically separate links to the single instance. Files of duplicate content have their data stored in a common store file by a single instance store (SIS) facility, which also converts the original file or files to links to that common store file and creates additional links thereto as needed. The SIS facility may reside above a file system as a filter driver. File system requests directed to the link file (e.g., open, write, read, close and delete) reach the SIS filter, which then transparently handles each request as if the link file was a normal file. To preserve logical separation, writes to a SIS link file are to the link file, and the written portion recorded as dirty. The SIS filter intercepts SIS read requests, and reads clean portions from the common store file and any dirty portions from the link file.

Abstract: A distributed file server system has multiple data servers connected to stream data files continuously over a network to multiple clients. Each data server supports at least one storage device, such as a disk. Data files are distributed across the data servers so that data blocks of the data files are stored on each of the storage devices. The file server system has a scheduler located at each of the data servers to manage distributed portions of a schedule. Each data server sees a different portion of the schedule, but no one data server sees the whole schedule. The scheduler facilitates service of requested data streams from its corresponding data server according to a schedule portion that is available to the data server. The scheduler determines whether to insert a new data stream into the current slot it presently owns in its schedule portion, or to wait for a subsequent slot in the schedule.

Abstract: A file that has been encrypted using a symmetric key and that has a corresponding access control entry with the symmetric key encrypted using the public key of a public/private key pair can be accessed. An encrypted key cache is also accessed to determine whether an access control entry to symmetric key mapping exists in the cache for the access control entry corresponding to the file. If such a mapping exists in the cache, then the mapped-to symmetric key is obtained form the cache, otherwise the encrypted symmetric key is decrypted using the private key of the public/private key pair. The encrypted key cache itself can also be encrypted and stored as an encrypted file.

Abstract: An exclusive encryption system is established using multiple computing devices. The exclusive encryption system allows for the exclusion of certain plaintext (e.g., by one of the computing devices) and ciphertext (e.g., by another of the computing devices) while at the same time maintaining the privacy created by the encryption (e.g., so the other computing device cannot see the plaintext). The exclusive encryption system may be implemented as part of a serverless distributed file system with directory entries (e.g., file names or folder names) being the plaintext, or alternatively as part of other systems.

Abstract: A distributed file server system has multiple data servers connected to stream data files continuously over a network to multiple clients. Data files are distributed across the data servers so that data blocks of the data files are stored at manage distributed portions of a schedule. Each data server sees a different portion of the schedule, but no one data server sees the whole schedule. The scheduler facilitates service of requested data streams from its corresponding data server according to a schedule portion that is available to the data server. The scheduler determines whether to insert a new data stream into the current slot it presently owns in its schedule portion, or to wait for a subsequent slot in the schedule based on a thrifty policy.

Abstract: In a network of interconnected multimedia source, transfer, and sink ports, the described subject matter defers port parameter selection until substantially all relevant information is available. Specifically, parameters are negotiated between multiple ports by specifying parameter sets corresponding to the ports. Each parameter set is expressed as a list of constraints on allowable values or ranges of values. When connecting multiple ports, the constraints on their parameter sets are conjoined to form a parameter set intersection. The parameter sets of the ports are limited to values included in the parameter set intersection. Parameter sets relating to other ports are potentially limited based on the intersection results. The conjoining and limiting operations are repeated until the various parameter sets exhibit no further limitations in response to further repetitions. This process is repeated for all port interconnections.

Abstract: A method and system that operates as a background process automatically identify and merge duplicate files into a single instance files, wherein the duplicate files become independent links to the single instance files. A groveler maintains a database of information about the files on a volume, including a file size and checksum (signature) based on the file contents. The groveler periodically acts in the background to scan the USN log, a log that dynamically records file system activity. New or modified files detected in the USN log are queued as work items, each work item representing a file. The volume may be scanned to add work items to the queue, which takes place initially or when there is a potential problem with the USN log. The groveler periodically removes items from the queue, calculates the signature of the corresponding file contents, and uses the signature and file size to query the database for matching files.

Abstract: A method and system for incrementally improving the layout of a program image of a computer program to reduce the working set. The system iteratively selects pairs of basic blocks and reorders the basic blocks in the range delimited by the selected pair of basic blocks. The system selects the pairs of basic blocks so that the working set of the computer program is improved by reordering the basic block in the range. Thus, during each iteration, the working set is improved. The system continues with these iterations until a termination condition (e.g., number of iterations) is satisfied. In one embodiment, during each iteration the system designates one of the basic blocks as an initial anchor basic block. The system then repeats the following until the same range of basic blocks is identified twice in a row.

Abstract: Components in a system have interconnectable ports, groups of which are to be interconnected. A method of negotiating negotiable parameters between a plurality of the ports includes specifying parameter sets corresponding respectively to the ports. Each parameter set is expressed as a list of constraints on allowable values or ranges of values. When connecting a group of ports, the constraints on their parameter sets are conjoined to form a parameter set intersection. The parameter sets of the ports are then limited to those values included in the parameter set intersection. In addition, parameter sets relating to other ports are potentially limited based on the results of the intersection. The conjoining and limiting steps are repeated until the various parameter sets exhibit no further limitations in response to further repetitions, and this process is repeated for all port interconnections. Subsequently, values for the negotiable parameters are selected from the limited parameter sets.

Abstract: A computer system for establishing a program image layout of a computer program. The computer program has basic blocks that each have temporal usage vector indicating time intervals during which the basic block was accessed during an execution of the computer program. For each of the basic blocks, the layout system initializes a cluster to contain that basic block and the layout system repeats the following until one cluster remains that contains all of the basic blocks. For each possible ordered pair of clusters, the layout system determines a metric value for the ordered pair of clusters, the metric value being derived from the temporal usage vectors of the basic blocks of the cluster and estimating an effect on the performance of the computer program when that ordered pair of clusters are position in the program image in that order and contiguously. The layout system then selects the ordered pair whose metric value estimates the smallest effect on the performance of the computer program.

Abstract: A method and computer program product for scheduling network communication packets in a multimedia environment where different packet streams have reservations of network bandwidth to form packet flows. The present invention divides the packet scheduling function into distinct components that may be implemented as separate drivers in a layered driver environment as exists, for example, in the Microsoft Windows NT operating system. One component is called a conformer and will generate and assign to each packet in the packet flow at least one conformance time that signifies the earliest a packet may be sent and still conform to the network resource requirements associated with the flow. Many different conformance algorithms can be supported so that the best algorithm is used for a particular packet flow and the service requirements that it represents. Should it be necessary to actually hold a packet until the conformance time is met, a shaper component is used to delay the packets.

Abstract: A method and system for remapping physical memory that is malfunctioning. The physical memory has memory locations with addresses. The addresses are ordered from a lowest to a highest address, and each address has bits ordered from a highest-order bit to a lowest-order bit. The system scans physical memory to determine which memory locations are malfunctioning. The system identifies a lowest address and a highest address of the memory locations that are malfunctioning. The system then identifies the highest-order, contiguous bits of the lowest address that are the same as the highest-order, contiguous bits of the highest address. The system generates a remapping value whose highest-order bits are equal to the inverse of the identified highest-order, contiguous bits of the address and whose lowest-order bits are all zeroes. When the system receives an address to access physical memory, the system generates a remapped address by performing a bitwise exclusive-OR of the received address with the remapping value.

Abstract: The present invention is embodied in a system and method for generating and validating reference handles for consumers requiring access to resources in a multi-threaded environment of a computer system. The system of the present invention includes a resource manager having a handle administrator, a plurality of consumers, and a plurality of resources. The handle administrator includes an assignment routine, a release routine, a dereference routine, and multi-threading sub-routines enabling operation of the present invention in multi-threaded environments.

Abstract: A method and computer program product for classifying data being processed by drivers included in a stack. The computer program product is a centralized packet classifier that receives classification requests from drivers or other clients according to a defined interface. This reduces the cost for producing a driver since less code is required initially at development time as well as a corresponding reduction in maintenance costs. The present invention groups drivers or other clients that utilize a certain body of classification information into classification families. Further, reference patterns used to make classifications are stored in separate databases depending on whether they are specific or general.

Abstract: The present invention is embodied in a system and method for generating and validating reference handles for consumers requiring access to resources in a computer system. The system of the present invention includes a resource manager having a handle administrator, a plurality of consumers, and a plurality of resources. The handle administrator includes an assignment routine, a release routine, and a dereference routine. The assignment routine issues new handles, the release routine releases handles that are no longer required (thus rendering the handle invalid), and the dereference routine dereferences handles into a pointer to a resource, which entails verifying that the handle is valid.