Abstract: Scheduling multiple processes with varying delay sensitivity is disclosed herein. In one example, a processor device iteratively executes a processing workload that includes a fixed-execution-time process and an adjustable-execution-time process. During each iteration of the processing workload, the processor device first determines, for that iteration, a maximum cycle time interval during which both the fixed-execution-time process and an adjustable-execution-time process will execute. The processor device further determines a maximum execution time interval for the adjustable-execution-time process, based on the maximum cycle time interval and a fixed execution time interval for the fixed-execution-time process. The processor device then modifies an adjustable execution time interval for adjustable-execution-time process in the current iteration of the processing workload based on the maximum execution time interval.

Abstract: Scheduling multiple processes with varying delay sensitivity is disclosed herein. In one example, a processor device iteratively executes a processing workload that includes a fixed-execution-time process and an adjustable-execution-time process. During each iteration of the processing workload, the processor device first determines, for that iteration, a maximum cycle time interval during which both the fixed-execution-time process and an adjustable-execution-time process will execute. The processor device further determines a maximum execution time interval for the adjustable-execution-time process, based on the maximum cycle time interval and a fixed execution time interval for the fixed-execution-time process. The processor device then modifies an adjustable execution time interval for adjustable-execution-time process in the current iteration of the processing workload based on the maximum execution time interval.

Abstract: Scheduling multiple processes with varying delay sensitivity is disclosed herein. In one example, a processor device iteratively executes a processing workload that includes a fixed-execution-time process and an adjustable-execution-time process. During each iteration of the processing workload, the processor device first determines, for that iteration, a maximum cycle time interval during which both the fixed-execution-time process and an adjustable-execution-time process will execute. The processor device further determines a maximum execution time interval for the adjustable-execution-time process, based on the maximum cycle time interval and a fixed execution time interval for the fixed-execution-time process. The processor device then modifies an adjustable execution time interval for adjustable-execution-time process in the current iteration of the processing workload based on the maximum execution time interval.

Abstract: Among other things, we describe a technique used in data deduplication that includes receiving a single data file designated to be written to a file storage system configured to store data in the form of blocks. The technique also includes identifying boundaries between portions of data within the single data file, and providing an indication to the file storage system to allocate blocks to the single data file based on the identified boundaries. Each block is allocated to, at most, One of the portions of data. The technique could also be used with objects instead of files.

Abstract: Among other things, we describe a technique used in data deduplication that includes receiving a single data file designated to be written to a file storage system configured to store data in the form of blocks. The technique also includes identifying boundaries between portions of data within the single data file, and providing an indication to the file storage system to allocate blocks to the single data file based on the identified boundaries. Each block is allocated to, at most, One of the portions of data. The technique could also be used with objects instead of files.

Abstract: Among other things, we describe a technique used in data deduplication that includes receiving a single data file designated to be written to a file storage system configured to store data in the form of blocks. The technique also includes identifying boundaries between portions of data within the single data file, and providing an indication to the file storage system to allocate blocks to the single data file based on the identified boundaries. Each block is allocated to, at most, one of the portions of data. The technique could also be used with objects instead of files.

Abstract: Among other things, a technique of data deduplication using multiple devices includes presenting, to host devices, a quantity of logical data storage greater than physical data storage available on a storage area network (SAN) device, by deduplicating at least some of the data received from the host devices for storage at the SAN device in a manner that is coordinated with over-provisioning capability of the SAN device.

Abstract: A method for storing data items in a data repository that allows a client storing a data item to lose the ability to either retrieve the stored data item or enable its retrieval, while preserving controlled access to the stored data item indefinitely. The client storing the data item encrypts it using an encryption key that it creates, which may be derived from the content of the data item. The encryption key is also stored in the data repository in an encrypted form, with its encryption making use of a public key. The method allows storage space to be shared for identical data items encrypted and stored by independent clients of the data repository.

Abstract: In one aspect, in general, a method for managing data includes receiving, at a data deduplication engine of a computer system, data to be stored in a data storage system, computing a first value corresponding to at least one portion of the received data, determining that the first value matches a second value stored in a collection of identifiers, each identifier corresponding to a particular portion of data stored in the data storage system, each identifier comprising a computed value corresponding to the particular portion of data and including metadata indicating a location where the particular portion of data is being stored in the data storage system, and based on the determination that the first value matches a second value stored in the collection of identifiers, reallocating one or more physical locations from corresponding one or more virtual locations to different respective one or more virtual locations.

Abstract: In one aspect, in general, a method for managing data in a data storage system comprises receiving data to be stored in the data storage system, computing values corresponding to different respective portions of the received data, generating identifiers corresponding to different respective portions of the received data, with an identifier corresponding to a particular portion of data including the computed value corresponding to the particular portion of data and metadata indicating a location where the particular portion of data is being stored in the data storage system, and storing at least some of the identifiers in an index until the index reaches a predetermined size.

Abstract: Managing data in a data storage system includes: receiving data to be stored in the data storage system; computing values corresponding to different respective portions of the received data; generating identifiers corresponding to different respective portions of the received data, with an identifier corresponding to a particular portion of data including the computed value corresponding to the particular portion of data and metadata indicating a location where the particular portion of data is being stored in the data storage system; storing at least some of the identifiers in an index until the index reaches a predetermined size; and in response to determining that a first identifier corresponding to a first portion of data, received after the index reached the predetermined size, was not already stored in the index before the first portion of data was received, storing the first identifier in the index and designating for removal at least a second identifier corresponding to a second portion of data to be

Abstract: A method by which a disk-based distributed data storage system is organized for protecting historical records of stored data entities. The method comprises recording distinct states of an entity, corresponding to different moments of time, as separate entity versions coexisting within the distributed data storage system, and assigning expiration times to the entity versions independently within each of a plurality of storage sites according to a shared set of rules, before which times deletion is prohibited.

Abstract: A method by which a disk-based distributed data storage system is organized for protecting historical records of stored data entities. The method comprises recording distinct states of an entity, corresponding to different moments of time, as separate entity versions coexisting within the distributed data storage system, and assigning expiration times to the entity versions independently within each of a plurality of storage sites according to a shared set of rules, before which times deletion is prohibited.

Abstract: A method by which a disk-based distributed data storage system is organized for protecting historical records of stored data entities. The method comprises recording distinct states of an entity, corresponding to different moments of time, as separate entity versions coexisting within the distributed data storage system, and assigning expiration times to the entity versions independently within each of a plurality of storage sites according to a shared set of rules, before which times deletion is prohibited.

Abstract: A method for keeping track, in an electronic data processing system, of when references made to an element that is stored in memory have all been removed, the method being designed to deal with failures in a manner that does not falsely conclude there are no references.

Abstract: In general, the invention features methods by which more than one client program connected to a network stores the same data item on a storage device of a data repository connected to the network. In one aspect, the method comprises encrypting the data item using a key derived from the content of the data item, determining a digital fingerprint of the data item, and storing the data item on the storage device at a location or locations associated with the digital fingerprint. In a second aspect, the method comprises determining a digital fingerprint of the data item, testing for whether the data item is already stored in the repository by comparing the digital fingerprint of the data item to the digital fingerprints of data items already in storage in the repository, and challenging a client that is attempting to deposit a data item already stored in the repository, to ascertain that the client has the full data item.

Abstract: A method by which a disk-based distributed data storage system is organized for protecting historical records of stored data entities. The method comprises recording distinct states of an entity, corresponding to different moments of time, as separate entity versions coexisting within the distributed data storage system, and assigning expiration times to the entity versions independently within each of a plurality of storage sites according to a shared set of rules, before which times deletion is prohibited.

Abstract: A method for turning variable-length file path names into fixed-length object identifiers in a data repository, the method comprising, assigning a fixed-value identifier to a file system in the data repository, computing a first deterministic pseudo-random function that combines a first component of a plurality of file system pathname components associated with a file with the fixed-value identifier, computing a second deterministic pseudo-random function that combines a second component of the plurality of file system pathname components with the result of the first function, and using at least one of the computed deterministic pseudo-random functions to produce a fixed-length object identifier for the file.

Abstract: In general, the invention features methods by which more than one client program connected to a network stores the same data item on a storage device of a data repository connected to the network. In one aspect, the method comprises encrypting the data item using a key derived from the content of the data item, determining a digital fingerprint of the data item, and storing the data item on the storage device at a location or locations associated with the digital fingerprint. In a second aspect, the method comprises determining a digital fingerprint of the data item, testing for whether the data item is already stored in the repository by comparing the digital fingerprint of the data item to the digital fingerprints of data items already in storage in the repository, and challenging a client that is attempting to deposit a data item already stored in the repository, to ascertain that the client has the full data item.

Abstract: A method by which a disk-based distributed data storage system is organized for protecting historical records of stored data entities. The method comprises recording distinct states of an entity, corresponding to different moments of time, as separate entity versions coexisting within the distributed data storage system, and assigning expiration times to the entity versions independently within each of a plurality of storage sites according to a shared set of rules, before which times deletion is prohibited.