Abstract: Transmitting or storing subsegments is disclosed. A data stream or a data block is received and broken into a plurality of segments. For at least one segment, the segment is broken into a plurality of subsegments. A previously stored or transmitted segment similar to the at least one segment is identified. A fingerprint is computed for at least one subsegment. And, using the fingerprint for the at least one subsegment, determining whether the at least one subsegment is identical to a subsegment of the previously stored or transmitted segment without directly comparing the content of the at least one subsegment with the content of the subsegment of the previously stored or transmitted segment.

Abstract: Determining a summary feature set is disclosed. A plurality of subsegments of a first segment are selected. For each subsegment, a plurality of values by applying a set of functions to each subsegment are computed. From all the values computed for all the subsegments, a first subset of values is selected.

Abstract: Stream locality delta compression is disclosed. A previous stream indicated locale of data segments is selected. A first data segment is then determined to be similar to a data segment in the stream indicated locale.

Abstract: A method of managing duplicate segments from a segmented file storage system is disclosed. The method comprises indexing a segment according to a key for the segments wherein the index includes an identification of a first data location where the segment is stored and identifying a duplicate segment having the same key that is stored in a second location. The method further comprises determining that the duplicate segment is an undesired duplicate segment and eliminating the undesired duplicate segment.

Abstract: Determining a summary feature set is disclosed. A plurality of subsegments of a first segment are selected. For each subsegment, a plurality of values by applying a set of functions to each subsegment are computed. From all the values computed for all the subsegments, a first subset of values is selected.

Abstract: A system and method are disclosed for providing efficient data storage. A plurality of data segments is received in a data stream. The system determines whether a data segment has been stored previously in a low latency memory. In the event that the data segment is determined to have been stored previously, an identifier for the previously stored data segment is returned.

Abstract: Data replication with delta compression is disclosed. A primary system and a replica system are determined to both have an identical first data segment that is similar to a second data segment. The second data segment is encoded, wherein the encoding refers to the first data segment.

Abstract: Determining a summary feature set is disclosed. A plurality of subsegments of a first segment are selected. For each subsegment, a plurality of values by applying a set of functions to each subsegment are computed. From all the values computed for all the subsegments, a first subset of values is selected.

Abstract: A system and method are disclosed for providing efficient data storage. A plurality of data segments is received in a data stream. The system determines whether a data segment has been stored previously in a low latency memory. In the event that the data segment is determined to have been stored previously, an identifier for the previously stored data segment is returned.

Abstract: Stream locality delta compression is disclosed. A previous stream indicated locale of data segments is selected. A first data segment is then determined to be similar to a data segment in the stream indicated locale.

Abstract: A system and method are disclosed for providing efficient data storage. A plurality of data segments is received in a data stream. The system preliminarily checks in a memory having a relatively low latency whether one of the plurality of data segments may have been stored previously in a data segment repository. The memory having the relatively low latency stores data segment information. In the event that the preliminary check determines that one of the plurality of data segments may have been stored in the data segment repository, a memory having a relatively higher latency is checked to determine whether the data segment has been stored previously in the data segment repository.

Abstract: Stream locality delta compression is disclosed. A previous stream indicated locale of data segments is selected. A first data segment is then determined to be similar to a data segment in the stream indicated locale.

Abstract: Data replication with delta compression is disclosed. A primary system and a replica system are determined to both have an identical first data segment that is similar to a second data segment. The second data segment is encoded, wherein the encoding refers to the first data segment.

Abstract: Transmitting or storing subsegments is disclosed. A data stream or a data block is received and broken into a plurality of segments. For at least one segment, the segment is broken into a plurality of subsegments. A previously stored or transmitted segment similar to the at least one segment is identified. A fingerprint is computed for at least one subsegment. And, using the fingerprint for the at least one subsegment, determining whether the at least one subsegment is identical to a subsegment of the previously stored or transmitted segment without directly comparing the content of the at least one subsegment with the content of the subsegment of the previously stored or transmitted segment.

Abstract: Data replication with delta compression is disclosed. A primary system and a replica system are determined to both have an identical first data segment that is similar to a second data segment. The second data segment is encoded, wherein the encoding refers to the first data segment.

Abstract: Transmitting or storing subsegments is disclosed. A data stream or a data block is received and broken into a plurality of segments. For at least one segment, the segment is broken into a plurality of subsegments. A previously stored or transmitted segment similar to the at least one segment is identified. A fingerprint is computed for at least one subsegment. And, using the fingerprint for the at least one subsegment, determining whether the at least one subsegment is identical to a subsegment of the previously stored or transmitted segment without directly comparing the content of the at least one subsegment with the content of the subsegment of the previously stored or transmitted segment.

Abstract: Delta compression after identity deduplication is disclosed. A first data segment is determined to be identical to a first previous data segment. A second data segment, not determined to be identical to a second previous data segment, is then determined to be similar to a third previous data segment.

Abstract: A system and method are disclosed for providing efficient data storage. A plurality of data segments is received in a data stream. The system preliminarily checks in a memory having a relatively low latency whether one of the plurality of data segments may have been stored previously in a data segment repository. The memory having the relatively low latency stores data segment information. In the event that the preliminary check determines that one of the plurality of data segments may have been stored in the data segment repository, a memory having a relatively higher latency is checked to determine whether the data segment has been stored previously in the data segment repository.

Abstract: Determining a summary feature set is disclosed. A plurality of subsegments of a first segment are selected. For each subsegment, a plurality of values by applying a set of functions to each subsegment are computed. From all the values computed for all the subsegments, a first subset of values is selected.

Abstract: A system and method are disclosed for providing efficient data storage. A plurality of data segments is received in a data stream. The system preliminarily checks in a memory having a relatively low latency whether one of the plurality of data segments may have been stored previously in a data segment repository. The memory having the relatively low latency stores data segment information. In the event that the preliminary check determines that one of the plurality of data segments may have been stored in the data segment repository, a memory having a relatively higher latency is checked to determine whether the data segment has been stored previously in the data segment repository.