Abstract: A method for creating an embedding node. The method includes creating a first hash-based directed acyclic graph (“HDAG”) having a first node, which includes data, and creating a second HDAG having a second node that includes one or more data fields that store the first node.

Abstract: Data objects are selectively stored across a plurality of differential data stores, where selection of the differential data stores for storing respective data objects is according to a criterion relating to compression of the data objects in each of the data stores, and where the differential data stores are stored in persistent storage media. Plural requests for accessing the differential data stores are batched, and one of the differential data stores is selected to page into temporary storage from the persistent storage media. The batched plural requests for accessing the selected differential data store that has been paged into the temporary storage are executed.

Abstract: A plurality of differential data stores are stored in persistent storage media. In response to receiving a first request to store a particular data object, one of the differential data stores that are stored in the persistent storage media is selected, wherein selecting the one differential data store is according to a criterion relating to compression of data objects in the differential data stores. The selected differential data store is copied into temporary storage media, where the copying is not delayed after receiving the first request to await receipt of more requests. The particular data object is inserted into the copy of the selected differential data store in the temporary storage media, where the inserting is performed without having to retrieve more data from the selected differential store in the persistent storage media.

Abstract: A method, apparatus, and system identity files with similar content. One embodiment is a method that divides files into plural segments. The method computes a hash value and a size for each of the plural segments of the files. In order to identify which files have similar content, the method adds together segments common between files. File similarity information of files with similar content is output.

Abstract: Input data is divided into multiple segments that are processed by processing elements of a computer. The processing of the segments produces a plurality of tentative sets of chunks. The plurality of tentative sets of chunks are stitched together to produce an output set of chunks.

Abstract: A file is divided into plural segments, and for each of at least some of the plural segments, plural windows covering different portions of the corresponding segment are defined. According to a predefined criterion, one of the windows associated with the corresponding segment is selected. A representative hash based on the portion in the selected window is calculated for representing the corresponding segment.

Abstract: A method of limiting redundant storage of data comprises receiving a data stream and partitioning the data stream into a series of data chunks. At least one content hash value for a set of data chunks is generated based on data content of the set of data chunks. One or more data chunks are grouped into a segment with at least one boundary of the segment defined based on an evaluation of content hash values of data chunks. Content hash values of data chunks of the segment are compared to content hash values of data chunks of segments stored on a backup mass storage device. A pointer to a stored data chunk of an existing segment is stored on the backup mass storage device if a content hash value of a data chunk of the segment matches the content hash value of the stored data chunk.

Abstract: Provided are, among other things, systems, methods and techniques for identifying target images in the following manner. Keypoints at different locations within a sample image and descriptor vectors for the keypoints are obtained, the descriptor vectors describing local image information around the keypoints. Features are generated based on hashes of data vectors that include at least one of the descriptor vectors, and an image-feature database is searched for the features. Target images are retrieved and provided based on the number of the features for which corresponding features exist in individual database images whose features are represented in the image-feature database. The foregoing hashes are performed using a hash function for which a probability that the hash of an arbitrary first vector corresponds to the hash of an arbitrary second vector is a function of an angle between the first vector and the second vector.

Abstract: Provided are systems, methods and techniques for hierarchically segmenting a sequence of records, by specifying windows, each window encompassing a portion of the sequence. For each specified window, a function of the data records within that specified window is calculated in order to generate a function value for the window. The function values are then used to define a hierarchical arrangement in which the sequence is divided into a plurality of segments and the segments are divided into smaller segments.

Abstract: A flash memory device is powered via an RF field generated by a host unit. A contactless communication channel is established between the flash memory device and the host unit. A first version of a data file, including a first plurality of data chunks, is downloaded from the flash memory device to the host unit. A second version of the data file, including a second plurality of data chunks, is generated at the host unit. Data chunk modifications to the first plurality of data chunks necessary to generate the second plurality of data chunks are identified at the host unit. The contactless communication channel is reestablished. The identified data chunk modifications to the first plurality of data chunks are performed thereby updating the first version of the data file stored on the flash memory device to the second version of the data file.

Abstract: To detect duplicative hierarchically arranged sets of files in a storage system, a method includes generating, for hierarchically arranged plural sets of files, respective collections of values computed based on files in corresponding sets of files. For a further set of files that is an ancestor of at least one of the plural sets of files, a respective collection of values that is based on the collection of values computed for the at least one set is generated. Duplicative sets according to comparisons of the collections of values are identified.

Abstract: The present invention provides a system for and a method of data cache management. In accordance with an embodiment, of the present invention, a method of cache management is provided. A request for access to data is received. A sample value is assigned to the request, the sample value being randomly selected according to a probability distribution. The sample value is compared to another value. The data is selectively stored in the cache based on results of the comparison.

Abstract: Provided are, among other things, systems, methods and techniques for scanning information across multiple different devices. In one representative system, remote data-processing devices are provided with scanning applications that repeatedly scan information on their respective data-processing devices to identify matching data units that satisfy a specified matching criterion, the specified matching criterion including required matches against a set of screening digests, and then transmit characteristic information regarding the matching data units; and a central processing facility receives the characteristic information from the remote data-processing devices and determines whether the corresponding matching data units satisfy a policy criterion.

Abstract: To manage storing of data in a data structure, a particular data value is represented as a group of segments stored in corresponding entries of the data structure. Additional data values represented by corresponding groups of segments are written into the data structure. A probability of overwriting segments representing the particular data value increases as a number of the additional data values increase. A correct version of the particular data value is retrieved even though one or more segments representing the particular data value has been overwritten.

Abstract: To generate at least one hash value for a feature vector that represents a data object, a discrete orthogonal transform is applied on a second vector produced from the feature vector. Applying the discrete orthogonal transform on the second vector produces a third vector. At least one value is selected from the third vector to produce the hash value. The at least one hash value is used to perform an action.

Abstract: Provided is a method for creating sketches of sets to permit comparing set members selected from a universe. The method includes selecting a size for the sketch and providing a randomizer. In addition, a set is provided as a subset of the universe. The set is processed with the randomizer to provide a new vector. The new vector is normalized to provide a value. Dividing each element of the new vector by the determined value results in the sketch.

Abstract: Provided are, among other things, systems, methods and techniques for identifying related objects in a computer database. In one representative implementation: (a) a feature vector that describes an existing object is obtained; (b) comparison scores are generated between the feature vector and various sample vectors; (c) a set that includes at least one designated vector is identified from among the sample vectors by evaluating the generated comparison scores; (d) a computer database is searched for matches between label(s) for the designated vector(s) and labels for representative vectors for other objects represented in the computer database; and (e) at least one related object is identified based on the identified match(es).

Abstract: Data processing apparatus comprising: a chunk store containing specimen data chunks, a manifest store containing at least one manifest that represents at least a part of a data set and that comprises at least one reference to at least one of said specimen data chunks, a sparse chunk index containing information on only those specimen data chunks having a predetermined characteristic, the processing apparatus being operable to process input data into input data chunks and to use the sparse chunk index to identify at least one of said at least one manifest that includes at least one reference to one of said specimen data chunks that corresponds to one of said input data chunks having the predetermined characteristic.

Abstract: Data processing apparatus comprising: a chunk store containing specimen data chunks, a manifest store containing a plurality of manifests, each of which represents at least a part of a data set and each of which comprises at least one reference to at least one of said specimen data chunks, a sparse chunk index containing information on only some specimen data chunks, the processor being operable to: process input data into input data chunks; identify manifests having at least one reference to one of said specimen data chunks that corresponds to one of said input data chunks and on which there is information contained in the sparse chunk index; and prioritize the identified manifests for subsequent operation.

Abstract: Embodiments of the present invention pertain to determining an approximate number of instances of an item for an organization. According to one embodiment, instances of items that reside on computer systems associated with the organization are determined. Instances of the same item can reside on different computers and an identification uniquely identifies an item. Random numbers are associated with identifications of the items. An approximate number of instances of the item is determined based on a highest random number associated with the item. The highest random number is the highest of the random numbers that were generated for the instances of the item.