Abstract: In one embodiment, circuitry may generate digests to be combined to produce a hash value. The digests may include at least one digest and at least one other digest generated based at least in part upon at least one CRC value and at least one other CRC value. The circuitry may include cyclical redundancy check (CRC) generator circuitry to generate the at least one CRC value based at least in part upon at least one input string. The CRC generator circuitry also may generate the at least one other CRC value based least in part upon at least one other input string. The at least one other input string resulting at least in part from at least one pseudorandom operation involving, at least in part, the at least one input string. Many modifications, variations, and alternatives are possible without departing from this embodiment.

Abstract: A method, system and apparatus for efficient storage of small files in a segment-based deduplication scheme by allocating multiple small files to a single data segment is provided. A mechanism for distinguishing between large files (e.g., files that are on the order of the size of a segment or larger) and smaller files, and starting a new segment at the beginning of a large file is also provided. A file attribute-based system for determining an identity of a small file at which to begin a new segment and then allocating subsequent small files to that segment and contiguous segments until a next small file having an appropriate attribute subsequently is encountered to begin a new segment is further provided. In one aspect of the present invention a filename hash is used for file attribute analysis to determine when a new segment should begin.

Abstract: The present invention refers to a computer-implemented method of anonymizing data of a database (2), wherein the method comprises the following steps: a. exporting at least one data record (10) from the database (2), wherein the data record (10) has a structure and comprises content (100); and b. anonymizing at least part of the content (100) to produce at least one anonymized data record (3); c. wherein the anonymized data record (3) has the same structure than the data record (10) read from the database (2).

Abstract: A method for storing electronic data blocks at a storage facility uses a public database and a select database. Hash for each data block is evaluated at the facility to determine whether the data block is already stored at the facility. New data blocks are assigned a new address in the select database when encrypted with a customer key. Otherwise, they are assigned a new address in the public database by default. Duplicate data blocks are assigned a previously established address for the data block in either the public or select database. All addresses are then sent to the customer location for file integrity and only the content of new data blocks need to be sent to the storage facility (i.e. no need for duplicates).

Abstract: An apparatus and methods that use trusted platform modules (TPM) to perform integrity measurements of multiple subsystems are disclosed. The state of platform configuration registers (PCRs) after boot up are stored as the base state of the system. In another embodiment, and application that is to be verified requests that its state be extended from the base state of the system. When such a request is received, the state of the system is extended directly from the base state PCR contents and not from the system state. In another embodiment, a virtual PCR is used, where such a virtual PCR uses a larger memory space than a conventional TPM provides for a physical PCR, by use of encrypted storage on external, protected memory.

Abstract: A resilient hashing system leverages a fast, non-cryptographic hash/checksum function that has good diffusion properties while remaining reasonably efficient on modern central processing units (CPUs). The hash function uses random secret data so that hash keys for particular data are difficult to predict. Due to its internal structure, well-chosen random secret data is difficult for an attacker to counter without having access to the direct output of the hash function. At every stage of the block function, there are at least two operations that can be performed in parallel, increasing performance on modern superscalar CPUs. Thus, the resilient hashing system provides a hash table and checksum that can be used in Internet-facing or other vulnerable sources of input data to manage performance in the face of malicious attacks.

Abstract: A method for electronically signing an electronic transcript. The method includes obtaining an image of a court reporter's signature which may be incorporated into an electronic transcript. A hash operation performed on the electronic transcript to generate a representation of the contents of the electronic transcript. The representation of the contents of the electronic transcript is recorded/time stamped by a digital notary service, from which a notary record of the time stamping is obtained from the digital notary service. The notary record is digitally signed, and an electronically signed electronic transcript is formed by bundling the digitally signed notary record with the electronic transcript and with the data identifying the user. In this manner, an electronic transcript with an electronic signature is created. The electronic transcript may be viewed, with the image displayed in the viewer having an image of the court reporter's signature.

Abstract: A method and apparatus for removing duplicated data in a file system utilizing the concept of storage tiers. A synthetic namespace is created via file virtualization, and is comprised of one or more file systems. Deduplication is applied at the namespace level and on all of the file systems comprising the synthetic namespace. All files in a file system in a higher storage tier whose contents are identical to at least one other file in the synthetic namespace are moved to a destination file system in a lower storage tier. For each set of duplicated files that are moved from the original servers, a single instance copy of the file is left behind as a mirror copy. Read access to a duplicated file is redirected to its mirror copy. When the first write to a duplicated file is received, the association from the duplicated file stored in the destination server to its mirror copy that is stored in the origin server is discarded.

Abstract: Surrogate heuristic identification is described, including a memory configured to store data associated with content, and a processor configured to select a portion of the content and the portion is standardized, to identify a characteristic associated with the portion, to use the characteristic to generate a data representation, the data representation being used to provide heuristic data, and to process the heuristic data to generate a fingerprint configured to compare against one or more stored fingerprints.

Abstract: Hashes of metadata of digital photographs on, e.g., a removable camera memory are compared against values in a hash table representing previously stored photographs on an archive computer to ensure that only previously unstored photos are copied onto the archive computer.

Abstract: Example apparatus, methods, and computers participate in collaborative, distributed, data de-duplication. One example method includes initializing a layered parser in a first node in a collaborative distributed data de-duplication (CDDD) topology with a first set of de-duplication control parameters. After transmitting some information to another node in the CDDD topology, the method includes selectively reconfiguring the layered parser in response to feedback acquired from the second node in the CDDD topology. The feedback concerns the data provided by the layered parser.

Abstract: Concepts and technologies are described herein for hash-based file comparison. In accordance with the concepts and technologies disclosed herein, a comparison tool is configured to compare hash values in a first hash list associated with a first file to hash values in a second hash list corresponding to a second file. The hash values are computed for functional blocks of the respective files and therefore can be compared to indicate if the functional blocks of the two files match one another. If the hash values match, the functional blocks associated with the respective hash values can be determined to match, while hash values that do not match can indicate that the corresponding functional blocks do not match one another.

Abstract: A system, method, and computer program product are provided for outputting a signal based on a partial hash comparison. In use, data is identified. In addition, a partial hash is determined utilizing a portion of the data. Further, the partial hash is compared with a plurality of known partial hashes, and an additional hash is conditionally determined based on the comparison. Still yet, a signal is output based on the comparison.

Abstract: Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system.

Abstract: A database search system ensures the privacy of a search request and the security of a database with high processing efficiency. The system comprises a database server residing on a network that stores predetermined data, and a client terminal issuing to the database server a search request for inquiring for desired data while keeping the desired data secret. The database server further comprises: a database storing predetermined data; and a search response message generation unit for receiving through a network the data search request in which data to be searched for is kept secret, searching the database by a predetermined computation based on the search request and a list of the data stored in the database while a data item involved with the search request is kept secret, and generating a response to the search request.

Abstract: A method for validating outsourced processing of a data stream arriving at a streaming data warehouse of a data service provider includes a proof protocol. A verifier acting on behalf of a data owner of the data stream may interact with a prover acting on behalf of the data service provider. The verifier may calculate a first root hash value of a binary tree during single-pass processing of the original data stream with limited computational effort. A second root hash value may be calculated using the proof protocol between the verifier and the prover. The prover is requested to provide certain queried values before receiving random numbers used to generate subsequent responses dependent on the provided values. The proof protocol may be used to validate the data processing performed by the data service provider.

Abstract: A test coverage analysis method and corresponding apparatus are disclosed, wherein, by executing the program under test using one or more test cases, generating one or more heapdump files containing the call stack information of the program under test, and analyzing the call stack information in the one or more heapdump files, the coverage information of the one or more test cases in terms of functions in the program under test is obtained.

Abstract: A memory system includes a storage device storing a plurality of instructions and a central processing unit processing an instruction fetched from the storage device, wherein the central processing unit detects a change in the instruction fetched from the storage device while processing the instruction.

Abstract: Systems and methods including storage systems that employ local file caching processes and that generate state variables to record, for subsequent use, intermediate states of a file hash process. In certain specific examples, there are systems that interrupt a hash process as it processes the data blocks of a file, and stores the current product of the interrupted hash process as a state variable that represents the hash value generated from the data blocks processed prior to the interruption. After interruption, the hash process continues processing the file data blocks. The stored state variables may be organized into a table that associates the state variables with the range of data blocks that were processed to generate the respective state variable. Such exemplary systems can be used with any type of storage system, including filers, database systems or other storage applications.

Abstract: Mechanisms are provided for accelerated data deduplication. A data stream is received an input interface and maintained in memory. Chunk boundaries are detected and chunk fingerprints are calculated using a deduplication accelerator while a processor maintains a state machine. A deduplication dictionary is accessed using a chunk fingerprint to determine if the associated data chunk has previously been written to persistent memory. If the data chunk has previously been written, reference counts may be updated but the data chunk need not be stored again. Otherwise, datastore suitcases, filemaps, and the deduplication dictionary may be updated to reflect storage of the data chunk. Direct memory access (DMA) addresses are provided to directly transfer a chunk to an output interface as needed.

Abstract: Methods and apparatus for on-the-fly identification of sub-asset modifications in a data pipeline are described.

Abstract: A method for detecting and combating an attack in an industrial control system includes sending a command stream from a protection network of an industrial control system to at least one zone, the command stream comprising at least one command; concatenating the at least one command into at least one sequential command package comprising units or work; passing the at least one sequential command package to a crypto hash generator; generating at least one of unit of work hash codes or sequence hash codes; comparing the generated hash codes against a database of existing valid unit of work hash codes and sequence hash codes; and if a command stream fault is detected, generating an alert and accessing a database comprising emergency procedures.

Abstract: A search environment of an embodiment includes name mining and matching features used in part to identify people-centric queries and provide an enriched search experience, but is not so limited. A method of an embodiment operates to provide an expanded query based in part on a geometric similarity measure, an edit distance measure, a string similarity measure, and a cumulative similarity measure. A search system of an embodiment includes a mined candidate generator component and a name matcher component used in part to identify name queries and provide an expanded query that includes original query terms and one or more valid mined names. Other embodiments are also disclosed.

Abstract: A system, method, and computer-readable medium for dynamic detection and management of data skew in parallel join operations are provided. Rows allocated to processing modules involved in a join operation are redistributed among the processing modules by a hash redistribution of the join attributes. Receipt by a processing module of an excessive number of redistributed rows having a skewed value on the join attribute is detected by a processing module which notifies other processing modules of the skewed value. Processing modules then terminate redistribution of rows having a join attribute value matching the skewed value and either store such rows locally or duplicate the rows. The processing module that has received an excessive number of redistributed rows removes rows having a skewed value of the join attribute from a redistribution spool allocated thereto and duplicates the rows to each of the processing modules.

Abstract: In one aspect, a method includes determining locations in a file that have changed using read signature commands, reading data around a changed file location for each file location that has changed, generating variable-size portions of the data read and sending the variable-size portions of the data to a backup system.

Abstract: A method and system for authenticating applications. The system includes a plurality of portable electronic devices, at least one process organizer, at least one module developer, at least one task performer and a workflow server. The process organizer generates an application. The process organizer can request modules from module developers that are incorporated into the application. Alternatively, the process organizer makes an agreement with a task performer to perform tasks to complete the application. The application is transmitted to the workflow server, which includes an online marketplace for displaying and selling the applications. A user purchases the application through the store. A routing module generates a signature and compares the signature to application, module or document metadata and a secret to determine consistency of the data before the application, module or document is routed.

Abstract: Example apparatus and methods concern de-duplication reference tag reconciliation associated with garbage collection and/or reference health checking. One example method may include accessing data associated with members of a set of references to blocks of data stored by a data de-duplication system. The method may process the first data to manipulate a Bloom filter into a state from which membership in the set of references can be assessed. The method may also include accessing a block identifier identified with a member of the set of blocks of data stored by the data de-duplication system and assessing membership in the set of references for the block identifier by querying the Bloom filter with the block identifier. If the block is not referenced, as determined by querying the Bloom filter, then the method may include performing a block reclamation action on the unreferenced block.

Abstract: A system for efficient isolation of backup versions of data objects affected by malicious software includes one or more processors and a memory coupled to the processors. The memory comprises program instructions executable by the processors to implement a backup manager configured to receive an indication that a data object is infected by malicious software. In response to the indication, the backup manager is configured to identify a particular backup version of the data object to be excluded from a data set to be used for a restore operation.

Abstract: According to certain embodiments, a characteristic function representing data is received. A first hash code calculated from the characteristic function is received. The characteristic function is transformed to an arithmetic function. A second hash code is calculated from the arithmetic function. If the first hash code equals the second hash code, the data is designated as valid; otherwise, the data is designated as invalid.

Abstract: Described are computer-based methods and apparatuses, including computer program products, for removing redundant data from a storage system. In one example, a data delineation process delineates data targeted for de-duplication into regions using a plurality of markers. The de-duplication system determines which of these regions should be subject to further de-duplication processing by comparing metadata representing the regions to metadata representing regions of a reference data set. The de-duplication system identifies an area of data that incorporates the regions that should be subject to further de-duplication processing and de-duplicates this area with reference to a corresponding area within the reference data set.

Abstract: A method for data synchronization (DS) is provided, which includes comparing data fingerprints in databases of two parties in need of DS, determining data that needs to be synchronized, and DS is then performed on the data that needs to be synchronized. A system and two devices for DS are further provided. Therefore, DS is implemented through data fingerprints in the technical solutions, thus avoiding transmitting a mass of data between the two parties in need of DS.

Abstract: A system and method for redundant transmission is provided. In one embodiment, an input signal S is encoded as a list of fragments. Each fragment includes an index value and a projection value. The index points to an entry in a dictionary of signal elements. A repetition factor is assigned to each fragment based on its importance. After a fragment is added, a reconstructed signal is generated by decoding the list of fragments. Encoding terminates once the reconstructed signal is sufficiently close to the original signal S.

Abstract: To permit non-TV CE devices to participate in a closed Internet Protocol television (IPTV) program, a non-TV CE device obtains a TV identification upon program registration which it subsequently uses to access content from the IPTV program.

Abstract: Mechanisms are provided for efficient resynchronization of replicated data. A hash value is generated for a chunk of data replicated from a source node to a target node. The chunk of data may be a file deduplicated and compressed at both a source node and a target node. A current sequence number is determined and a sequence number and hash tuple is maintained for the chunk of data at both the source node and the target node. Sequence numbers are modified whenever the data is modified. Current sequence numbers and sequence number and hash values in the sequence number hash tuples at the source node and the target node may be compared to determine whether data is still synchronized at a later point in time or whether data requires resynchronization.

Abstract: Described are embodiments of an invention for identifying chunk boundaries for optimization of fingerprint-based deduplication in a computing environment. Storage objects that are backed up in a computing environment are often compound storage objects which include many individual storage objects. The computing device of the computing environment breaks the storage objects into chunks of data by determining a hash value on a range of data. The computing device creates an artificial chunk boundary when the end of data of the storage object is reached. When an artificial chunk boundary is created for the end of data of a storage object, the computing device stores a pseudo fingerprint for the artificial chunk boundary. If a hash value matches a fingerprint or a pseudo fingerprint, then the computing device determines that the range of data corresponds to a chunk and the computing system defines the chunk boundaries.

Abstract: Embodiments of the present invention include a method and system for verifying intended contents of an electronic message comprising: accessing an electronic message destined for a recipient comprising a link to an intended data source. The method further includes accessing the intended data source and computing a first message digest value associated with the intended data source. Additionally, the method includes sending the electronic message and the first message digest value to the recipient. The recipient side may check the message digest value against a recently generated value and alert the recipient of any digest mismatch that represents a recent change in the intended data source.

Abstract: In one embodiment of the present invention, a data-backup method includes partitioning a fingerprint namespace among a cluster of backup servers, the fingerprint namespace comprising fingerprints for representing units of data, each backup server of the cluster of backup servers managing units of data having fingerprints corresponding to an assigned partition of the fingerprint namespace. The method further includes receiving backup information from a client computing device for a block of data comprising units of data, the backup information including at least a fingerprint for each of the units of data and client-specific backup information. In addition, the method includes, utilizing the fingerprint for each of the units of data, deduplicating the units of data in parallel at the cluster of backup servers in accordance with the partitioning step, the deduplicating step comprising identifying ones of the units data already stored by the cluster of backup servers.

Abstract: A system and method include selecting a file for potential backing up. A metadata signature is calculated on the selected file. The metadata signature of the selected file is compared to a metadata signature of a corresponding file already backed up. The file is only backed up the file if the signatures do not match.

Abstract: Some embodiments of the inventive subject matter are directed to detecting a file that a first instance of an operating system is configured to load where the file is configured to extend functionality of a kernel when loaded. In some embodiments, the first instance of the operating system is managed by a second instance of the operating system (e.g., the first instance of the operating system is a virtual operating system and the second instance is a global operating system). The first and second instances of the operating system share the kernel. Some embodiments are further directed to generating a unique electronic identifier based on properties of the file. The unique electronic identifier uniquely identifies the file. Some embodiments are further directed to providing a copy of the file to the first instance of the operating system, and some time later, receiving a request from the first instances of the operating system to load the copy of the file.

Abstract: In one embodiment, a method includes accessing data; partitioning the data into sub-blocks; determining whether a first one of the sub-blocks is identical to another one of the sub-blocks or similar to another one of the sub-blocks; if the first one of the sub-blocks is identical to another one of the sub-blocks, applying by the one or more computer systems hash-based deduplication to storage of the first one of the sub-blocks with respect to the other one of the sub-blocks; and, if the first one of the sub-blocks is similar to another one of the sub-blocks, applying by the one or more computer systems sub-block differencing to storage of the first one of the sub-blocks with respect to the other one of the sub-blocks.

Abstract: A reference to auxiliary data is steganographically encoded within a media signal. The auxiliary data is stored in a metadata database that maps the reference encoded in the media signal to its corresponding metadata. Various application programs and devices can access the metadata by extracting the reference from the media signal, and querying the database for metadata corresponding to the reference. The metadata database may be implemented in a network server to make metadata readily available to devices and applications distributed throughout a network. This steganographic link to metadata may be used to retrieve metadata across media management systems. In one configuration, for example, media management systems have metadata servers that maintain metadata for a collection of media signals. The metadata server is responsible for responding to requests for metadata associated with media titles in its collection.

Abstract: Methods and systems for generating a hashcode for a hash table are provided to improve the hash table performance. A particular method includes receiving an input of a key; applying a hash function to the key to generate an incoming hashcode of the key; encoding a key type into the incoming hashcode to generate a first hashcode; and storing the first hashcode. A particular method includes encoding the key type into at least one bit of a bucket index portion of the incoming hashcode.

Abstract: An electronic file approval management system includes information terminal devices for a creator creating an electronic file and for an approver approving the file. The device for approver includes an electronic information transmitter-receiver storing and retrieving information with the device for creator, an input unit being input an approval result, a first print data converter converting the file into first print data, a first approval information extractor extracting first approval information, and a key creation unit creating an approval key.

Abstract: A method is provided in one example embodiment that includes identifying a file format identifier associated with a beginning of a file, parsing the file based on the file format identifier until an end of the file is identified, and calculating a hash from the beginning of the file to the end of the file. The method may also include sending the hash to a reputation system and taking a policy action based on the hash's reputation received from the reputation system.

Abstract: A communication control device includes a plurality of receive buffers each storing therein received information that corresponds to all or a part of a received message or an argument of a receive function, a hash-value generating unit that generates a hash value from a receive key contained in the received message in accordance with a hash-value generation rule, a storing unit that stores the received information in a selected one of the receive buffers corresponding to the hash value, and an output unit that outputs the received information from one of the receive buffers corresponding to the hash value in response to a transmission request from a receiving unit that performs a receiving operation by determining a matching based on a receive key specified by the receive function.

Abstract: A multi-dimensional perfect hash table construction technique is based on which the well-known AC automaton, and can be implemented by very compact perfect hash tables. The technique may place transitions, each from a source state to a destination state, of an automaton into a hash table to generate a perfect hash table by: (a) dividing the transitions into multiple independent sets according to their respective source states; (b) ordering the sets of transitions based on the number of transitions belonging to the set, thereby defining an order of the sets from largest to smallest; and (c) constructing a perfect hash table by, for each of the sets of transitions, in the order from largest to smallest, hashing the transitions of the set into the hashing table to generate a perfect hashing table.

Abstract: Data deduplication compression in a streaming storage application, is provided. The disclosed deduplication process provides a deduplication archive that enables storage of the archive to, and extraction from, a streaming storage medium. One implementation involves compressing fully sequential data stored in a data repository to a sequential streaming storage, by: splitting fully sequential data into data blocks; hashing content of each data block and comparing each hash to an in-memory lookup table for a match, the in-memory lookup table storing all hashes that have been encountered during the compression of the fully sequential data; for each data block without a hash match, adding the data block as a new data block for compression of fully sequential data; and encoding duplicate data blocks using the in-memory lookup table into data segments.

Abstract: A mechanism is provided for security screening image analysis simplification through object pattern identification. Popular consumer electronics and other items are scanned in a control system, which creates an electronic signature for each known object. The system may reduce the signature to a hash value and place each signature for each known object in a “known good” storage set. For example, popular mobile phones, laptop computers, digital cameras, and the like may be scanned for the known good signature database. At the time of scan, such as at an airport, objects in a bag may be rotated to a common axis alignment and transformed to the same signature or hash value to match against the known good signature database. If an item matches, the scanning system marks it as a known safe object.

Abstract: Techniques that assist in processing of failure detection protocol (FDP) packets. Techniques are provided that assist a CPU of a network device in processing incoming FDP packets. In one embodiment, only a subset of FDP packets received by the network device is forwarded to the CPU for processing, the other FDP packets are dropped and not forwarded to the CPU. In this manner, the amount of processing that a CPU of the network device has to perform for incoming FDP packets is reduced. This enables the network device to support newer FDPs with shorter periodic interval requirements.

Abstract: When accepting a write request including a data, an apparatus 100A acquires a first hash value based on a first hash function and, meanwhile, acquires a second hash value based on a second hash function. When a storage device 110A has not stored the acquired first hash value and second hash value in correlation with each other, the apparatus correlates the data, the first hash value, the second hash value, and information of referenced times, and then stores the correlated items into the storage device. On the other hand, when the storage device has stored the acquired first hash value and second hash value in correlation with each other, the apparatus changes the information of referenced times stored in correlation with the first hash value and the second hash value so as to add one to the number of times denoted by the information of referenced times.