Abstract: Techniques are provided for hardware-accelerated relational joins. A first table comprising one or more rows is processed through a hardware accelerator. At least one join column in at least one of the one or more rows of the first table is hashed to set at least one bit in at least one bit vector. A second table comprising one or more rows is processed through a hardware accelerator. At least one join column in at least one of the one or more rows of the second table is hashed to generate at least one hash value. At least one bit vector is probed using the at least one hash value. A joined row is constructed responsive to the probing step. The row-construction step is performed in the hardware accelerator.

Abstract: In one embodiment, a method includes receiving a bit value from a key vector. The bit value has a position within the key vector correlating to a counter value. The method includes receiving a sequence of bit values from a plurality of predefined bit values based on the counter value and in response to a condition associated with the bit value from the key vector being satisfied.

Abstract: An administrator system can generate a version identifier for a computing system based on the software packages installed on a computing system. The version identifier can represent the software packages installed on the computing system. The administrator system can identify the software packages installed on a computing system and data specific to the software packages. The administrator system can apply a hash function to the specific data to generate hashed data. Then, the administrator system can concentrate the hashed data to generate the version identifier.

Abstract: A method and apparatus for logging based identification are described. In one embodiment, the method comprises extracting entries of a hash chained log that represents a series of previous transactions. The method may also comprise ordering hash values of the entries extracted from the hash chained log into an ordered list. In one embodiment, the method may further comprise producing a cryptographic hash of the ordered list.

Abstract: Systems and methods facilitating random number generation of hashes for video and/or audio are provided. In one embodiment, a system can include: a memory, and a microprocessor that executes computer executable components. The components can include a weighted distribution generation component that can generate a sampling distribution of a weighted combination of uniform distributions, and obtain a sample value from the sampling distribution. In one embodiment, horizontal regions of substantially equal area can be identified. The sample value can be obtained by selecting one of the horizontal regions, and uniformly selecting a coordinate from the horizontal region. The coordinate can correspond to a value on a horizontal axis of the sampling distribution, and the value can be equal to a sample value. The sample value can be employed to compute a hash employed in video and/or audio fingerprinting and/or in computing image descriptors for video.

Abstract: A representation of a new rule, defined as a set of a new transition(s), is inserted into a perfect hash table which includes previously placed transitions to generate an updated perfect hash table. This may be done by, for each new transition: (a) hashing the new transition; and (b) if there is no conflict, inserting the hashed new transition into the table. If, however, the hashed new transition conflicts with any of the previously placed transitions, either (A) any transitions of the state associated with the conflicting transition are removed from the table, the hashed new transition is placed into the table, and the removed transitions are re-placed into the table, or (B) any previously placed transitions of the state associated with the new transition are removed, and the transitions of the state associated with the new transition are re-placed into the table.

Abstract: Segment sizes are controlled by setting the size of a segment boundary in a hash-based deduplication system. A subsequence of size K of a sequence of characters S is set. An increasing sequence of n probabilities and a corresponding sequence of n decreasingly restrictive logical tests are chosen to be applied on the sequence of characters S. Segment boundaries are set by using the sequence of the decreasingly restrictive logical tests by deciding to declare a segment boundary at a current position if one of the sequence of the decreasingly restrictive logical tests, with a corresponding probability of the sequence of n probabilities, returns a true value when applied on the sequence of characters S.

Abstract: A transformation method and system is provided. The method includes generating a data hub application configured to embed extract, transform, and load (ETL) processes. The data hub application is linked to source tables and target tables. Meta data associated with the source and target tables is transferred from virtual views of the data hub application to an ETL work area of the ETL processes. An ETL job is generated and linked to the data hub application. ETL processes are executed and results are determined.

Abstract: A computer implemented method is provided to verify an integrity of a solution. The computer implemented method comprises hashing, by a computer, a set of virtual machine instances in a solution topology of the solution. The computer hashes a set of connections in the solution topology. The set of connections comprising a connection between ones of the set of virtual machine instances, a connection between a first component of a first one of the set of virtual machine instances and a second component of a second one of the set of virtual machine instances, and combinations thereof. The computer hashes a set of solution-specific information, and then signs the hashes to create a first signed topology.

Abstract: A method and system for automatically rotating image files and preventing duplicate uploads in a remote file storage system is described. The system receives an image file. The system calculates a hash value associated with newly uploaded files. The system then automatically rotates the images based on orientation data associated with the files (if included). The system then compares the hash value(s) associated with the new file(s) with the hash values from the existing files stored on the remote file storage system. If the hash values of any of the new files to be uploaded match a hash value on the hash value list, then the system prevents the new files from being uploaded to the remote file storage system.

Abstract: A system for storing compressed data comprises a processor and a memory. The processor is configured to receive a compressed segment. The compressed segment is determined by breaking a data stream, a data block, or a data file into one or more segments and compressing each of the one or more segments. The processor is further configured to determine whether the compressed segment has been previously stored, and in the event that the compressed segment has not been previously stored, store the compressed segment. The memory is coupled to the processor and configured to provide the processor with instructions.

Abstract: Digital objects within a fixed-content storage cluster use a page mapping table and a hash-to-UID table to store a representation of each object. For each object stored within the cluster, a record in the hash-to-UID table stores the object's hash value and its unique identifier (or portions thereof). To detect a duplicate of an object, a portion of its hash value is used as a key into the page mapping table. The page mapping table indicates a node holding a hash-to-UID table indicating currently stored objects in a particular page range. Finding the same hash value but with a different unique identifier in the table indicates that a duplicate of an object exists. Portions of the hash value and unique identifier may be used in the hash-to-UID table. Unneeded duplicate objects are deleted by copying their metadata to a manifest and then redirecting their unique identifiers to point at the manifest.

Abstract: Systems and methods for facilitating combining media fingerprinters are provided. In one implementation, a system can include a memory and a microprocessor. The system can also include: a communication component that receives media; a wavelet-based hash generation component that generates a plurality of hashes based on wavelet descriptors associated with the media; a local features information generation component that generates local descriptors associated with the media; a grouping component that groups the plurality of hashes into two or more subgroups, and generates a combined group of descriptors. The combined group of descriptors can include quantized local descriptors and groups of hashes. The system can also include a hash generation component that hashes the combined group of descriptors.

Abstract: An improved computer system may include a controller including a computer processor. The system may also include a selector apparatus in communication with the controller to choose a table having a higher collision quality index than other tables under consideration by the selector apparatus. The system may further include an exchanger apparatus to configure a standby table that replaces the table chosen by the selector apparatus. The system may additionally include a switch that changes a hash function based upon the exchanger apparatus? replacement of the chosen table to enable the controller to reduce insertion times and/or collisions when interfacing with new components introduced to the controller.

Abstract: Storing data from mobile devices may comprise receiving an asynchronous communication from a mobile device over a wireless network notifying that data was dropped in a synchronizing folder; computing a hash value associated with the data; determining based on the computed hash value, a partition of an object grid comprising a plurality of virtual machines; and storing the data on the partition.

Abstract: Storing data from mobile devices may comprise receiving an asynchronous communication from a mobile device over a wireless network notifying that data was dropped in a synchronizing folder; computing a hash value associated with the data; determining based on the computed hash value, a partition of an object grid comprising a plurality of virtual machines; and storing the data on the partition.

Abstract: Systems and methods enabling search of a repository for the location of data that is similar to input data, using a defined measure of similarity, in a time that is independent of the size of the repository and linear in a size of the input data, and a space that is proportional to a small fraction of the size of the repository. Additionally, remote operations are accomplished with significantly reduced system bandwidth by implementing remote differencing operations.

Abstract: A method to sequentially deduplicate data, wherein the method receives a plurality of computer files, wherein each of the plurality of computer files comprises a label comprising a file name, a file type, a version number, and file size, and stores that plurality of computer files in a deduplication queue. The method then identifies a subset of the plurality of computer files, wherein each file of the subset comprises the same file name but a different version number, and wherein the subset comprises a maximum count of version numbers, and wherein the subset comprises a portion of the plurality of computer files. The method deduplicates the subset using a hash algorithm, and removes the subset from said deduplication queue. During the deduplicating, the method receives new computer files comprising the same file name, stores those new computer files to the deduplication queue, but does not add those new computer files to the subset.

Abstract: Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system.

Abstract: A system and method that accepts material from the Internet, computes and maintains a computationally unique representation of the received material in a database, computes a computationally unique representation of the updated database state as a whole, publishes material and database representations in various forms for the global Internet audience to witness, and returns a time stamped attestation to the submitter of material as proof of the material being witnessed on the Internet along with unique database state and revenue generating advertisements. Published representations may include pictures, common words, or hexadecimal character string to facilitate Internet searching mechanisms.

Abstract: An improved scalable object storage system allows multiple clusters to work together. Users working with a first cluster, or with a multi-cluster gateway, can ask for services and have the request or data transparently proxied to a second cluster. This gives transparent cross-cluster replication, as well as multi-cluster compute or storage farms based upon spot availability or various provisioning policies. Vendors providing a cloud storage “frontend” can provide multiple backends simultaneously. In one embodiment, a multi-cluster gateway can have a two, three, or higher-level ring that transparently matches an incoming request with the correct cluster. In the ring, a request is first mapped to an abstract “partition” based on a consistent hash function, and then one or more constrained mappings map the partition number to an actual resource. In another embodiment, the multi-cluster gateway is a dumb gateway, and the rings are located only at the cluster level.

Abstract: An apparatus having a circuit is disclosed. The circuit may be configured to (i) generate a sequence of hash values in a table from a stream of data values with repetitive values, (ii) find two consecutive ones of the hash values in the sequence that have a common value and (iii) create a shortened hash chain by generating a pointer in the table at an intermediate location that corresponds to a second of the two consecutive hash values. The pointer generally points forward in the table to an end location that corresponds to a last of the data values in a run of the data values.

Abstract: Methods and systems are provided for managing records in a database. An exemplary method involves a server coupled to the database determining a value indicative of a relative uniqueness of an input record based on a subset of a plurality of fields of the input record and storing the input record in the database when that value is unique among one or more values for one or more existing records in the database. Each value of the one or more values for the existing records is also based on the subset of the plurality of fields of a respective existing record and is indicative of the relative uniqueness of that respective existing record.

Abstract: Exemplary embodiments provide various techniques for detecting and correcting corrupted or dangling data object handles. In one example, a method is provided for correcting corrupted data object handles. In this method, a redirector file stored on a first logical data container is accessed. This redirector file includes a forward object handle pointing to a first data object stored on a second logical data container and includes a first checksum of the forward object handle. A second checksum of the forward object handle is determined and a comparison is made between the first and second checksums. If a mismatch is detected, the data objects stored on the second logical data container are scanned to extract backward object handles. A backward object handle can be identified to point back to the redirector file. The forward object handle can then be restored to point to the second data object based on the identification.

Abstract: A storage device, a directory entry lookup method for the storage device, and a host running the method can provide a quick directory entry lookup. The host includes an interface for exchanging data with the storage device which stores a multi-level hash table comprising directory entries of each directory, and a file system module receiving a file lookup command designating a target directory and a target filename, calculating a hash value which reflects the target filename and a lookup level, and searching for a directory entry which comprises the target filename in a bucket corresponding to the hash value from among buckets at the lookup level which are included in a multi-level hash table of the target directory. If the search for the directory entry fails, the file system module increases the lookup level and again calculates the hash value and searches for the directory entry for the target filename.

Abstract: A system for identifying and notifying computers of malicious software programs over an Internet.

Abstract: When an index is partitioned and managed in a plurality of indexes in a search system which deals with large amounts of data, the number of documents registered in each partitioned index may be varied, and inefficient data transfer may occur at the time of index reconfiguration when search nodes are added or deleted. In the present invention, mapping information is used to map search node IDs to slot regions obtained by equally partitioning a circular hash value space in which a start point and an end point of the hash value space are assumed to be identical, so that each document is mapped to each partitioned index.

Abstract: At the first data access by a navigation unit to a recording medium that records updating right information necessary for updating map data in a rewritable data area in which map data are recorded, the updating right information is read from the data area and is deleted from the data area, and a map updating due date created based on the read updating right information is written in a memory of the navigation unit together with the medium identification information read from a non-rewritable management area.

Abstract: An encrypted file is decrypted to gain access to a stored hash value for a credentials setting component. A test hash value of the credentials setting component is formed. Before decrypting a set of encrypted credentials to form decrypted credentials, it is required that the test hash value of the credentials setting component match the stored hash value of the credentials setting component. The decrypted credentials are then passed to the credentials setting component to set credentials that instructions are to be executed under.

Abstract: A data management system respectively computes first hash values while sliding a window a prescribed amount at a time with respect to a prescribed range from a start location of a data block to a prescribed size. The system extracts, from among the first hash values, a first hash value, which is equivalent to a characteristic value, and partitions the data block into a first chunk of data at a location corresponding to this first hash value. The system determines coincidence between a first chunk of data and a stored second chunk of data, and prevents duplicate data from being stored twice.

Abstract: Embodiments of the present invention provide a system and computer program product for test data generation using unique common factor sequencing. In an embodiment of the invention, a computer program product for test data generation using unique common factor sequencing is provided. The computer program product includes loading a table for population with test data in a test data generation tool executing in memory of a computer. A column set of multiple columns in the table associated with a key to the table can be selected for processing and different cardinality sequence values are assigned to the columns in the set such that the cardinality sequence values do not share a common factor except for unity as in the case of prime numbers.

Abstract: An input lookup key is hashed and the hashed key divided into stride bits into a multi-level Trie structure. A compression function logically combines the stride bits to generate the compressed index bits into the stride tables. The bucket in the last stride table found by the hashed key may have several keys that collide at the same hash value. Discriminant bits are read from the key and select a stored key in the bucket table for verification of its result. Since the hashed key is a compression of the longer input key, more information is contained per bit of the hashed key than in the long key. The multi-stride lookup is performed first on the hashed key, allowing a faster convergence to the lookup result. The first stride can index a single hash table, with the remaining hash bits and discriminant bits used to select from among colliding keys.

Abstract: A highly reliable data archival and retrieval method that enables fine grained control over data availability is implemented across a Quality of Service driven archival system, configured to fragment the data into data and parity chunks for storing onto the storage node. The technique employed by the archival system enables files to be read without having need to access any metadata, thereby tolerating complete loss of such metadata. Further, the Quality of Service driven system architecture improves upon the system performance and throughput by means of a storage node regeneration process which ensures balanced load on participating storage node during various storage, retrieval and regeneration operations.

Abstract: Dynamic blocking determines which pairs of records in a data set should be examined as potential duplicates. Records are grouped together into blocks by shared properties that are indicators of duplication. Blocks that are too large to be efficiently processed are further subdivided by other properties chosen in a data-driven way. We demonstrate the viability of this algorithm for large data sets. We have scaled this system up to work on billions of records on an 80 node Hadoop cluster.

Abstract: In some example, a computerized method includes receiving a first electronic document and a second electronic document. The method also includes determining a difference between the first electronic document and the second electronic document based on matching of a component of the first electronic document to a component of the second electronic document in a hierarchical order. The method includes storing the difference in a machine-readable medium.

Abstract: A method for efficiently computing a hash value for a string is disclosed. In one embodiment, such a method includes receiving an original string comprising multiple characters. The method computes an original hash value for the original string. The method produces an updated string by performing at least one of the following updates on the original string: adding leading/trailing characters to the original string; removing leading/trailing characters from the original string, and modifying characters of the original string while preserving the length of the original string. The method then computes an updated hash value for the updated string by performing at least one operation on the original hash value, wherein the at least one operation takes into account the updates that were made to the original string. A corresponding apparatus and computer program product are also disclosed.

Abstract: Systems and methods enabling parallel processing of hash functions are provided. A data string including a plurality of pieces arranged in an order is hashed using a hash function to determine a plurality of authentication checkpoint hashes associated with the pieces. To authenticate the data string, the pieces are grouped into sets, and the authentication checkpoint hash associated with the piece following all other pieces of that set in the order is associated with that set. The system simultaneously performs a separate hash process on each set. That is, the system hashes the pieces of that set using the hash function to determine a result hash, and compares that result hash with the authentication checkpoint hash associated with that set. The initial input to the hash function for the hash process for each set includes one of the pieces and either a default seed or an authentication checkpoint hash.

Abstract: One or more techniques and/or systems are provided for performing host side deduplication. Host side deduplication may be performed upon writeable data within a write request received at a host computing device configured to access data stored by a storage server. The host side deduplication may be performed at the host computing device to determine whether the writeable data is already stored by the storage server based upon querying a host side cache comprising data stored by a storage server and/or a data structure comprising unique signatures of data stored by the storage server. If the writeable data is stored by the storage server, then a deduplication notification excluding the writeable data may be sent to the storage server, otherwise a write command comprising the writeable data may be sent. Accordingly, unnecessary network traffic of redundant data already stored by the storage server may be reduced.

Abstract: In association with a data processing system that includes one or more servers, one or more clients and a partitionable distributed directory contained in a database, a computer implemented method is provided for selectively processing data entries that reside in the directory. The method comprises the steps of generating a request to perform an operation on each data entry in a specified group of intended entries, and specifying a hashing control index that uniquely identifies each entry of the specified group, and excludes all other entries. The requested operation is applied only to data entries in the directory that are identified by the specified hashing control index.

Abstract: A network packet includes a packet key that includes one or more source-destination field pairs that each include a source field and a destination field. For each selected source-destination field pair, first and second sections are selected in the packet key. A source field value is extracted from the source field and a destination field value is extracted from the destination field. For each source bit of the source field value: a destination bit is selected from the destination field; an OR logic function is applied to the source bit and the destination bit to generate a first resulting value is stored at the same bit position as the source bit in the first section; an AND logic function is applied to the source bit and the destination bit to generate a second resulting value stored at the same bit position as the source bit in the second section.

Abstract: According to one embodiment, an alteration detecting apparatus includes an input unit, a storage unit, an output unit, and an alteration detecting unit. The input unit inputs a file. The storage unit stores the file. The output unit outputs the file. The alteration detecting unit produces first alteration detection data that is uniquely determined, from the file on the basis of an alteration detection data production process in response to an input of the file, stores the file and the first alteration detection data in the storage unit, produces second alteration detection data that is uniquely determined, from the file stored in the storage unit on the basis of the alteration detection data production process in response to an output request for the file, compares the first alteration detection data with the second alteration detection data and detects alteration of the file on the basis of the compared result.

Abstract: Embodiments are directed to, for a database comprising a set of transactions, with each transaction including an itemset comprising one or more individual items, obtaining a set of all closed frequent itemsets by checking frequencies of transaction sets, wherein a given itemset, I, is frequent where at least some threshold number (?) of transactions contain I, and wherein the given itemset I is closed wherein it is impossible to add any item to I without changing the frequency thereof.

Abstract: Embodiments are directed to, for a database comprising a set of transactions, with each transaction including an itemset comprising one or more individual items, obtaining a set of all closed frequent itemsets by checking frequencies of transaction sets, wherein a given itemset, I, is frequent where at least some threshold number (?) of transactions contain I, and wherein the given itemset I is closed wherein it is impossible to add any item to I without changing the frequency thereof.

Abstract: Embodiments of the invention include systems and methods for providing block-level verification of replicated file systems. Embodiments operate in context of data storage environments, which may typically have multiple file systems, snapshots of file systems, and replicas of file systems. In one illustrative scenario, a replica is created of a file system having multiple associated snapshots, and a user desires to verify the accuracy of the replica. A signature is created for each of the source active file system and the target replica file system, so that each signature includes records of both block-level signatures and block-level allocations. The signatures are compared to discover any differences. The differences may then be reconciled, where possible, to determine whether the differences indicate a corrupt or otherwise invalid replica.

Abstract: An improved computer system may include a controller including a computer processor. The system may also include a selector apparatus in communication with the controller to choose a table having a higher collision quality index than other tables under consideration by the selector apparatus. The system may further include an exchanger apparatus to configure a standby table that replaces the table chosen by the selector apparatus. The system may additionally include a switch that changes a hash function based upon the exchanger apparatus' replacement of the chosen table to enable the controller to reduce insertion times and/or collisions when interfacing with new components introduced to the controller.

Abstract: In one embodiment, a computer-implemented method comprises receiving a query in a controller. A data store stores data in a relational database. The database comprises a plurality of fields and a plurality of data elements. Each field is associated with a portion of the data elements. The database is arranged as at least one table, and the query identifies data of an object to be stored in a data set. The method further comprises determining, by the controller, whether the relational database includes a field associated with a class of the object; and if the relational database does not include a field associated with the class of the object, adding a column to the relational database, the added column having a field corresponding to the class. The method further comprises adding the data of the object to the relational database.

Abstract: The invention relates to a method and a device for checking a communication system (3) comprising a plurality of modules (7) adapted to be installed in an aircraft under development (5), said checking device including: means (11) for building an identification and synchronization database (17) for said communication system (3), said database contractually defining interfaces between said plurality of modules from change notes relative to an initial technical definition, means (11) for defining, in said database (17), signals configured to be exchanged between said plurality of modules (7) via a plurality of connections (3) interconnecting said interfaces, said signals being defined to be synchronized with each other as well as with the physical materialization of said connections, and means (11) for checking, before an evaluation of a maturation test of the communication system (3), an interface consistency for all of said signals of said database (17).

Abstract: A fingerprint is computed from selected metadata of binary files, and used in determining whether to regenerate code-behind file(s). In one case, after ascertaining that a binary-file-set's current metadata-fingerprint differs from a previous metadata-fingerprint, the corresponding code-behind-file-set is regenerated. A code-behind file may also be regenerated, even though its current fingerprint matches its previous fingerprint, because its metadata has changed. If the current fingerprint matches the previous fingerprint and metadata is unchanged, regeneration can be avoided. The fingerprint may consist of, or merely contain, definitions of classes, properties, and other items, and may be limited to metadata that is used when generating the code-behind-file-set. System assemblies are not fingerprinted. Timestamps may be used. Metadata-fingerprint computation code and regeneration determination code are described.

Abstract: A method begins by a processing module generating an integrity check value for each encoded data slice of a set of encoded data slices to produce a set of integrity check values. The method continues with the processing module encoding the set of integrity check values to produce encoded integrity check values. The method continues with the processing module sending the encoded integrity check values for storage in a memory system.

Abstract: A transformation method and system is provided. The method includes generating a data hub application configured to embed extract, transform, and load (ETL) processes. The data hub application is linked to source tables and target tables. Meta data associated with the source and target tables is transferred from virtual views of the data hub application to an ETL work area of the ETL processes. An ETL job is generated and linked to the data hub application. ETL processes are executed and results are determined.