Abstract: Methods for creation of a content association system in which a client application communicates to a distributed hash table network an association between one piece of content in a client's library and one other piece of content in the client's library are described. The client creates a hash for a first file in the library and a reference to a second file in the library. The client stores the reference to the second file in the distributed hash table at a node corresponding to the hash. To discover files associated with a file in a peer's library, the peer creates a hash for a file in its library and retrieves references to other files from the distributed hash table.

Abstract: A system and method for space and time efficient bound calculation is disclosed. The method comprises inserting a plurality of key/value pairs into a “Bloom bounder”, each key/value pair comprising a key and a value. For each pair, the inserting includes calculating a plurality of hash values, each calculated by applying a different one of a plurality of hash functions to the key, and selectively updating one or more data arrays based on the plurality of hash values and the value received key/value pair. A bound may then be determined for a given query key by analyzing information in the one or more data arrays to determine a bound value, such that for every received key/value pair with a key matching the query key, the corresponding value is less than or equal to the bound value.

Abstract: A system, method, and computer-readable medium for optimized processing of queries that feature maximum or minimum equality conditions are provided. The disclosed mechanisms provide for a single-scan of the table on which the group-by query is applied. When the table is scanned, each processing module dynamically keeps track of the row(s) having a value of the attribute on which the equality condition is applied that equals or exceeds the maximum attribute value (assuming a maximum equality condition is applied) previously encountered by the processing module. Subsequently, a global aggregation process is then performed to compute the query's result without rescanning the table. Queries featuring a minimum equality condition are similarly processed in accordance with the disclosed embodiments.

Abstract: Some embodiments provide a system that facilitates concurrency control in a computer system. During operation, the system generates a set of signatures associated with memory accesses in the computer system. To generate the signatures, the system creates a set of hierarchical Bloom filters (HBFs) corresponding to the signatures, and populates the HBFs using addresses associated with the memory accesses. Next, the system compares the HBFs to detect a potential conflict associated with the memory accesses. Finally, the system manages concurrent execution in the computer system based on the detected potential conflict.

Abstract: Aspects of a method and system for hash table based routing via prefix transformation are provided. Aspects of the invention may enable translating one or more network addresses as a coefficient set of a polynomial, and routing data in a network based on a quotient and a remainder derived from the coefficient set. In this regard, the quotient and the remainder may be calculated via modulo 2 division of the polynomial by a primitive generator polynomial. In one example, the remainder may be calculated with the aid of a remainder table. The primitive generator polynomial may be x16+x8+x6+x5+x4+x2+1. Additionally, entries in one or more hash tables may comprise a calculated quotient and may be indexed by a calculated remainder. In this manner, the hash tables may be accessed to determine a longest prefix match for the one or more network addresses. The hash tables may comprise 2deg(g(x)) sets, where deg(g(x)) is the degree of the primitive generator polynomial.

Abstract: A DAS system that implements RAID technology is provided in which an array of solid state disks (SSDs) that is external to the DAS controllers of the DAS system is used by the DAS controllers as WB cache memory for performing WB caching operations. Using the external SSD array as WB cache memory allows the DAS system to be fully cache coherent without significantly increasing the complexity of the DAS system and without increasing the amount of bandwidth that is utilized for performing caching operations. In addition, using the external SSD array as WB cache memory obviates the need to mirror DAS controllers.

Abstract: A semiconductor integrated circuit includes: a data path configured to transmit data from a first region to a second region; a first hash value calculator configured to read first data being transmitted through the data path within the first region and to calculate a first hash value from the first data; a register being disposed on the data path within the second region and configured to read second data transmitted through the data path; a second hash value calculator configured to read the second data output from the register and to calculate a second hash value from the second data; and a comparator configured to compare the first hash value and the second hash value and to determine whether the first hash value and the second hash value coincide with each other.

Abstract: Several deterrence mechanisms suitable for content distribution networks (CDN) are provided. These include a hash-based request routing scheme and a site allocation scheme. The hash-based request routing scheme provides a way to distinguish legitimate requests from bogus requests. Using this mechanism, an attacker is required to generate O(n2)amount of traffic to victimize a CDN-hosted site when the site content is served from n CDN caches. Without these modifications, the attacker must generate only O(n) traffic to bring down the site. The site allocation scheme provides sufficient isolation among CDN-hosted Web sites to prevent an attack on one Web site from making other sites unavailable. Using an allocation strategy based on binary codes, it can be guaranteed that a successful attack on any individual Web site that disables its assigned servers, does not also bring down other Web sites hosted by the CDN.

Abstract: Various systems and methods for implementing a Galois-based incremental hash module are disclosed. For example, a method involves computing a first hash of a first string of an input stream. The first hash is computed by performing one or more Galois mathematical operations upon portions of the first string. A second hash of a second string, which overlaps the first string, can then be computed by processing the first hash.

Abstract: A directory name lookup cache (DNLC) provides a hashed forward mapping for finding the “child handle” associated with a “parent handle” and a “child name.” To provide an efficient reverse lookup capability, a second set of links is added to each cache entry for a “child hash list” indexed by a hashing of the child handle. For dynamically enabling and disabling the reverse mapping, when a new cache entry is added to its parent hash list, if the reverse mapping is enabled, then the new cache entry is also added to its child hash list; otherwise, the new cache entry is marked to indicate that it is not in any child hash list. To save memory, the parent hash lists and the child hash lists may share hash buckets.

Abstract: An embodiment of a method of hashing an address space to a plurality of storage servers begins with a first step of dividing the address space by a number of the storage servers to form data segments. Each data segment comprises a base address. A second step assigns the data segments to the storage servers according to a sequence. The method continues with a third step of measuring a load on each of the storage servers. According to an embodiment, the method concludes with a fourth step of adjusting data shares assigned to the storage servers according to the sequence to approximately balances the loads on the storage servers while maintaining the base address for each data segment on an originally assigned storage server. According to another embodiment, the method periodically performs the third and fourth steps to maintain an approximately balanced load on the storage servers.

Abstract: A computing environment maintains the confidentiality of data stored in system memory. The computing environment has an encryption circuit in communication with a CPU. The system memory is also in communication with the encryption circuit. An address bus having a plurality of address lines forms part of the system and a value of at least one of the address lines determines a key selected from a plurality of keys to use in the encryption circuit to encrypt data being transferred by the CPU to the memory.

Abstract: The present invention relates to a data storage method, device and system and a management server. The data storage method includes: constituting a data pool from all of n data storage devices; when there is data for storage, polling all the devices in the data pool to select a group of m devices, and storing the data onto each of the selected group of m devices, where m is larger than one and smaller than n. The embodiments of the invention can address the problems of an existing data storage approach that a failing node causes an increased load on and instability of another node and that each node in the existing data storage approach has a low utilization ratio and poor predictability, so as to achieve uniform loads on the devices and high reliability of the nodes despite any failing node and improve the resource utilization ratio and predictability of the nodes.

Abstract: A method and system for dynamically determining hash values for file transfer integrity validation. In response to a request for a transfer of a data file between a first computing system and a second computing system, the first computing system loads a first portion of the data file to a buffer. The first computing system determines a first hash function value based on the first portion. The first computing system loads a second portion of the data file portion to the buffer and determines a second hash function value incrementally based on the first and second data file portions. The first and second data file portions are non-overlapping portions of the data file being transferred.

Abstract: A serial decoding technique may employ one or more circular shift register strings in which an input to an element of a shift register string may be gated by either an address input or the inverse of the address input. An output word line of the decoder may be driven by a respective shift register stage in the case of a single shift register string, or by a logical combination of shift register stages from respective shift register strings in the case of multiple shift register strings.

Abstract: Searching of objects captured by a capture system can be improved by eliminating irrelevant objects from a query. In one embodiment, the present invention includes receiving such a query for objects captured by a capture system, the query including at least one search term. This search term is then hashed to a term bit position using a hash function. Then objects can be eliminated if, in a word index associated with the object, the term bit position is not set.

Abstract: Data associated with the services in a service oriented architecture are stored in a primary repository and replicated across secondary repositories. Functionality can be implemented to efficiently synchronize data across the primary repository and the secondary repositories. Data synchronization can comprise calculating and comparing hash values of one or more nodes, based in part on concatenated hash values of child nodes and data that comprise the one or more nodes, of a tree structure representing data stored in the repositories.

Abstract: Aspects for achieving efficient data access to data elements in a relational database management system are provided. In a computer-implemented method aspect, the efficient data access occurs by establishing a hash table for data elements of a database in a predetermined continuous space of allocated storage, and optimizing utilization of the hash table during database query operations through linear hashing, wherein extension of the hash table occurs automatically to increase a number of pages in the hash table without discernible interruptions of data access to the data elements.

Abstract: A method and system to perform exact match searches for fixed- or variable-length keys stored in a search database. The method is implemented using a plurality of hash tables, each indexed using an independent hash function. A system implementing this method provides deterministic search time, independent of the number of keys in the search database. The method permits two basic implementations; one which minimizes memory storage, and another which minimizes search time. The latter requires only two memory accesses to locate a key.

Abstract: Methods and apparatus provide a lock resizer for resizing of a lock array of a lock-based concurrent hash table. The lock resizer provides a data structure with memory locations which is apportioned into buckets that contain a plurality of the memory locations. It is understood that the data structure can dynamically add new memory locations. The lock resizer further provides a location lock for each distinct memory location and a bucket lock for each distinct bucket. A resizing flag can reference a thread to indicate whether or not the thread is resizing the amount of locks. Upon detection of the existence of a policy condition, the lock resizer resizes the amount of location locks and/or bucket locks in order to create new location locks and new bucket locks, thereby ensuring that as new memory locations are added, all buckets can contain up to a fixed number of memory locations.

Abstract: A caching technique involves receiving a cache request to move data into a cache (or a particular cache level of a cache hierarchy), and generating a comparison between content of the data and content of other data already stored within the cache. The caching technique further involves providing a caching response based on the comparison between the content of the data and the content of the other data already stored within the cache. The caching response includes refraining from moving the data into the cache when the comparison indicates that the content of the data is already stored within the cache. The caching response includes moving the data into the cache when the comparison indicates that the content of the data is not already stored within the cache. Such a technique is capable of eliminating data redundancies within a cache (or within a particular cache level of a cache hierarchy).

Abstract: A page table mechanism translates virtual addresses to real addresses. In a first aspect, page table entries are contained in equal-sized blocks, the entries within each block corresponding to contiguous pages of virtual address space. Preferably, the common high-order portion of the virtual address is contained in segments distributed among multiple page table entries of the block. In a second aspect, the virtual address indexes a binary tree definitional structure. Decode logic traverses a binary tree defined by the definitional structure by testing selective bits of the virtual address to reach a leaf of the binary tree, which defines the location of data defining the real address.

Abstract: One embodiment retrieves a first portion of a plurality of stored objects from at least one storage device. The embodiment further performs a base type deduplication estimation process on the first portion of stored objects. The embodiment still further categorizes the first portion of the plurality of stored objects into deduplication sets based on a deduplication relationship of each object of the plurality of stored objects with each of the estimated first plurality of deduplication chunk portions. The embodiment further combines deduplication sets into broad classes based on deduplication characteristics of the objects in the deduplication sets. The embodiment still further classifies a second portion of the plurality of stored objects into broad classes using classifiers. The embodiment further selects an appropriate deduplication approach for each categorized class.

Abstract: A system for deduplicating data comprises a card operable to receive at least one data block and a processor on the card that generates a hash for each data block. The system further comprises a first module that determines a processing status for the hash and a second module that discards duplicate hashes and their data blocks and writes unique hashes and their data blocks to a computer readable medium. In one embodiment, the processor also compresses each data block using a compression algorithm.

Abstract: System and methods are described for sorting information in order O(n) time using O(n) space and searching for information in that sorted list in order O(1) time by using one single dimensioned array, without the use of other data structures and techniques, parallel processing, recursion, or other-sorting algorithm.

Abstract: To reduce consumption of the data capacity of a data migration-source storage by information necessary for accessing entity data that has been migrated to the other storage, compared to that of the conventional system. Provided is a storage system including a first storage that is a migration-destination storage having stored therein entity data and first index information associated with the entity data, and a second storage that is a migration-source storage having stored therein link information for accessing the entity data and second index information associated with the link information, wherein the second index information includes the same hash value as a hash value included in the first index information.

Abstract: Disclosed is a data caching system and a method for implementing a large capacity cache. The system includes: a record processing apparatus and a record storage apparatus which is configured with a first storage unit configured in a disk unit, a second storage unit and a third storage unit.

Abstract: The address generator has a hash network for producing hashed Y1, which is obtained by hushing X1, to an input vector X=(X1, X2), a tentative address generator Y1 for making an address generation function f(X) to a tentative address A? when no hash collision occurs and otherwise making one of unique addresses A to A?, a data regenerator for producing X?=f?1(A?), a unique address generator for producing A? when X? coincides with X and otherwise producing “invalid value”, a complementary address generator for producing (X) to X, to which the unique address generator produces “invalid value”, and otherwise producing “invalid value”, and an output combiner which produces, when the outputs of the unique address generator and the complementary address generator have values other than the “invalid value”, the values as a unique address A and otherwise produces “invalid value” as A.

Abstract: The present invention extends to methods, systems, and computer program products for updating Bloom filters. Embodiments of the invention facilitate more efficient use Bloom filters across multiple computers connected across a WAN (potentially having limited bandwidth and latency characteristics), such as, for example, computers located on different continents. The acceptability of false positives is leveraged by allowing the operation of removing items from a set to be batched and delayed. On the other hand, insert operations may be more latency sensitive as a delayed insert results in the semantic equivalent to a false negative. As such, additions to a set are processed in closer to real time to update Bloom filters. In some embodiments, Bloom filters are used to check set membership for electronic mail addresses.

Abstract: A content addressable memory (CAM) architecture comprises two components, a small, fast on-chip cache memory that stores data that is likely needed in the immediate future, and an off-chip main memory in normal RAM. The CAM allows data to be stored with an associated tag that is of any size and identifies the data. Via tags, waves of data are launched into a machine's computational hardware and re-associated with related tags upon return. Tags may be generated so that related data values have adjacent storage locations, facilitating fast retrieval. Typically, the CAM emits only complete operand sets. By using tags to identify unique operand sets, computations can be allowed to proceed out of order, and be recollected later for further processing. This allows greater computational speed via multiple parallel processing units that compute large sets of operand sets, or by opportunistically fetching and executing operand sets as they become available.

Abstract: A method is described that involves hashing a key value to locate a slot in a primary table, then, hashing the key value to locate a first slot in a secondary table, then, linearly probing the secondary table starting from the first slot.

Abstract: In one general aspect, various embodiments are directed to a method of writing a data block to a memory comprising receiving an electronic write request from an application. A content address of a first data block considering the value for the first data block. A mapping of the first data block to the content address may be written to a logical end of the local block map. The mapping may also be written to a remote block map. If the content address is not present at a local data storage, the value of the first data block may be written to the local data storage at a first location and metadata associating the content address with the first location may be written to the local data storage.

Abstract: A hash table system having a first hash table and a second hash table is provided. The first hash table may be in-memory and the second hash table may be on-disk. Inserting an entry to the hash table system comprises inserting the entry into the first hash table, and, when the first hash table reaches a threshold load factor, flushing entries into the second hash table. Flushing the first hash table into the second hash table may comprise sequentially flushing the first hash table segments into corresponding second hash table segments. When looking up a key/value pair corresponding to a selected key in the hash table system, the system checks both the first and second hash tables for values corresponding to the selected key. The first and second hash tables may be divided into hash table segments and collision policies may be implemented within the hash table segments.

Abstract: A multi-bit trie network search engine is implemented by a number of pipeline logic units corresponding to the number of longest-prefix strides and a set of memory blocks for holding prefix tables. Each pipeline logic unit is limited to one memory access, and the termination point within the pipeline logic unit chain is variable to handle different length prefixes. The memory blocks are coupled to the pipeline logic units with a meshed crossbar and form a set of virtual memory banks, where memory blocks within any given physical memory bank may be allocated to a virtual memory bank for any particular pipeline logic unit. An embedded programmable processor manages route insertion and deletion in the prefix tables, together with configuration of the virtual memory banks.

Abstract: There is provided a system and method for quality assured media file storage. There is provided a method for use by a processor to verify quality of a new media file by transcoding a master media file into a first media file, determining a quality of the first media file, applying a first quality assurance scheme to the first media file for an assured quality, calculating a first hash value using a hash function for the first media file, storing the first hash value in a memory, transcoding the master media file into a new media file, calculating a test hash value using the hash function for the new media file, and searching the memory for the test hash value to decide whether the new media file has the assured quality, without having to apply the first quality assurance scheme again to the new media file.

Abstract: Computer-implemented systems and associated operating methods implement a fast join for databases which is adapted for usage with flash storage. A system comprises a processor that performs a join of two tables stored in a storage in pages processed in a column orientation wherein column values for all rows on a page are co-located in mini-pages within the page. The processor reduces input/output operations of the join by accessing only join columns and mini-pages containing join results.

Abstract: A search device used to search for a search keyword from an object to be searched based on an automaton generated in accordance with the search keyword, includes: a first memory region; a second memory region; an information generation part that generates third information from first information and second information; a determination part; and a selection part, wherein: a first state corresponding to the first information is read from the first memory region; fourth information and a second state corresponding to the third information are read from the second memory region; the determination part determines whether or not fifth information including the first information and the second information matches with the fourth information; and the selection part selects the first state when not match in the determination and when matches in the determination, selects the second state.

Abstract: Generating and using a high-speed, scalable and easily updateable data structures are described. The proposed data structure provides minimal perfect hashing functionality while intrinsically supporting low-cost set-membership queries. In other words, in some embodiments, it provides at most one match candidate in a set of known arbitrary-length bit strings that is used to match the query.

Abstract: A new method and framework for scheduling receive-side processing of data streams received from a remote requesting client by a multiprocessor system computer is disclosed. The method receives data packets from the remote requesting client via a network and, for each data packet, applies a cryptographically secure hashing function to portions of the received data packet yielding a hash value. The method further applies the hash value to a processor selection policy to identify a processor in the multiprocessor system as a selected processor to perform receive-side processing of the data packet. The method queues the received data packet for processing by the selected processor and invokes a procedure call to initiate processing of the data packet.

Abstract: A data processing apparatus comprises a monolithic integrated circuit having a data processor, a non-volatile memory storing at least one security code, and at least one interface at the boundary of the integrated circuit via which communication with the data processor can occur. Processing by the data processor of data received at the at least one interface is controlled by the at least one security code.

Abstract: A system is described for providing scalable in-memory caching for a distributed database. The system may include a cache, an interface, a non-volatile memory and a processor. The cache may store a cached copy of data items stored in the non-volatile memory. The interface may communicate with devices and a replication server. The non-volatile memory may store the data items. The processor may receive an update to a data item from a device to be applied to the non-volatile memory. The processor may apply the update to the cache. The processor may generate an acknowledgement indicating that the update was applied to the non-volatile memory and may communicate the acknowledgment to the device. The processor may then communicate the update to a replication server. The processor may apply the update to the non-volatile memory upon receiving an indication that the update was stored by the replication server.

Abstract: A computer implemented method, data processing system, and computer usable program code are provided for storing data items in a computer. A plurality of hash functions of data values in a data item are computed. A corresponding memory location is determined for one of the plurality of hash functions. The data item and a key portion and a payload portion of all data items are stored contiguously within the memory location. Also provided for are retrieving data items in a computer. A plurality of hash functions of data values in a probe key are computed. A corresponding memory location is determined for each of the plurality of hash functions. Data items in each memory location are examined to determine a match with the probe key. Responsive to a match, a payload of the matching stored data item is returned. All of the steps are performed free of conditional branch instructions.

Abstract: A computer implemented method, data processing system, and computer usable program code are provided for storing data items in a computer. A plurality of hash functions of data values in a data item are computed. A corresponding memory location is determined for one of the plurality of hash functions. The data item and a key portion and a payload portion of all data items are stored contiguously within the memory location. Also provided for are retrieving data items in a computer. A plurality of hash functions of data values in a probe key are computed. A corresponding memory location is determined for each of the plurality of hash functions. Data items in each memory location are examined to determine a match with the probe key. Responsive to a match, a payload of the matching stored data item is returned. All of the steps are performed free of conditional branch instructions.

Abstract: Managing data includes: receiving at least one group of individually accessible data units over an input device or port, each data unit identified by a key value, with key values of the received data units being sorted such that the key value identifying a given first data unit that is received before a given second data unit occurs earlier in a sort order than the key value identifying the given second data unit; and processing the data units for storage in a data storage system.

Abstract: A compression device recognizes patterns of data and compressing the data, and sends the compressed data to a decompression device that identifies a cached version of the data to decompress the data. In this way, the compression device need not resend high bandwidth traffic over the network. Both the compression device and the decompression device cache the data in packets they receive. Each device has a disk, on which each device writes the data in the same order. The compression device looks for repetitions of any block of data between multiple packets or datagrams that are transmitted across the network. The compression device encodes the repeated blocks of data by replacing them with a pointer to a location on disk. The decompression device receives the pointer and replaces the pointer with the contents of the data block that it reads from its disk.

Abstract: In general, this disclosure describes techniques of storing data in and retrieving data from a cache of a computing device. More specifically, techniques are described for utilizing a “perfect hash” function to implement an associative cache within a computing device. That is, the associative cache implements a fully associative map between a predetermined set of addresses and data values, employing only a single tag fetch comparison.

Abstract: In one embodiment, a method includes receiving a portion of a hash key vector. The hash key vector can be defined based on a range value and based on at least a portion of an address value from a data packet queued within a multi-stage switch. The method also includes defining, based on the hash key vector, a hash value associated with a location in a hash table when the portion of the hash key vector matches a bit vector stored in a tag table.

Abstract: The transmission of data is distributed evenly and predictably over a given number of communication channels using a hash function.

Abstract: One embodiment of the present invention provides a system that implements a content-addressable memory (CAM) which has multiple banks. During operation, the system receives a request to insert an item into the CAM, wherein the request includes a key which is used to index the item and a body containing data. Next, for each bank in the CAM, the system calculates a different hash function based on the key to produce an index and a tag. The system then uses the calculated index and the tag for each bank to lookup an entry in each bank. If the lookups do not generate a hit in any bank, the system stores an entry for the request into a highest priority bank which does not contain a valid entry in the location accessed by the lookup. In one embodiment of the present invention, the multiple banks in the CAM have varying sizes.

Abstract: A multi-level lookup table includes a plurality of search levels with each search level including a plurality of subtrees, each subtree representing a plurality of nodes. A search of the multi-level lookup table for an entry corresponding to a search key results in a value stored in an entry associated with the node in a subtree. A default value is associated with the root of the subtree. Multiple entries for the subtree can store the default value. To minimize route update time, the default value associated with the subtree is stored in a single location. Instead of storing the default value in multiple entries, each entry stores a use default indicator to indicate that the default value stored in the single location is to be used. To further reduce the number of locations to modify to update the default route, the single location can store an inherit indicator to indicate that the default value for the subtree is inherited from another subtree.