Abstract: One embodiment of the present invention includes a method for routing a data object, comprising a sequence of data units, to a particular component data-storage system, or particular group of component data-storage systems, within a distributed, differential electronic-data storage system by selecting one or more subsequences of data units from the data object, computing a characteristic value from the selected subsequences, computing an index from the characteristic value; and directing the data object to the particular component data-storage system, or to the particular group component data-storage systems, identified by the computed index.

Abstract: Embodiments related to fetching instructions and alternate versions achieving the same functionality as the instructions from an instruction cache included in a microprocessor are provided. In one example, a method is provided, comprising, at an example microprocessor, fetching an instruction from an instruction cache. The example method also includes hashing an address for the instruction to determine whether an alternate version of the instruction which achieves the same functionality as the instruction exists. The example method further includes, if hashing results in a determination that such an alternate version exists, aborting fetching of the instruction and retrieving and executing the alternate version.

Abstract: Described embodiments provide a hash processor for a system having multiple processing modules and a shared memory. The hash processor includes a descriptor table with N entries, each entry corresponding to a hash table of the hash processor. A direct mapped table in the shared memory includes at least one memory block including N hash buckets. The direct mapped table includes a predetermined number of hash buckets for each hash table. Each hash bucket includes one or more hash key and value pairs, and a link value. Memory blocks in the shared memory include dynamic hash buckets available for allocation to a hash table. A dynamic hash bucket is allocated to a hash table when the hash buckets in the direct mapped table are filled beyond a threshold. The link value in the hash bucket is set to the address of the dynamic hash bucket allocated to the hash table.

Abstract: Methods and apparatus, including computer program products, implementing and using techniques for generating a hash. A data store including multiple hashing tables is provided. A set of data is received on which the hash is to be based. The set of data includes one or more components. An identifier is received. The identifier identifies one or more hashing tables to use when generating the hash. The received one or more components are processed in accordance with rules defined in the identified one or more hashing tables. The processed components are combined into a final hash.

Abstract: A mechanism is provided in a storage system for efficient inline data de-duplication. The mechanism receives a write command and a hash key for a portion of data to be written from an application host to a write address. The write command indicates whether the application host is tolerant or intolerant to data loss. Responsive to the write command indicating the application host is tolerant to data loss, the mechanism performs a hash key lookup in a hash index. The mechanism determines whether the portion of data has previously been written to the storage system. Responsive to determining the portion of data has previously been written to the storage system, the mechanism stores a pointer to the previously written data at the write address.

Abstract: A memory system includes a storage device storing a plurality of instructions and a central processing unit processing an instruction fetched from the storage device, wherein the central processing unit detects a change in the instruction fetched from the storage device while processing the instruction.

Abstract: Apparatus and method for placing data based on the content of the data in random access memory such that indexing operations are not required. A strong (e.g., cryptographic) hash is applied to a data element resulting in a signature. A weaker hash function is then applied to the signature to generate a storage location in memory for the data element. The weaker hash function assigns multiple data elements to the same storage location while the signature comprises a unique identifier for locating a particular data element at this location. In one embodiment a plurality of weak hash functions are applied successively to increase storage space utilization. In other embodiments, the assigned storage location can be determined by one or more attributes of the data element and/or the storage technology, e.g, long-lived versus short-lived data and/or different regions of the memory having different performance (e.g., access latency memory lifetime) characteristics.

Abstract: Method and apparatus for storing records in non-uniform access memory. In various embodiments, the placement of records is localized in one or more regions of the memory. This can be accomplished utilizing different ordered lists of hash functions to preferentially map records to different regions of the memory to achieve one or more performance characteristics or to account for differences in the underlying memory technologies. For example, one ordered list of hash functions may localize the data for more rapid access. Another list of hash functions may localize the data that is expected to have a relatively short lifetime. Localizing such data may significantly improve the erasure performance and/or memory lifetime, e.g., by concentrating the obsolete data elements in one location. Thus, the two or more lists of ordered hash functions may improve one or more of access latency, memory lifetime, and/or operation rate.

Abstract: A method, apparatus, and system of a software technique for improving disk write performance on raid system where write sizes are not an integral multiple of number of data disks are disclosed. In one embodiment, a method includes configuring a queue module to place an amount of data of a write operation into a data buffer module associated with a memory system if writing the amount of data to the memory system would generate a read-modify-write operation to occur, using the data buffer module to temporarily store the amount of data, writing the amount of data from the data buffer module to the memory system. The method may include algorithmically determining the amount of data to place in the data buffer module as a portion of the write operation that may cross a boundary between a striped sector unit (SSU) and/or an other SSU.

Abstract: In one aspect, a method includes determining locations in a file that have changed using read signature commands, reading data around a changed file location for each file location that has changed, generating variable-size portions of the data read and sending the variable-size portions of the data to a backup system.

Abstract: In a method for the management of tasks in a decentralized data network with a plurality of nodes for carrying out the tasks, resources are distributed based on a mapping rule, in particular a hash function. A task that is to be suspended is distributed by dividing the process image of the task into segments and by distributing the segments over the nodes using the mapping rule. Thus, a distributed swap space is created so that tasks can also be carried out on nodes whose swap space is not sufficient on its own. The method can be used in embedded systems with a limited storage capacity and/or in a voltage distribution system, wherein the nodes are, for example, switching units in the voltage distribution system. The method can also be used in any other technical systems such as, for example, a power generation system, an automation system and the like.

Abstract: A storage controller of the present invention narrows down the target for data comparison by comparing hash codes beforehand and rapidly detects duplicated data. A hash value setting unit sets a hash code in data received from a host. Hash code-attached data is stored in a logical volume. A microprocessor unit compares the hash codes for each comparison-targeted data. When hash codes match with one another, a data comparator compares the target data, and determines whether or not the data is duplicated data. When duplicated data is detected, the microprocessor unit removes the duplicated data.

Abstract: Managing storage identifiers in a pool is facilitated by providing a hashing-based management protocol in association with a stack which accommodates storage identifiers of the pool. The hashing-based management protocol includes: based on a request, popping a storage identifier from the stack without evaluating for update of a hash link associated with the stack, potentially allowing the hash link to become inconsistent with storage identifiers remaining in the stack; and based on return of a freed storage identifier to the stack, hashing the freed storage identifier and identifying whether there is an inconsistency in the hash link related to return of the freed storage identifier, and based on identifying the inconsistency, one of updating the hash link to remove the inconsistency, or indicating, where ascertained, that the freed storage identifier is a duplicate storage identifier.

Abstract: A hash table method and structure comprises a processor that receives a plurality of access requests for access to a storage device. The processor performs a plurality of hash processes on the access requests to generate a first number of addresses for each access request. Such addresses are within a full address range. Hash table banks are operatively connected to the processor. The hash table banks form the storage device. Each of the hash table banks has a plurality of input ports. Specifically, each of the hash table banks has less input ports than the first number of addresses for each access request. The processor provides the addresses to the hash table banks, and each of the hash table banks stores pointers corresponding to a different limited range of addresses within the full address range (each of the different limited range of addresses is less than the full address range).

Abstract: A method for deploying one or more virtual machines on a host computing system is provided. The method comprises receiving mapping information from a data storage system. The mapping information associates a first data chunk stored in the data storage system with a unique identifier to support deployment of a first virtual machine on a host computing system. Once the mapping information is received, the mapping information is utilized to determine whether any copies of the first data chunk have already been loaded into a memory of the host computing system in association with deployment of the first virtual machine or a second virtual machine on the host computing system. If no copies of the first data chunk have already been loaded into the memory, the first data chunk is retrieved from the data storage system, loaded into the memory, and utilized to deploy the first virtual machine on the host computing system.

Abstract: A method for operating a communication system is provided. The method includes receiving an arrival rate of a plurality of real-time packets, and receiving a real-time packet including a first plurality of identifiers, for transmission on a first link or a second link, where the first link has a first bandwidth. The method also includes processing the real-time packet to select a first selected link from the first link and the second link based on the first plurality of identifiers, the arrival rate of the plurality of real-time packets, and the first bandwidth, and transmitting the real-time packet on the first selected link.

Abstract: A data read/write system receives a key associated with a data read request. The data read/write system hashes the key to obtain a first hash value and hashes the key to obtain a second hash value, where the second hash value is different than the first hash value. The data read/write system obtains a pointer from a pointer array using the first and second hash values, and uses one or more bits of the pointer and the first hash value to retrieve data from a data look-up array.

Abstract: Embodiments of the present invention provide a system that generates an index for a cache memory. The system starts by receiving a request to access the cache memory, wherein the request includes address information. The system then obtains non-address information associated with the request. Next, the system generates the index using the address information and the non-address information. The system then uses the index to fulfill access the cache memory.

Abstract: A method for data backup including (a) forming an image of the storage device; (b) for each block to be backed up to the image, generating a hash; (c) for each block stored in the image, storing a hash in a hash table; (c) for each additional block of the storage device to be backed up, generating a hash; (d) sorting all the generated hashes and deleting duplicate hashes; (e) comparing the hashes to identify candidate blocks that might have identical contents with contents of blocks stored in the image; (f) if the hashes are not coincident, then backing up, to the image, contents of unidentified blocks and blocks that do not have identical hashes, and (g) otherwise, storing links in the image instead of the contents of the blocks, (h) after checking all the hashes for those blocks that need to be backed up, writing only unique hashes to the hash table; (h) links for multiple blocks with identical contents point to a single block in the image, (i) the image contains a bitmap of the backup; (j) the bitmap conta

Abstract: Some embodiments provide a system that facilitates concurrency control in a computer system. During operation, the system generates a set of signatures associated with memory accesses in the computer system. To generate the signatures, the system creates a set of hierarchical Bloom filters (HBFs) corresponding to the signatures, and populates the HBFs using addresses associated with the memory accesses. Next, the system compares the HBFs to detect a potential conflict associated with the memory accesses. Finally, the system manages concurrent execution in the computer system based on the detected potential conflict.

Abstract: De-duplication operations are applied to mirror volumes. Data stored to a first volume is mirrored to a second volume. The second volume is a virtual volume having a plurality of logical addresses, such that segments of physical storage capacity are allocated for a specified logical address as needed when data is stored to the specified logical address. A de-duplication operation is carried out on the second volume following a split from the first volume. A particular segment of the second volume is identified as having data that is the same as another segment in the second volume or in the same consistency group. A link is created from the particular segment to the other segment and the particular segment is released from the second volume.

Abstract: Systems and methods for pre-fetching of data in a memory are provided. By pre-fetching stored data from a slower memory into a faster memory, the amount of time required for data retrieval and/or processing may be reduced. First, data is received and pre-scanned to generate a sample fingerprint. Fingerprints stored in a faster memory that are similar to the sample fingerprint are identified. Data stored in the slower memory associated with the identified stored fingerprints is copied into the faster memory. The copied data may be compared to the received data. Various embodiments may be included in a network memory architecture to allow for faster data matching and instruction generation in a central appliance.

Abstract: Methods and apparatus relating to ring protocols and techniques are described. In one embodiment, a first agent generates a request to write to a cache line of a cache over a first ring of a computing platform. A second agent that receives the write request forwards it to a third agent over the first ring of the computing platform. In turn, a third agent (e.g., a home agent) receives data corresponding to the write request over a second, different ring of the computing platform and writes the data to the cache. Other embodiments are also disclosed.

Abstract: A hash table system having a first hash table and a second hash table is provided. The first hash table may be in-memory and the second hash table may be on-disk. Inserting an entry to the hash table system comprises inserting the entry into the first hash table, and, when the first hash table reaches a threshold load factor, flushing entries into the second hash table. Flushing the first hash table into the second hash table may comprise sequentially flushing the first hash table segments into corresponding second hash table segments. When looking up a key/value pair corresponding to a selected key in the hash table system, the system checks both the first and second hash tables for values corresponding to the selected key. The first and second hash tables may be divided into hash table segments and collision policies may be implemented within the hash table segments.

Abstract: Embodiments include methods, apparatus, and systems for virtual address hashing. One embodiment evenly distributes page-table entries throughout a hash table so applications do not generate a same hash index for mapping virtual addresses to physical addresses.

Abstract: Methods, systems and data structures produce a rasterizer. A graphical state is detected on a machine architecture. The graphical state is used for assembling a shell rasterizer. The machine architecture is used for selecting replacement logic that replaces portions of shell logic in the shell rasterizer. The machine architecture is used for selectively inserting memory management logic into portions of the shell logic to produce.

Abstract: A distributed caching system for storing and serving information modeled as a graph that includes nodes and edges that define associations or relationships between nodes that the edges connect in the graph.

Abstract: The present disclosure provides a data read and write method and apparatus and a storage system. The data read and write method includes: receiving, by a block storage driver of an origination device, a request for an LBA-based operation on a volume; converting the LBA-based operation request into a Key addressing-based operation request; and sending the Key addressing-based operation request to a routing library. Therefore, according to the present disclosure, reliability, availability, extensibility, and cheapness of the storage system may be improved, satisfying mass storage requirements for high extensibility, high reliability, high availability, and cheapness.

Abstract: In one embodiment, a system comprises multiple memory ports distributed into multiple subsets, each subset identified by a subset index and each memory port having an individual wait time. The system further comprises a first address hashing unit configured to receive a read request including a virtual memory address associated with a replication factor, and referring to graph data. The first address hashing unit translates the replication factor into a corresponding subset index based on the virtual memory address, and converts the virtual memory address to a hardware based memory address that refers to graph data in the memory ports within a subset indicated by the corresponding subset index. The system further comprises a memory replication controller configured to direct read requests to the hardware based address to the one of the memory ports within the subset indicated by the corresponding subset index with a lowest individual wait time.

Abstract: A system for improving the reliability of transmission of multibroadcast data between a source of data and a plurality of customer receivers of the data via a telecommunications network. The system includes a supervision server adapted to provide each customer receiver with a list of at least one other receiver, said customer receiver being adapted to receive missing data from said the other customer receiver via a reliable peer-to-peer connection.

Abstract: An apparatus for abstract memory addressing. A processor for generating an abstract memory address. A base register for storing a base memory address. An adder for adding the base memory address to the abstract memory address and generating a physical address for a device memory. A pointer register for storing the physical address, wherein the pointer register is directly coupled to the device memory.

Abstract: A method for implementing a hash map to improve performance consistency is disclosed herein. In one embodiment, such a method includes providing a hash map comprising a set of tables, the set of tables initially containing a first table. When the first table is full, the method augments the hash map by adding a second table to the set of tables. Similarly, when the second table is full, the method augments the hash map by adding a third table to the set of tables. A similar technique may be used to add additional tables to the hash map. When searching for a value in any of the tables in the hash map, the method uses the same hash code. A corresponding computer program product and apparatus are also disclosed herein.

Abstract: Systems and methods are disclosed for using a first deduplicating store to update a second deduplicating store with information representing how data objects change over time, said method comprising: at a first and a second deduplicating store, for each data object, maintaining an organized arrangement of temporal structures to represent a corresponding data object over time, wherein each structure is associated with a temporal state of the data object and wherein the logical arrangement of structures is indicative of the changing temporal states of the data object; finding a temporal state that is common to and in temporal proximity to the current state of the first and second deduplicating stores; and compiling and sending a set of hash signatures for the content that has changed from the common state to the current temporal state of the first deduplicating store.

Abstract: An information processing system includes: a data processing unit that executes verification processing for a content recorded in a disk and reproduces the disk-recorded content under a condition that the verification succeeds, wherein the data processing unit randomly selects hash units, which are objects of collation, from among a plurality of hash units formed with component data items of the content, reads the selected hash units sequentially from the disk, calculates hash values, and collates the calculated hash values with collation hash values; and the data processing unit executes reading sequence determination processing so as to determine a reading sequence in which the selected hash units are sorted according to recording positions in a disk, and reads the selected hash units according to the determined reading sequence.

Abstract: A mechanism is provided for security screening image analysis simplification through object pattern identification. Popular consumer electronics and other items are scanned in a control system, which creates an electronic signature for each known object. The system may reduce the signature to a hash value and place each signature for each known object in a “known good” storage set. For example, popular mobile phones, laptop computers, digital cameras, and the like may be scanned for the known good signature database. At the time of scan, such as at an airport, objects in a bag may be rotated to a common axis alignment and transformed to the same signature or hash value to match against the known good signature database. If an item matches, the scanning system marks it as a known safe object.

Abstract: A system and method are provided for deduplication (dedup) of data file information in a network of distributed data filers. A host, including a metadata service (MDS) and a data node (DN), receives a block count and hash group calculations for a file from a network-connected client device. The MDS creates a file map with target addresses, and compares the calculated hash group to hash groups associated with stored blocks of data in a global dedup hash table. If a match is found, the MDS directs the client device not to send the block of data associated with matching hash group, and reallocates the address of the block of data from the target address.

Abstract: Systems and methods are disclosed for forming deduplicated images of a data object that changes over time using difference information between temporal states of the data object. The method includes organizing the content of the data object for a first temporal state as a plurality of content segments and storing the content segments in a data store; creating an organized arrangement of hash structures to represent the data object in its first temporal state; receiving difference information for the data object; forming at least one hash signature for the changed content; and storing the changed content that is unique in the data store as content segments, whereby a deduplicated image of the data object for a second temporal state is stored without requiring reception of a complete image of the data object for the second temporal state.

Abstract: A hash table system having a first hash table and a second hash table is provided. The first hash table may be in-memory and the second hash table may be on-disk. Inserting an entry to the hash table system comprises inserting the entry into the first hash table, and, when the first hash table reaches a threshold load factor, flushing entries into the second hash table. Flushing the first hash table into the second hash table may comprise sequentially flushing the first hash table segments into corresponding second hash table segments. When looking up a key/value pair corresponding to a selected key in the hash table system, the system checks both the first and second hash tables for values corresponding to the selected key. The first and second hash tables may be divided into hash table segments and collision policies may be implemented within the hash table segments.

Abstract: Systems and methods are disclosed for performing garbage collection to identify content segments no longer referenced in a deduplicating storage system in which redundant mark operations in a mark-and-sweep technique are avoided. An organized arrangement of hash structures is created for each data object, wherein each structure includes a hash signature for a corresponding content segment and is associated with a reference to the corresponding content segment, and the logical organization of the arrangement represents the logical organization of the content segments as they are represented within the data object. Additionally, for each data object, temporal states are maintained over time. Garbage collection iterates over the temporal structures and, for each temporal structure, marks the garbage collection state for the associated content segments for only the content segments that have changed relative to an immediately prior temporal state of the data object.

Abstract: Systems, methods embodied on computer-readable media, and other embodiments associated with index entry eviction are described. One example method includes selecting an index entry for eviction from a bucket of index entries based on a time value, a utility value, and a precedence value. A precedence value may be a value associated with an index entry that is static over time. Additionally, results of a function that compares two precedence values may be static over time. The example method may also include providing an index entry identifier that identifies the index entry.

Abstract: A method and apparatus for determining a cache line in an N-way set associative cache are disclosed. In one example embodiment, a key associated with a cache line is obtained. A main hash is generated using a main hash function on the key. An auxiliary hash is generated using an auxiliary hash function on the key. A bucket in a main hash table residing in an external memory is determined using the main hash. An entry in a bucket in an auxiliary hash table residing in an internal memory is determined using the determined bucket and the auxiliary hash. The cache line in the main hash table is determined using the determined entry in the auxiliary hash table.

Abstract: A management method for reducing the utilization rate of random access memory (RAM) while reading data from or writing data to the flash memory is disclosed. A physical memory set is constructed from a plurality of physical memory blocks in the flash memory. A logical set is constructed from a plurality of logical blocks wherein the data stored in the logical set are stored in the physical memory set. Further, the data stored in each of the logical blocks are stored in one number of physical memory blocks. A mapping table is constructed and includes a hash function, a logical set table, a physical memory set table, and a set status table for managing the relationship among the physical memory sets, physical memory blocks, and logical blocks while reading data from or writing data to the flash memory.

Abstract: An inner product if formed from an input signal and a hashing vector, to which a dither scalar is added. A binary digit is produced according to a quantization function, wherein the binary digit is zero or one, and wherein the quantization function is non-monotonic and subject to a sensitivity parameter. The steps are iterated while decreasing the sensitivity parameter until a termination condition is reached.

Abstract: A method for writing data to persistent storage. The method include receiving a first write request including a key and a value, hashing the key to obtain a hashed key, obtaining a hash table depth (d), obtaining d bits from the hashed key, and making a first determination that a hash frag associated with the d bits from the hashed key exists. In response to the first determination, obtaining the hash frag, making a second determination that there is available space to store a hash frag entry in the hash frag, storing the hash frag entry in the hash frag to obtain an updated hash frag, where the hash frag entry includes the hashed key and value information for the value, and storing the updated hash frag in the persistent storage.

Abstract: A computer readable storage medium, embodying instructions executable by a computer to perform a method, the method including: validating a memory write of data segments using a first number of leaf hashes of a first hash tree, where each of the first number of leaf hashes is associated with one of the data segments of a first block size, generating interior node hashes based on the first number of leaf hashes, where each of the interior node hashes is associated with a second block size, generating a first root hash using the interior node hashes, where the first root hash is associated with a remote procedure call size, transmitting the first root rash and the data segments to a network file system, where the transmission is performed using the remote procedure call size, and validating the transmission of the data segments using the first root hash.

Abstract: A method, system, and article for compressing an input stream of uncompressed data. The input stream is divided into one or more data segments. A hash is applied to a first data segment, and an offset and length are associated with this first segment. This hash, together with the offset and length data for the first segment, is stored in a hash table. Thereafter, a subsequent segment within the input stream is evaluated and compared with all other hash entries in the hash table, and a reference is written to a prior hash for an identified duplicate segment. The reference includes a new offset location for the subsequent segment. Similarly, a new hash is applied to an identified non-duplicate segment, with the new hash and its corresponding offset stored in the hash table. A compressed output stream of data is created from the hash table retained on storage media.

Abstract: A packet inspection device and method for use with a packet-retrievable network apparatus are provided. The packet inspection method includes: converting header information of a packet received into a hashing function value in presence of handshaking underway at the Transmission Control Protocol (TCP) layer and comparing the hashing function value by a hashing function unit of the pending processing module, storing the hashing function value in a memory unit, and performing packet state comparison and packet screening and then creating by the session processing module a transmission connection according to the packet screened and selected by the pending processing module upon determination that data stored in the memory unit match the hashing function value resulting from conversion by the hashing function unit, thereby expediting packet inspection, reducing occupied memory space, and cutting costs.

Abstract: Implementations and techniques for flash memory-type hash tables are generally disclosed.

Abstract: A method of limiting redundant storage of data comprises receiving a data stream and partitioning the data stream into a series of data chunks. At least one content hash value for a set of data chunks is generated based on data content of the set of data chunks. One or more data chunks are grouped into a segment with at least one boundary of the segment defined based on an evaluation of content hash values of data chunks. Content hash values of data chunks of the segment are compared to content hash values of data chunks of segments stored on a backup mass storage device. A pointer to a stored data chunk of an existing segment is stored on the backup mass storage device if a content hash value of a data chunk of the segment matches the content hash value of the stored data chunk.

Abstract: An Ethernet controller, for use within an Ethernet network of other Ethernet controller connected together by a bus, is provided. The Ethernet controller includes a plurality of ports including at least one bus port associated with ports connected to other switching Ethernet controllers, a hash table for storing addresses of ports within the Ethernet network, a hash table address control, a storage buffer including a multiplicity of contiguous buffers in which to temporarily store said packet, an empty list including a multiplicity of single bit buffers, a packet storage manager, a packet transfer manager and a write-only bus communication unit. The hash table address control hashes the address of a packet to initial hash table location values, changes the hash table location values by a fixed jump amount if the address values stored in the initial hash table location do not match the received address, and provides at least an output port number of the port associated with the received address.