Abstract: The present disclosure describes a system and method for high performance, power efficient store buffer forwarding. Some illustrative embodiments may include a system, comprising: a processor coupled to an address bus; a cache memory that couples to the address bus and comprises cache data (the cache memory divided into a plurality of ways); and a store buffer that couples to the address bus, and comprises store buffer data, a store buffer way and a store buffer index. The processor selects the store buffer data for use by a data load operation if a selected way of the plurality of ways matches the store buffer way, and if at least part of the bus address matches the store buffer index.

Abstract: Techniques for reducing storage space and detecting corruption in hash-based applications are presented. Data strings are hashed or transformed into numerically represented strings. Groupings of the numeric strings form a set. Each numeric string of a particular set is associated with a unique co-prime number. All the numeric strings and their corresponding co-prime numbers for a particular set are processed using a Chinese Remainder Theorem algorithm (CRT) to produce a single storage value. The single storage value is retained in place of the original numeric strings. The original numeric strings can be subsequently reproduced and verified using the single storage value and the co-prime numbers.

Abstract: Exemplary embodiments for increased in-line deduplication efficiency in a computing environment are provided. Embodiments include incrementing the size of data samples from fixed size data chunks for each nth iteration for reaching a full size of an object requested for in-line deduplication, calculating in nth iterations hash values on data samples from fixed size data chunks extracted from the object, and matching in a nth hash index table the calculated nth iteration hash values for the data samples from the fixed size data chunks with a corresponding hash value of existing objects in storage, wherein the nth hash index table is built for each nth iteration of the data samples belonging to the fixed data chunks.

Abstract: Devices, systems, methods, and other embodiments associated with selectively scheduling memory accesses in parallel are described. In one embodiment, a method determines an access speed for a page request. The access speed is a number of clock cycles used to access a memory device of a group of memory devices. The page request is a request to access a memory page mapped to the memory device. Different page requests are selectively scheduled to access different memory devices in parallel. The different page requests access the different memory devices in a same number of clock cycles.

Abstract: A system for processing packets includes a communications interface and a processor. A communications interface receives a packet between a source and a destination. The processor identifies a flow between the source and the destination based on the packet. The processor determines whether some of packet data of the packet matches to storage data in storage using hashes. The processor then stores the packet data in a block of memory in the storage based on the flow and if the packet data does not match the storage data.

Abstract: Delta compression after identity deduplication is disclosed. A first data segment is determined to be identical to a first previous data segment. A second data segment, not determined to be identical to a second previous data segment, is then determined to be similar to a third previous data segment.

Abstract: An approach identifies an amount of high order bits used to store a memory address in a memory address field that is included in a memory. This approach calculates at least one minimum number of low order bits not used to store the address with the calculation being based on the identified amount of high order bits. The approach retrieves a data element from one of the identified minimum number of low order bits of the address field and also retrieves a second data element from one of the one of the identified minimum number of low order bits of the address field.

Abstract: A hash table controller may include a hash calculator configured to receive a key and to determine, based thereon, a first entry in a first bank of a hash table for a value associated with the key and determine a second entry in a second bank of the hash table for the value. The hash table controller also may include a table operations manager configured to determine that the first entry and the second entry are empty, and to store the value and a duplicate of the value at both the first entry and the second entry, respectively.

Abstract: An improved computer system may include a controller including a computer processor. The system may also include a selector apparatus in communication with the controller to choose a table having a higher collision quality index than other tables under consideration by the selector apparatus. The system may further include an exchanger apparatus to configure a standby table that replaces the table chosen by the selector apparatus. The system may additionally include a switch that changes a hash function based upon the exchanger apparatus? replacement of the chosen table to enable the controller to reduce insertion times and/or collisions when interfacing with new components introduced to the controller.

Abstract: A method, computer program product, and computing system for receiving a read request on a first cache system, wherein the read request identifies previously-written content included within a data array. The previously-written content identified in the read request is obtained from the data array. A read request content identifier is generated for the previously-written content identified in the read request. The read request content identifier associated with the previously-written content identified in the read request is compared to a plurality of content identifiers included within a content directory for the first cache system to determine if a matching content identifier exists. Each of the plurality of content identifiers is associated with a piece of previously-written content included within the first cache system.

Abstract: A method, computer program product, and computing system for receiving a read request on a first cache system, wherein the read request identifies previously-written content included within a data array. A read request content identifier associated with the previously-written content identified in the read request is received from the data array. The read request content identifier associated with the previously-written content identified in the read request is compared to a plurality of content identifiers included within a content directory for the first cache system to determine if a matching content identifier exists. Each of the plurality of content identifiers is associated with a piece of previously-written content included within the first cache system.

Abstract: A memory device includes a hash table storing a hash value, a bit value, and a page address for each of a plurality of pages, a memory cell unit configured to store the pages and output contents corresponding to the page addresses of the pages having a same hash value, and a controller including a comparator configured to compare the contents output from the memory cell unit and change at least one bit value associated with a respective one of the pages upon determining that the contents of the pages are the same.

Abstract: A transactional memory (TM) receives a lookup command across a bus from a processor. The command includes a memory address. In response to the command, the TM pulls an input value (IV). The memory address is used to read a word containing multiple result values (RVs), multiple reference values, and multiple mask values from memory. A selecting circuit within the TM uses a starting bit position and a mask size to select a portion of the IV. The portion of the IV is a lookup key value (LKV). The LKV is masked by each mask value thereby generating multiple masked values. Each masked value is compared to a reference value thereby generating multiple comparison values. A lookup table generates a selector value based upon the comparison values. A result value is selected based on the selector value. The selected result value is then communicated to the processor via the bus.

Abstract: Systems and methods for pre-fetching of data in a memory are provided. By pre-fetching stored data from a slower memory into a faster memory, the amount of time required for data retrieval and/or processing may be reduced. First, data is received and pre-scanned to generate a sample fingerprint. Fingerprints stored in a faster memory that are similar to the sample fingerprint are identified. Data stored in the slower memory associated with the identified stored fingerprints is copied into the faster memory. The copied data may be compared to the received data. Various embodiments may be included in a network memory architecture to allow for faster data matching and instruction generation in a central appliance.

Abstract: A cache memory is utilized effectively because data redundancy elimination is executed. A controller manages the cache memory by dividing it into a first area and a second area. When receiving a write access request from an access requestor, the controller divides a data block, which is an access target, into a plurality of chunks and searches the first area first and then the storage apparatus based on each chunk. If chunk storage information, indicating that each chunk is stored in the storage apparatus, does not exist in the first area or the storage apparatus, the controller executes chunk storage processing and creates and stores the chunk storage information. If the chunk storage information exists, the controller eliminates the chunk storage processing for storing the chunks. If the chunk storage information does not exist in the first area, the controller stages the chunk storage information from the storage apparatus to the first area on condition that the first area has an unused area.

Abstract: An example method is provided and can include initiating a probe session at a source network element; identifying multiple paths from the source network element to a destination network element in a network; transmitting packets from the source network element along the multiple paths; compiling a list of network characteristics associated with the multiple paths; and selecting a particular one of the multiple paths for packet routing based on the network characteristics.

Abstract: Data de-duplication (“de-dupe”) allows IT organizations to replace aging tape systems with disk-based backup solutions and minimize the storage allocated to backup and data protection. The effectiveness of de-dupe technology is dependent on the data being protected. Data streams with little data repetitiveness provide disappointing results when processed through a block-level de-dupe engine. To avoid this problem, Assisted Mainframe De-Dupe (AMDD) technology can insure that filesystem block-level de-dupe products efficiently de-dupe tape backup streams received from IBM™ and/or compatible mainframes. By pre-processing backup tape volumes before sending the data to storage, AMDD insures that large amounts of unchanged data lines up on de-dupe block boundaries each time the data is sent to the de-dupe process engine.

Abstract: A user device includes a flash memory configured to store an index including a plurality of index nodes and a controller configured to control the flash memory. The controller is configured to detect a pointer ID corresponding to a selected key of a first index node, translate the detected pointer ID to an index address by using a pointer table, and access a second index node corresponding to the selected key by using the index address.

Abstract: Described herein is a method and system for distributing whole and fragmented requests and responses across a multi-core system. Each core executes a packet engine that further processes data packets and data packet fragments allocated to that core. A flow distributor executing within the multi-core system forwards client requests to a packet engine on a core that is selected based on a value generated when a hash is applied to a tuple comprising a client IP address, a client port, a server IP address and a server port identified in the request. The packet engine maintains each element of the tuple and forwards the request to the selected core. The packet engine can also process data packet fragments by assembling the fragments prior to transmitting them to the selected core, or by transmitting the data packet fragments to the selected core.

Abstract: Distribution of content between publishers and consumers is accomplished using an overlay network that may make use of XML language to facilitate content identification. The overlay network includes a plurality of routers that may be in communication with each other and the publishers and consumers on the Internet. Content and queries are identified by content descriptors that are routed from the originator to a nearest router in the overlay network. The nearest router, for each unique content descriptor, generates a hash identification of the content descriptor which is used by remaining routers in the overlay network to provide the appropriate functions with respect to the content descriptor. In particular, this allows all routers in the overlay network except the nearest router to properly route content without processing every content descriptor.

Abstract: A non-transitory computer-readable medium storing an data storing program executed by an archive device including a first storage unit for storing data and a second storage unit for storing hash value determined from the data, the program causing the archive device to execute a process includes receiving hash value determined from data to be stored from an external device which requests storage of the data, comparing the received hash value with the hash value stored in the second storage unit, and transmitting request information for transmitting the data corresponding to the received hash value to the external device which transmits the hash value when the received hash value has not been stored in the second storage unit.

Abstract: The invention provides a method for reducing identification of chunk portions in data de-duplication. The method includes detecting sequences of stored identification of chunk portions of at least one data object, indexing the detected stored identification of chunk portions based on a sequence type, encoding first repeated sequences of the stored identifications with a first encoding, encoding second repeated sequences of the stored identifications with a second encoding, and avoiding repeated stored identifications of chunk portions.

Abstract: A system has a data structure in which a value can be obtained from a key. In a write access, a first pair <Key,Hash(Value)> and a second pair <Hash(Value),Value> are stored respectively in a volatile storage device. The first pair <Key,Hash(Value)> is saved in a nonvolatile storage device before returning a response, and the second pair <Hash(Value),Value> is saved in the first storage device at any time with the second pair saved in the volatile storage device. In a read access in which a value is obtained from a key, it is determined that data is not stored normally if the second pair is not found in processing in which after obtaining the hash value of the value from the first pair, the second pair is read.

Abstract: An example computing device includes a prefix lookup module, and a Bloom filter that includes a set of queues. The prefix lookup module is configured to receive policy configuration information, examine a state of a queue of the set of queues, and determine whether to bypass the first Bloom filter based on the policy configuration information and the state of the queue. In one example, the prefix lookup module may be configured to, using the policy configuration information, determine to bypass the Bloom filter when the queue is full. In another example, the prefix lookup module may be configured to, using the policy configuration information, determine not to bypass the Bloom filter and send a lookup request to the Bloom filter upon determining that the queue is no longer full.

Abstract: Techniques are described for using chunk stores as building blocks to construct larger chunk stores. A chunk store constructed of other chunk stores (a composite chunk store) may have any number and type of building block chunk stores. Further, the building block chunk stores within a composite chunk store may be arranged in any manner, resulting in any number of levels within the composite chunk store. The building block chunk stores expose a common interface, and apply the same hash function to content of chunks to produce the access key for the chunks. Because the access key is based on content, all copies of the same chunk will have the same access key, regardless of the chunk store that is managing the copy. In addition, no other chunk will have that same access key.

Abstract: Systems and methods for managing mapping information for objects maintained in a distributed storage system are provided. The distributed storage system can include a keymap subsystem that manages the mapping information according to object keys. Requests for specific object mapping information are directed to specific keymap coordinators within the keymap subsystem. Each keymap coordinator can maintain a cache for caching mapping information maintained at various information sources. Additionally, the keymap coordinators can optimize cache entries by maintaining selected mapping information while identifying object sources that correspond to differences from the selected mapping information.

Abstract: A technique for routing data for deduplication in a storage server cluster includes computing, for each node in the cluster, a value collectively representative of the data stored on the node, such as a “geometric center” of the node. New or modified data is routed to the node which has stored data identical or most similar to the new or modified data, as determined based on those values. Each node stores a plurality of chunks of data, where each chunk includes multiple deduplication segments. A content hash is computed for each deduplication segment in each node, and a similarity hash is computed for each chunk from the content hashes of all segments in the chunk. A geometric center of a node is computed from the similarity hashes of the chunks stored in the node.

Abstract: A method of resizing a concurrently accessed hash table is disclosed. The method includes acquiring the locks in the hash table. The hash table, in a first state, is dynamically reconfigured in size into a second state. Additionally, the amount of locks is dynamically adjusted based on comparing the size of the hash table in the second state to the size of the hash table in the second state.

Abstract: A translation lookaside buffer (TLB) is disclosed formed using RAM and synthesisable logic circuits. The TLB provides logic within the synthesisable logic for pairing down a number of memory locations that must be searched to find a translation to a physical address from a received virtual address. The logic provides a hashing circuit for hashing the received virtual address and uses the hashed virtual address to index the RAM to locate a line within the RAM that provides the translation.

Abstract: Some embodiments provide a method for creating an image of a virtual machine. The method identifies a particular computer system operating as a virtual machine with a particular configuration on a hardware resource of a hosting system that includes several hardware resources. The method captures data representing the particular computer system. Capturing the data includes copying a particular section of the data, computing a checksum for the particular section of the data, and streaming the particular section with the computed checksum to a storage.

Abstract: A measurement engine performs active platform observation. A program includes an integrity manifest to indicate an integrity check value for a section of the program's source code. The measurement engine computes a comparison value on the program's image in memory and determines if the comparison value matches the expected integrity check value. If the values do not match, the program's image is determined to be modified, and appropriate remedial action can be triggered. The integrity manifest can include a secure signature to verify the validity of the integrity manifest.

Abstract: Methods and articles of manufacture relating to hash tables and in particular to minimally disruptive hash tables are disclosed. In one aspect, the method includes creating the hash table with a plurality of hash table buckets, wherein a count of the plurality of hash table buckets is greater than or equal to a maximum count of elements to be stored in the hash table over a period in which the hash table is used, storing a plurality of elements in the plurality of hash table buckets such that each hash table bucket has one of the plurality of elements, and adding a new element to the hash table. Adding the new element includes determining, using a hash function, a plurality of hash table indexes and inserting the new element in the identified hash table buckets by replacing existing elements stored in the hash table buckets.

Abstract: A method for writing data to a storage pool. The method includes receiving a virtual identifier (ID) and an offset for an object, extracting a node identifier (ID) that identifies a first storage server in the storage pool from the virtual ID, obtaining an object layout (OL) for the object from the first storage server, hashing an object ID and an offset ID to obtain a hashed value, where the virtual ID comprises the object ID and where the offset ID is generated from the offset, identifying a second storage server in the storage pool using a global namespace layout (GL), the OL, and the hashed value, and issuing a write request to write data to the object in the second storage server, where the write request comprises the object ID and the offset.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, manage metadata for virtual volumes. In some implementations, a method and system include defining multiple metadata blocks in a persistent storage, including information that links a virtual address space to the storage system, where the defining includes, for at least one of the multiple metadata blocks, determining multiple output addresses corresponding to the storage system, and writing the multiple output addresses and an identifier corresponding to the multiple metadata blocks in a metadata block in the persistent storage. In some implementations, a method and system include reading the multiple metadata blocks into the memory from the persistent storage, including identifying the metadata block based on the identifier; receiving an input address of the virtual address space; and obtaining a corresponding output address to the storage system using the multiple metadata blocks in the memory.

Abstract: A content alignment system according to certain embodiments aligns a sliding window at the beginning of a data segment. The content alignment system performs a block alignment function on the data within the sliding window. A deduplication block is established if the output of the block alignment function meets a predetermined criteria. At least part of a gap is established if the output of the block alignment function does not meet the predetermined criteria. The predetermined criteria is changed if a threshold number of outputs fail to meet the predetermined criteria.

Abstract: A computer implemented method, apparatus, and computer usable program code for processing packets for transmission. A set of interface specific network buffers is identified from a plurality of buffers containing data for a packet received for transmission. A data structure describing the set of interface specific network buffers within the plurality of buffers is created, wherein a section in the data structure for an interface specific network buffer in the set of interface specific network buffers includes information about a piece of data in interface specific network buffer, wherein the data structure is used to process the packet for transmission.

Abstract: A method, system and apparatus for efficient storage of small files in a segment-based deduplication scheme by allocating multiple small files to a single data segment is provided. A mechanism for distinguishing between large files (e.g., files that are on the order of the size of a segment or larger) and smaller files, and starting a new segment at the beginning of a large file is also provided. A file attribute-based system for determining an identity of a small file at which to begin a new segment and then allocating subsequent small files to that segment and contiguous segments until a next small file having an appropriate attribute subsequently is encountered to begin a new segment is further provided. In one aspect of the present invention a filename hash is used for file attribute analysis to determine when a new segment should begin.

Abstract: In one embodiment, an apparatus can include a policy vector module configured to retrieve a compressed policy vector based on a portion of a data packet received at a multi-stage switch. The apparatus can also include a decompression module configured to receive the compressed policy vector and configured to define a decompressed policy vector based on the compressed policy vector. The decompressed policy vector can define a combination of bit values associated with a policy.

Abstract: In one embodiment, a system includes memory ports distributed into subsets identified by a subset index, where each memory port has an individual wait time based on a respective workload. The system further comprises a first address hashing unit configured to receive a read request including a virtual memory address associated with a replication factor and referring to graph data. The first address hashing unit translates the replication factor into a corresponding subset index based on the virtual memory address, and converts the virtual memory address to a hardware based memory address referring to graph data in the memory ports within a subset indicated by the corresponding subset index. The system further comprises a memory replication controller configured to direct read requests to the hardware based address to the one of the memory ports within the subset indicated by the corresponding subset index with a lowest individual wait time.

Abstract: Data to be processed through deduplication product testing is arranged into a single, continuous stream. At least one of a plurality of random modifications are applied to the arranged data in a self-similar pattern exhibiting scale invariance. A plurality of randomly sized subsets of the arranged data modified with the self-similar pattern is mapped into each of a plurality of randomly sized deduplication test files.

Abstract: The present invention overcomes the disadvantages of the prior art by providing a technique that stripes data containers across volumes of a striped volume set (SVS) using one of a plurality of different data placement patterns to thereby reduce the possibility of hotspots arising due to each data container using the same data placement pattern within the SVS. The technique is illustratively implemented by calculating a first index value, an intermediate index value and calculating a hash value of an mode associated with a data container to be accessed within the SVS. A final index value is calculated by multiplying the intermediate index value by the hash value, modulo the number of volumes of the SVS. Further, a Locate( ) function may be used to compute the location of data container content in the SVS to which a data access request is directed to ensure consistency of such content.

Abstract: Multiple data slices are generated from an original data segment. The data slices are constructed to prevent recovery of the original data segment using a single related data slice, but to allow recovery of the original data segment using fewer than all of the data slices. Each data slice is stored in the same memory stripe as the other data slices. The memory stripe extends across multiple memory devices and multiple different distributed storage units. The memory device in which each data slice is stored can be determined based on a source name associated with each data slice.

Abstract: Searching of objects captured by a capture system can be improved by eliminating irrelevant objects from a query. In one embodiment, the present invention includes receiving such a query for objects captured by a capture system, the query including at least one search term. This search term is then hashed to a term bit position using a hash function. Then objects can be eliminated if, in a word index associated with the object, the term bit position is not set.

Abstract: A method and apparatus for unified concurrency control in a Software Transactional Memory (STM) is herein described. A transaction record associated with a memory address referenced by a transactional memory access operation includes optimistic and pessimistic concurrency control fields. Access barriers and other transactional operations/functions are utilized to maintain both fields of the transaction record, appropriately. Consequently, concurrent execution of optimistic and pessimistic transactions is enabled.

Abstract: A method, apparatus and algorithm for quickly detecting an address match in a deeply pipelined processor design in a manner that may be implemented using a minimum of physical space in the critical area of the processor. The address comparison is split into two parts. The first part is a fast, partial address match comparator system. The second part is a slower, full address match comparator system. If a partial match between a requested address and a registry address is detected, then execution of the program or set of instructions requesting the address is temporarily suspended while a full address match check is performed. If the full address match check results in a full match between the requested address and a registry address, then the program or set of instructions is interrupted and stopped. Otherwise, the program or set of instructions continues execution.

Abstract: One embodiment of the present invention includes a method for routing a data object, comprising a sequence of data units, to a particular component data-storage system, or particular group of component data-storage systems, within a distributed, differential electronic-data storage system by selecting one or more subsequences of data units from the data object, computing a characteristic value from the selected subsequences, computing an index from the characteristic value; and directing the data object to the particular component data-storage system, or to the particular group component data-storage systems, identified by the computed index.

Abstract: Embodiments related to fetching instructions and alternate versions achieving the same functionality as the instructions from an instruction cache included in a microprocessor are provided. In one example, a method is provided, comprising, at an example microprocessor, fetching an instruction from an instruction cache. The example method also includes hashing an address for the instruction to determine whether an alternate version of the instruction which achieves the same functionality as the instruction exists. The example method further includes, if hashing results in a determination that such an alternate version exists, aborting fetching of the instruction and retrieving and executing the alternate version.

Abstract: Described embodiments provide a hash processor for a system having multiple processing modules and a shared memory. The hash processor includes a descriptor table with N entries, each entry corresponding to a hash table of the hash processor. A direct mapped table in the shared memory includes at least one memory block including N hash buckets. The direct mapped table includes a predetermined number of hash buckets for each hash table. Each hash bucket includes one or more hash key and value pairs, and a link value. Memory blocks in the shared memory include dynamic hash buckets available for allocation to a hash table. A dynamic hash bucket is allocated to a hash table when the hash buckets in the direct mapped table are filled beyond a threshold. The link value in the hash bucket is set to the address of the dynamic hash bucket allocated to the hash table.

Abstract: Methods and apparatus, including computer program products, implementing and using techniques for generating a hash. A data store including multiple hashing tables is provided. A set of data is received on which the hash is to be based. The set of data includes one or more components. An identifier is received. The identifier identifies one or more hashing tables to use when generating the hash. The received one or more components are processed in accordance with rules defined in the identified one or more hashing tables. The processed components are combined into a final hash.

Abstract: A mechanism is provided in a storage system for efficient inline data de-duplication. The mechanism receives a write command and a hash key for a portion of data to be written from an application host to a write address. The write command indicates whether the application host is tolerant or intolerant to data loss. Responsive to the write command indicating the application host is tolerant to data loss, the mechanism performs a hash key lookup in a hash index. The mechanism determines whether the portion of data has previously been written to the storage system. Responsive to determining the portion of data has previously been written to the storage system, the mechanism stores a pointer to the previously written data at the write address.