Abstract: Mechanisms are provided for data transfer reduction in scale out architectures. When a compute node receives a write input/output (I/O) request for a data stream, the compute node separates the data stream into chunks and generates fingerprints for the individual chunks. Fingerprints are then sent to a scale out node and compared to fingerprints of chunks already maintained at the scale out node. Write data transfers are only made for chunks not already maintained at the scale out node. For a read I/O request for a data stream, fingerprints for chunks of the data stream are requested by the compute node from a scale out node. Fingerprints received are compared to fingerprints of chunks already maintained at the compute node and read data transfers are only made for chunks not already maintained at the compute node.

Abstract: A user device includes a flash memory configured to store an index including a plurality of index nodes and a controller configured to control the flash memory. The controller is configured to detect a pointer ID corresponding to a selected key of a first index node, translate the detected pointer ID to an index address by using a pointer table, and access a second index node corresponding to the selected key by using the index address.

Abstract: Concurrent resizing and modification of a first RCU-protected hash table includes allocating a second RCU-protected hash table, populating it by linking each hash bucket of the second hash table to all hash buckets of the first hash table containing elements that hash to the second hash table bucket, and publishing the second hash table. If the modifying comprises insertion, a new element is inserted at the head of a corresponding bucket in the second hash table. If the modifying comprises deletion, then within an RCU read-side critical section: (1) all pointers in hash buckets of the first and second hash tables that reference the element being deleted are removed or redirected, and (2) the element is freed following a grace period that protects reader references to the deleted element. The first table is freed from memory after awaiting a grace period that protects reader references to the first hash table.

Abstract: A system and method are provided for deduplication (dedup) of data file information in a network of distributed data filers. A host, including a metadata service (MDS) and a data node (DN), receives a block count and hash group calculations for a file from a network-connected client device. The MDS creates a file map with target addresses, and compares the calculated hash group to hash groups associated with stored blocks of data in a global dedup hash table. If a match is found, the MDS directs the client device not to send the block of data associated with matching hash group, and reallocates the address of the block of data from the target address.

Abstract: A system and method of creating archive copies of data sets is described. In some examples, the system creates an archive copy from an original data set. In some examples, the system creates an archive copy when creating a recovery copy for a data set. In some examples, the system creates a copy without redundant data, and then encrypts the data set.

Abstract: A management method for reducing the utilization rate of random access memory (RAM) while reading data from or writing data to the flash memory is disclosed. A physical memory set is constructed from a plurality of physical memory blocks in the flash memory. A logical set is constructed from a plurality of logical blocks wherein the data stored in the logical set are stored in the physical memory set. Further, the data stored in each of the logical blocks are stored in one number of physical memory blocks. A mapping table is constructed and includes a hash function, a logical set table, a physical memory set table, and a set status table for managing the relationship among the physical memory sets, physical memory blocks, and logical blocks while reading data from or writing data to the flash memory.

Abstract: Embodiments of the invention relate to associating a source string with a target content unit stored on a content addressable storage (CAS) system. This may be accomplished, in some embodiments, by storing on the CAS system an associative content unit that includes the source string in its binding part and includes the target content unit in its non-binding part.

Abstract: The present invention is directed toward methods and systems for data de-duplication. More particularly, in various embodiments, the present invention provides systems and methods for data de-duplication that may utilize a vectoring method for data de-duplication wherein a stream of data is divided into “data sets” or blocks. For each block, a code, such as a hash or cyclic redundancy code may be calculated and stored. The first block of the set may be written normally and its address and hash can be stored and noted. Subsequent block hashes may be compared with previously written block hashes.

Abstract: A method for data storage includes, in a storage device that communicates with a host over a storage interface for executing a storage command in a memory of the storage device, estimating an expected data under-run between fetching data for the storage command from the memory and sending the data over the storage interface. A data size to be prefetched from the memory, in order to complete uninterrupted execution of the storage command, is calculated in the storage device based on the estimated data under-run. The storage command is executed in the memory while prefetching from the memory data of at least the calculated data size.

Abstract: A method for protecting the integrity of a set of memory pages to be accessed by an operating system of a data processing system, includes running the operating system in a virtual machine (VM) of the data processing system; verifying the integrity of the set of memory pages on loading of pages in the set to a memory of the data processing system for access by the operating system; in response to verification of the integrity, designating the set of memory pages as trusted pages and, in a page table to be used by the operating system during the access, marking non-trusted pages as paged; and in response to a subsequent page fault interrupt for a non-trusted page, remapping the set of pages to a region of the data processing system memory which is inaccessible to the virtual machine.

Abstract: In response to a request including a state object, which can indicate a state of an enumeration of a cache, the enumeration can be continued by using the state object to identify and send cache data. Also, an enumeration of cache units can be performed by traversing a data structure that includes object nodes, which correspond to cache units, and internal nodes. An enumeration state stack can indicate a current state of the enumeration, and can include state nodes that correspond to internal nodes in the data structure. Additionally, a cache index data structure can include a higher level table and a lower level table. The higher level table can have a leaf node pointing to the lower level table, and the lower level table can have a leaf node pointing to one of the cache units. Moreover, the lower level table can be associated with a tag.

Abstract: A method for minimizing cache conflict misses is disclosed. A translation table capable of facilitating the translation of a virtual address to a real address during a cache access is provided. The translation table includes multiple entries, and each entry of the translation table includes a page number field and a hash value field. A hash value is generated from a first group of bits within a virtual address, and the hash value is stored in the hash value field of an entry within the translation table. In response to a match on the entry within the translation table during a cache access, the hash value of the matched entry is retrieved from the translation table, and the hash value is concatenated with a second group of bits within the virtual address to form a set of indexing bits to index into a cache set.

Abstract: A cache system includes a cache having a plurality of cache units, a prediction table and a hashing module. The prediction table is utilized to store way information of at least one cache unit corresponding to at least one accessing address, and the hashing module generates a hashing value corresponding to a target accessing address and reads way information from the prediction table or writes the way information to the prediction table by using the hashing value as an index.

Abstract: Described is using flash memory (or other secondary storage), RAM-based data structures and mechanisms to access key-value pairs stored in the flash memory using only a low RAM space footprint. A mapping (e.g. hash) function maps key-value pairs to a slot in a RAM-based index. The slot includes a pointer that points to a bucket of records on flash memory that each had keys that mapped to the slot. The bucket of records is arranged as a linear-chained linked list, e.g., with pointers from the most-recently written record to the earliest written record. Also described are compacting non-contiguous records of a bucket onto a single flash page, and garbage collection. Still further described is load balancing to reduce variation in bucket sizes, using a bloom filter per slot to avoid unnecessary searching, and splitting a slot into sub-slots.

Abstract: A system and method of creating archive copies of data sets is described. In some examples, the system creates an archive copy from an original data set. In some examples, the system creates an archive copy when creating a recovery copy for a data set. In some examples, the system creates a copy without redundant data, and then encrypts the data set.

Abstract: Memory management of processing systems running in a virtual computer environment and of processes running in an operating system environment includes identifying a usage pattern of a page in memory. The usage pattern is identified by tracking operations conducted with respect to the page. The memory management also includes designating the page as a candidate for sharing when the usage pattern reflects that a number of updates made to the page does not exceed a predefined threshold value. The candidate page is allocated to a first process or virtual machine. The memory management also includes sharing access to the candidate page with a second process or virtual machine when content in the candidate page matches content of page allocated for the second process or virtual machine to an address space of the candidate page.

Abstract: In a computing system having virtualization software including a guest operating system (OS), a method for providing page tables that includes: providing a guest page table used by the guest OS and a shadow page table used by the virtualization software wherein at least a portion of the guest page table and the shadow page table share computer memory; wherein: machine pages have a predetermined size; and the virtualization software maps guest OS physical pages to machine pages at a predetermined alignment.

Abstract: A system, method and computer program product for resizing a hash table while supporting hash table scalability and concurrency. The hash table has one or more hash buckets each containing one or more items that are chained together in a linked list. Each item in the hash table is processed to determine if the item requires relocation from a first bucket associated with a first table size to second bucket associated with a second table size. If the item requires relocation, it is linked to the second bucket without moving or copying the item in memory. The item is unlinked from the first bucket after waiting until there is no current hash table reader whose search of the hash table could be affected by the unlinking, again without moving or copying the item in memory.

Abstract: An apparatus, system, and method are disclosed for improved support of MPS segments in a microprocessor. The virtual address is used to generate possible TLB index values for each of the supported page sizes of the MPS segment associated with the virtual address. The possible TLB index values may be a hash generated using the virtual address and one of the supported page sizes. The TLB is searched for actual TLB index values that match the possible TLB index values calculated using the different supported page sizes. TLB entries associated with those actual TLB index values are checked to determine whether any TLB entry is associated with the virtual address. If no match is found, the real address is retrieved from the PT. The actual page size in the PT is used to generate an actual TLB index value for the virtual address and the TLB entry is inserted into the TLB.

Abstract: The present invention is directed toward methods and systems for data de-duplication. More particularly, in various embodiments, the present invention provides systems and methods for data de-duplication that may utilize a vectoring method for data de-duplication wherein a stream of data is divided into “data sets” or blocks. For each block, a code, such as a hash or cyclic redundancy code may be calculated and stored. The first block of the set may be written normally and its address and hash can be stored and noted. Subsequent block hashes may be compared with previously written block hashes.

Abstract: A configuration management system and related methods are provided to facilitate intersection-based management of configuration information for network devices. In one example, a method includes polling a plurality of network devices to determine address entries stored by address tables of the network devices. The method also includes applying a hash function to the address entries to determine a plurality of key-value associations. The method further includes maintaining a hash table comprising the key-value associations. Each key of the hash table corresponds to an address entry stored by one or more of the address tables. Each value of the hash table identities which of the address tables are storing the address entry corresponding to the associated key.

Abstract: A data processing apparatus is provided comprising a plurality of master devices configured to issue memory access requests including virtual addresses. A memory management unit is configured to receive memory access requests and to translate a virtual address included in a memory access request from a requesting master device into a physical address indicating a storage location in memory. The memory management unit has an internal storage unit having a plurality of entries wherein indications of corresponding virtual address portions and physical address portions are stored. The memory management unit is configured to select an entry of the internal storage unit in dependence on the virtual address and an identifier of the requesting master device. Conflict between the master devices in their usage of the internal storage unit is thus avoided.

Abstract: A lookup unit matrix combines a plurality of lookup units to provide a longest prefix match for a search key longer than the lookup unit's mapper key. A portion of the search key is provided to each of the plurality of lookup units in a single search request issued to the lookup unit matrix. Each lookup unit in the lookup unit matrix performs a multi-level search for the result value based on the portion of the search key forwarded as the mapper key and the result of a multilevel search in the previous lookup unit. The search results in a value corresponding to the search key stored in a single location in one of the lookup units.

Abstract: The application describes a data processor operable to process data, and comprising: a cache in which a storage location of a data item within said cache is identified by an address, said cache comprising a plurality of storage locations and said data processor comprising a cache directory operable to store a physical address indicator for each storage location comprising stored data; a hash value generator operable to generate a generated hash value from at least some of said bits of said address said generated hash value having fewer bits than said address; a buffer operable to store a plurality of hash values relating to said plurality of storage locations within said cache; wherein in response to a request to access said data item said data processor is operable to compare said generated hash value with at least some of said plurality of hash values stored within said buffer and in response to a match to indicate a indicated storage location of said data item; and said data processor is operable to access

Abstract: A method and apparatus for managing memory allocation using memory pages. A first physical memory page is compared with a second physical memory page, wherein the first physical memory page is associated with a first page table and the second physical memory page is associated with a second page table. If the second physical memory page matches the first physical memory page, the second physical memory page is deallocated, and the second page table is associated with the first physical memory page.

Abstract: A compression device recognizes patterns of data and compressing the data, and sends the compressed data to a decompression device that identifies a cached version of the data to decompress the data. Both the compression device and the decompression device cache the data in packets they receive. Each device has a disk, on which each device writes the data in the same order. The compression device looks for repetitions of any block of data between multiple packets or datagrams that are transmitted across the network. The compression device encodes the repeated blocks of data by replacing them with a pointer to a location on disk. The decompression device receives the pointer and replaces the pointer with the contents of the data block that it reads from its disk.

Abstract: Aspects for achieving efficient data access to data elements in a relational database management system are provided. In a computer-implemented method aspect, the efficient data access occurs by establishing a hash table for data elements of a database in a predetermined continuous space of allocated storage, and optimizing utilization of the hash table during database query operations through linear hashing, wherein extension of the hash table occurs automatically to increase a number of pages in the hash table without discernible interruptions of data access to the data elements.

Abstract: The present invention extends to methods, systems, and computer program products for updating Bloom filters. Embodiments of the invention facilitate more efficient use Bloom filters across multiple computers connected across a WAN (potentially having limited bandwidth and latency characteristics), such as, for example, computers located on different continents. The acceptability of false positives is leveraged by allowing the operation of removing items from a set to be batched and delayed. On the other hand, insert operations may be more latency sensitive as a delayed insert results in the semantic equivalent to a false negative. As such, additions to a set are processed in closer to real time to update Bloom filters. In some embodiments, Bloom filters are used to check set membership for electronic mail addresses.

Abstract: A compression device recognizes patterns of data and compressing the data, and sends the compressed data to a decompression device that identifies a cached version of the data to decompress the data. In this way, the compression device need not resend high bandwidth traffic over the network. Both the compression device and the decompression device cache the data in packets they receive. Each device has a disk, on which each device writes the data in the same order. The compression device looks for repetitions of any block of data between multiple packets or datagrams that are transmitted across the network. The compression device encodes the repeated blocks of data by replacing them with a pointer to a location on disk. The decompression device receives the pointer and replaces the pointer with the contents of the data block that it reads from its disk.

Abstract: A method for protecting the integrity of a set of memory pages to be accessed by an operating system of a data processing system, includes running the operating system in a virtual machine (VM) of the data processing system; verifying the integrity of the set of memory pages on loading of pages in the set to a memory of the data processing system for access by the operating system; in response to verification of the integrity, designating the set of memory pages as trusted pages and, in a page table to be used by the operating system during the access, marking non-trusted pages as paged; and in response to a subsequent page fault interrupt for a non-trusted page, remapping the set of pages to a region of the data processing system memory which is inaccessible to the virtual machine.

Abstract: Methods and apparatus for dual hash tables are disclosed. An example method includes logically dividing a hash table data structure into a first hash table and a second hash table, where the first hash table and the second hash table are substantially logically equivalent. The example method further includes receiving a key and a corresponding data value, applying a first hash function to the key to produce a first index to a first bucket in the first hash table, and applying a second hash function to the key to produce a second index to a second bucket in the second hash table. In the example method the key and the data value are inserted in one of the first hash table and the second hash table based on the first index and the second index.

Abstract: A system, method and computer program product for moving a shared list element from a first list location to a second list location includes making the list element appear simultaneously in the first list and the second list under a first key associated with the first list, changing the first key associated with the first list to a second key associated with the second list, and making the list element appear only in the second list under the second key associated with the second list.