Abstract: A method for mapping a first address space to a second address space is provided. In some embodiments, the method includes creating a first array of lookup entries and one or more second arrays of metadata entries for maintaining an ordering among the lookup entries using a tree structure. Each of the metadata entries includes one or more data index values identifying a corresponding one of the lookup entries by its position in the first array and one or more metadata index values identifying a corresponding one of the metadata entries by its position in one of the one or more second arrays. The method further includes receiving a request including a lookup value, traversing the tree structure to locate a lookup entry corresponding to the lookup value, and when the lookup value is located among the lookup entries, using the located lookup entry to process the request.

Abstract: Instructions and logic provide extended vector suffix comparisons for Boyer-Moore searches. Some embodiments, responsive to an instruction specifying: a pattern source operand and a target source operand, compare each of m data elements of the pattern operand with each data element of the target operand. A first and second equal ordered aggregation operation are performed from the comparisons according to the m data elements of the pattern source operand. A result of the first and second aggregation operations indicating whether or not a possible match exists between the m data elements of the pattern source operand and d data element positions relative to data elements of the target source operand is stored. Ordering of the data elements of the pattern and the target operands may be reversed for the second aggregation operation, and d may be a sum of m?1 and the quantity of target operand elements in some embodiments.

Abstract: A method and apparatus is disclosed herein for converting a virtual machine from an initial hypervisor format to a target hypervisor format utilizing deduplication metadata. The method may include generating a copy of a virtual machine by replication of deduplication metadata for one or more files corresponding to the virtual machine. Furthermore, the method may include opening the copy of the virtual machine from the replicated deduplication metadata. The method may also include converting the replicated deduplication metadata from an initial hypervisor format to a target hypervisor format to generate a converted virtual machine.

Abstract: An open address probing barrier is utilized in association with a memory container. A starting memory slot is calculated for an item to be found in the memory container. Responsive to the item failing to occupy the starting memory slot, a first predetermined number of memory slots immediately following the starting memory slot are probed. Responsive to the item occupying one of those memory slots, the item is returned to the application. Responsive to the item failing to occupy one of those memory slots, a determination is made as to whether a barrier bit has been set in association with the last of the first predetermined number of memory slots. If so, at least a portion of the memory container is probed for the item and, if found, the item is returned to the application. If the barrier bit is not set, a notification is returned to the application.

Abstract: A hash database configuration method and apparatus, where the method includes establishing an index area and a data area on a disk, where the index area includes p disk pages with an equal size, and receiving an allocation request of key-value pairs, allocating m keys in the key-value pairs to disk pages in the index area, and allocating n values in the key-value pairs to the data area, where m, n, and p are all integers greater than 1, and the efficiency of accessing the hash database is improved.

Abstract: A method for flagging advertisement frames for automatic content recognition is provided. The method includes receiving broadcast fingerprints indicative of broadcast frames of a media stream comprising a series of broadcast scenes. The method also includes receiving advertisement fingerprints indicative of ad frames of ad scenes. The method further includes determining a scene change between a first broadcast scene and a second broadcast scene. The scene change is based on a Pearson correlation coefficient between an initial broadcast fingerprint of an initial broadcast frame of the second broadcast scene and a last broadcast fingerprint of a last broadcast frame of the first broadcast scene. The method also further includes determining whether the second broadcast scene is one of the ad scenes. When the second broadcast scene is one of the ad scenes, the method associates an identification of the second broadcast scene as the one of the ad scenes.

Abstract: A system and method for searching a symmetrically encrypted database for conjunctive keywords is provided. More particularly, the system and method provides means for searching a symmetrically encrypted database for conjunctive keywords by utilizing an encrypted index table and a Bloom filter. The encrypted index table contains encrypted keywords and their associated index pairs while the Bloom filter contains elements that represent encrypted identity-keyword pairs of the database.

Abstract: A network processor has a “bi-level” architecture including a classification algorithm level and a single-record search level to search a hash database that stores packet classification information based on packet field values. The classification algorithm level implements multiple different classification algorithm engines, wherein the individual algorithm applied to a received packet can be selected based on a field of the packet, a port at which the packet was received, or other criteria. Each classification algorithm engine generates one or more single-record search requests to search the hash database for classification information based on one or more field values of the received packet or other classification parameters. Each single-record search requests is provided to the single-record search level, which executes the requests at the hash database and returns the corresponding record to the requesting classification algorithm engine.

Abstract: The invention is an eighth layer cloud administrative framework that provides for both monitoring users and providing needed software. The cloud framework guides software tool implementation so that users can accomplish business tasks efficiently.

Abstract: Embodiments relating to garbage collection for a deduplicated and compressed storage device are described. One embodiment provides for a computer implemented method including creating a multiple sets of Bloom filters distributed across a set of multiple computing device nodes. One set of Bloom filters stores differing ranges of fingerprints for data stored on deduplicated storage containers, while a second set of Bloom filters includes a set of fingerprints correlated with files in a file system directory structure. A set of live fingerprints is determined for live fingerprints and storage segments associated with those fingerprints are copied to new storage containers.

Abstract: An entries array having a pre-selected number of initially empty buckets and structured as a circular array is maintained. Each of the pre-selected number of entries in the entries array has a corresponding index value. A chaining array having the pre-selected number of buckets and structured as a circular array is also maintained. Each of the pre-selected number of entries in the chaining array has the same corresponding index value as the corresponding bucket in the entries array. A hash operation is performed on data to be stored to determine a hash value corresponding to the data. The data is stored in a bucket in the entries array corresponding to the hash value as an unmovable head-of-chain entry, and moving previous data, if any, stored in the entries array corresponding to the hash value if the previous data has a different hash value. The data is stored in another bucket in the entries array if the bucket in the entries array corresponding to the hash value is occupied by a head-of-chain entry.

Abstract: Power management of a routing table is provided by supporting various power domain configurations. Each power domain configuration can be associated with a different number of power domains than other power domain configurations. Efficient power management can be achieved by switching between a lower power domain configuration and a higher power domain configuration during run-time based on the dynamic load conditions.

Abstract: A consistency checker is configured to perform repairs to one or more multi-level dense tree metadata structures shared between volumes managed by a volume layer of a storage input/output (I/O) stack executing on one or more nodes of a cluster. The volumes include a parent volume and a snapshot and/or clone, wherein the snapshot/clone may be represented as an independent volume, and embodied as a respective read-only copy (snapshot) or read-write copy (clone) of the parent volume. Illustratively, the consistency checker verifies and/or fixes (i.e., repairs) on-disk structures of the volume layer, e.g., the shared dense tree, according to a distributed repair procedure that maintains consistency properties across all volumes in a volume family and avoid cyclic repairs made in the context of different volumes sharing the dense tree.

Abstract: A data processing method comprises: using an application server computer, receiving a workflow for single period processing, the workflow comprising steps for processing data items related to the period, processing the data items according to the workflow to generate an output, capturing, during processing, metadata about the output, storing the output, the workflow, the data items, and the metadata in a database, receiving a modification to one or more of the data items and storing the modification to result in modified data items, receiving, during an open period, a request to reprocess the period using the modified data items, wherein the open period is subsequent to the SPP, in response to the request, processing, according to the workflow and the metadata, the modified data items to generate modified output, determining a difference between the output and the modified output, carrying the difference over to the open period.

Abstract: A probabilistic counting structure such as a hyperloglog may be formed during a table scan for each of a selected set of columns. The columns may be selected based on an initial estimate of relatedness, which may be based on data types of the respective columns. An estimated cardinality of an intersection or union of columns may be formed based on an intersection of the probabilistic data structures. A join path may be determined based on the estimated cardinality of an intersection or union of the columns.

Abstract: A system for searching files across clients includes a server coupled to multiple clients. A first client creates a first index of files that are stored on the first client and uploads the first index to the server. A second client creates a second index of files that are stored on the second client. The second client downloads the first index from the server and executes a search across the first index and the second index. A method for searching files across clients includes generating and storing on a first client an index of locally stored files; uploading the index from the first client to a server; storing the index on the server; downloading the index from the server to the second client; and performing a search for files stored on the first client by querying the index downloaded onto the second client.

Abstract: Disclosed herein are system, method, and computer program product embodiments for constraining the amount of memory used during data aggregation. An embodiment operates by separating input data into a plurality of partitions. The embodiment then inserts portions of the input data into blocks from a free list at a given level of a pipeline. The embodiment then inserts the blocks into buffers for processing at a subsequent level of the pipeline. The embodiment processes the inserted blocks at the subsequent level of the pipeline and concatenates the intermediate results into a final aggregate result.

Abstract: A computer-implemented method for generating probabilistically searchable messages includes obtaining, by a computing device, a textual message. The method includes extracting, by the computing device, from the textual message, a plurality of words. The method includes cryptographically hashing, by the computing device, each word of the plurality of words. The method includes mapping, by the computing device, each cryptographically hashed word to a plurality of tokens. The method includes generating, by the computing device, a set of tokens associated with the textual message, the set of tokens including each plurality of tokens associated with each cryptographically hashed word. The method includes storing, by the computing device, the set of tokens associated with the textual message in metadata associated with the textual message.

Abstract: A system for estimating a quantity of unique identifiers comprises a processor and a memory. The processor is configured to, for each of k times, associate a bin of a set of bins with each received identifier. The processor is further configured to determine an estimate of the quantity of unique identifiers based at least in part on an average minimum associated bin value. The memory is coupled to the processor and configured to provide the processor with instructions.

Abstract: A data storage system may implement rate-limiting secondary index creation for an online table. A secondary index may be generated for a table stored in a data store. The table may be incrementally indexed, maintaining the updates determined according to indexing different portions of the table in a queue of pending updates that are subsequently applied at the secondary index. Prior to indexing a portion of the table, an evaluation of a current number of pending updates in the queue of pending updates may be performed with respect to a throttle threshold. If the current number of pending updates exceeds the throttle threshold, then indexing the portion of the table may be throttled. Received updates to previously indexed portions of the table, may be applied to the table and placed in the queue of pending updates without an evaluation of the current number of pending requests.

Abstract: An indexing system uses a cascade of hash structures to process data entries upon ingest for indexing. The indexing system may be used for enhancing database privacy, anonymization, or data compression. A hash structure, for example, a bloom filter or hash table, passes a representation of the data entries to a subsequent hash structure in the cascade responsive to determining that the hash structure previously filtered an instance of the same representation. The indexing system can generate the representations of the data entries using one or more hash functions. A terminal hash structure of the cascade may index the data entries responsive to determining that the data entries satisfy a criteria for anonymization. For instance, the indexing system determines that there exists a threshold number of data entries describing a population of subjects having the same combination of data elements.

Abstract: Disclosed are systems, methods, and computer-readable storage media for load balance resources amongst computing nodes. Various embodiments of the present technology can be used to, prior to assigning a first data shard included in a first cluster of data shards as a leader of the first cluster of data shards, determine whether a first computing node storing the first data shard also stores a second data shard designated as a leader of a second cluster of data shards, yielding a determination, and when the determination indicates that the first computing node stores the second data shard designated as the leader of the second cluster of data shards, designate the first data shard as non-electable to be the leader of the first cluster of data shards and electing an alternate data shard included in the first cluster of data shards as the leader of the first cluster of data shards.

Abstract: A current file is obtained in the data. It is determined whether a similar historical file exists based on a sampled data block from at least one predetermined location in the current file. In response to non-existence of the similar historical file, the current file and corresponding metadata are stored on a file basis. In response to existence of the similar historical file, a deduplication operation is applied on the current file on a block basis.

Abstract: A computer method and system provide means for recommending an organization to a user. The system includes a database of business relationships between organizations. A user may search for an organization according to search criteria and the system will select a set of organizations that match the criteria. The organizations may be displayed based on the relevance of the selected organization and the similarity of organizations connected to the selected organizations.

Abstract: Aggregating input into hashtables using just-in-time compilation of compilable code in response to a database query. Compilable code can be generated that is configured to cause a programmable processor to produce one or more hashmaps based on the input database. The one or more hashmaps can correspond to each individual thread from the input. The compilable code can be configured to cause the one or more processors to insert the hashmaps into a scheduler. Compilable code can be generated that is configured to: aggregate elements from the one or more hashmaps into buckets of elements having the same partition identity; rehash the buckets of elements having the same partition identity to reduce the number of groups within the bucket; facilitate the merger of all non-empty elements from each target-partition into a merged-partition.

Abstract: A geographic coordinate encoding device including a processing unit, wherein the processing unit performs: a first encoding process of determining, from among a plurality of areas obtained by hierarchically dividing a geographic range, an area specified by geographic coordinates of at least two points detected by a sensor, and encoding data representing the determined area; and an output process of outputting data representing a relationship between the specified area and the coordinates of the at least two points, along with code data encoded by the first encoding process.

Abstract: Power management of a routing table, which comprises a plurality of hash tables, is provided by supporting various power domain configurations. Each power domain configuration can be associated with a different number of power domains than other power domain configurations. Each power domain can add vertical capacity to the entire routing table by adding a fixed number of buckets to each of the hash tables. Efficient power management can be achieved by switching between a lower power domain configuration and a higher power domain configuration during runtime based on the dynamic load conditions.

Abstract: Systems, methods, and media for managing an in-memory NoSQL database are provided. In some embodiments, systems for managing an in-memory NoSQL database are provided, the systems comprising a hardware processor that is configured to: identify a group of keys capable of being included in a complex multi-key database command, the group including a first key and a second key, the first key being associated with a first value and the second key being associated with a second value, the first value and the second value being stored on different servers that are part of the database; and cause the first value and the second value to be both stored on a given server.

Abstract: A deduplication memory module, which is configured to internally perform memory deduplication, includes a hash table memory for storing multiple blocks of data in a hash table array including hash tables, each of the hash tables including physical buckets and a plurality of virtual buckets each including some of the physical buckets, each of the physical buckets including ways, an address lookup table memory (ALUTM) including a plurality of pointers indicating a location of each of the stored blocks of data in a corresponding one of the physical buckets, and a buffer memory for storing unique blocks of data not stored in the hash table memory when the hash table array is full, a processor, and memory, wherein the memory has stored thereon instructions that, when executed by the processor, cause the memory module to exchange data with an external system.

Abstract: An apparatus sets a layer counter to point to a first layer of a data structure. The apparatus determines the layer counter to reference an overflowing cell. The apparatus increments the layer counter to point to a second layer of the data structure. The apparatus determines the incremented layer counter to reference a non-overflowing cell. The apparatus increments a value of the non-overflowing cell, wherein the first layer is stored in a first cache and the second layer is stored in a second cache, and wherein the first cache differs from the second cache with respect to one or more of speed or size.

Abstract: Techniques to provide secure cloud-based storage of data shared across file system objects and clients are disclosed. In various embodiments, a primary encryption key is determined for an object associated with a plurality of component chunks of file system data. The primary encryption key is used to generate for each of said component chunks a corresponding chunk key, based at least in part on the primary encryption key and data comprising or otherwise associated with the chunk. The respective chunk keys are provided to a file system client configured to create and store the object at least in part by encrypting each chunk included in the plurality of component chunks using the chunk key provided for that chunk to generated encrypted chunk data, and combining the encrypted chunk data to create and store the object.

Abstract: Disclosed are devices, systems, and methods for the use of memory including a data table configured to store a plurality of elements, wherein the plurality of elements are arranged into a plurality of buckets and each of the plurality of buckets comprising a plurality of entries. A first power domain can be associated with an entry of each bucket or with a first bucket. A second power domain can be associated with a second entry of each bucket or a second bucket. Processing logic can be configured to search for a particular value stored in an element of the plurality of elements by selecting buckets of the plurality of buckets and selecting at least one entry of each of the buckets. A programmable register can be used to select a powered state of the second power domain based on a configuration of the programmable register.

Abstract: A method for entity recognition employs document-level entity tags which correspond to mentions appearing in the document, without specifying their locations. A named entity recognition model is trained on features extracted from text samples tagged with document-level entity tags. A text document to be labeled is received, the text document being tagged with at least one document-level entity tag. A document-specific gazetteer is generated, based on the at least one document-level entity tag. The gazetteer includes a set of entries, one entry for each of a set of entity names. For a text sequence of the document, features for tokens of the text sequence are extracted. The features include document-specific features for tokens matching at least a part of the entity name of one of the gazetteer entries. Entity labels are predicted for the tokens in the text sequence with the named entity recognition model, based on the extracted features.

Abstract: Data matches are calculated in a data deduplication system by matching input and repository digests using a digest based data matching process where the reference digests corresponding to a repository interval of data identified as similar to an input interval of data are loaded into two data structures. The two data structures include a sequential buffer containing digests in a sequence of occurrence in the data and a search structure for searching of the reference digests matching a version digest.

Abstract: The present disclosure involves systems, software, and computer implemented methods for providing a bucket skiplist. In one example, a method comprises identifying a value to be searched for within a skiplist comprising a data structure for referencing a plurality of ordered nodes, the data structure having a plurality of linked lists, each node representing at least one memory block, wherein each node is associated with a bucket including a set of ordered values, where the first value in each bucket of each node is the relatively lowest value in the bucket, comparing the value to the first values in each bucket to identify a particular node in which the first value may be located, and, in response to determining that the first value of the particular node is not the same as the value to be searched, comparing the value to be searched with the ordered values in the bucket.

Abstract: In one embodiment, forwarding information with respect to a particular data packet is requested from a distributed hash table (DHT) that stores key-network flow information pairs. A plurality of nodes in a network participate in the DHT, each node locally stores network flow information as dictated by a hashing function of the DHT, and the network flow information can be used to process data packets in the network. Then, forwarding information is received from the DHT indicating a destination node of the plurality of nodes to which the particular data packet should be forwarded according to the hashing function of the DHT. The destination node stores network flow information that corresponds to the particular data packet. Finally, the particular data packet is forwarded toward the destination node in order for the particular data packet to be processed using the network flow information stored at the destination node.

Abstract: Embodiments for calculating a representative value for an input similarity unit in data deduplication of snapshots data by a processor. A corresponding similarity unit of a previous snapshot is identified. A calculation based on digests of the input similarity unit and digests of the corresponding similarity unit is performed. A representative value is produced for the input similarity unit.

Abstract: Partitioned join operations are performed between a first database object and a second database object by determining an agent group for an agent in response to the agent receiving rows of the second database object to process; partitioning the rows to determine a target hash table for each row and adding the partitioned rows to work performed by the agent group; and distributing the work for the group to agents of the group by assigning to a single agent all the rows associated with a particular hash table to perform a join operation on the assigned rows. Each partition is assigned a first counter value indicating an upper bound of a task id range that is most recently assigned to an agent in the agent group for processing, and a second counter value indicating the highest task id that has been processed for that partition.

Abstract: According to an aspect, a key-value store (KVS) system includes a data management unit that stores a data KVS storing a pair of a data KVS key including information on a storage location of application data to be an access target object and the application data; and a key KVS storing a pair of an application key and the data KVS key. The data KVS includes a normal partition in which a size of a record for storing one pair is a predetermined specific size; and a special partition in which the size of the record for storing one pair is a size set according to a data size of the pair to be stored. A data relocation unit relocates a pair of a relocation target object to the special partition having the record size suitable for the data size of the pair.

Abstract: Aspects of the present invention are directed to system and methods for optimizing identification of locations within a search area using hash values. A hash value represents location information in a single dimension format. Computing points around some location includes calculating an identification boundary that surrounds the location of interest based on the location's hash value. The identification boundary is expanded until it exceeds a search area defined by the location and a distance. Points around the location can be identified based on having associated hash values that fall within the identification boundary. Hashing operations let a system reduce the geometric work (i.e. searching inside boundaries) and processing required, by computing straightforward operations on hash quantities (e.g. searching a linear range of geohashes), instead of, for example, point to point comparisons.

Abstract: The embodiments described herein are directed to the use of hashing in a file system metadata arrangement that reduces an amount of metadata stored in a memory of a node in a cluster and that reduces the amount of metadata needed to process an input/output (I/O) request at the node. Illustratively, the embodiments are directed to cuckoo hashing and, in particular, to a manner in which cuckoo hashing may be modified and applied to construct the file system metadata arrangement. In an embodiment, the file system metadata arrangement may be illustratively include a hash collision technique that employs a hash collision computation to determine a unique candidate extent key (having a candidate hash table index) in the event of a collision, i.e., a hash table index collides with a slot of a hash table matching a key found in the slot.

Abstract: A system and method for accessing a hash table are provided. A hash table includes buckets where each bucket includes multiple chains. When a single instruction multiple data (SIMD) processor receives a group of threads configured to execute a key look-up instruction that accesses an element in the hash table, the threads executing on the SIMD processor identify a bucket that stores a key in the key look-up instruction. Once identified, the threads in the group traverse the multiple chains in the bucket, such that the elements at a chain level in the multiple chains are traversed in parallel. The traversal continues until a key look-up succeeds or fails.

Abstract: Embodiments are disclosed for using an improved locality sensitive hashing (LSH) operation for the K-means clustering algorithm. In some embodiments, parameters of an LSH function are optimized with respect to a new cost model. In other embodiments, an LSH operation is applied with optimized parameters to a K-means clustering algorithm.

Abstract: For on-demand migration of data in a distributed memory storage configuration, an identifier is transformed at a client into a transformed identifier. From a current configuration of a first plurality of servers operating on a server-side at a current time, a current server is identified at the client. From a previous configuration of a second plurality of servers operating on a server-side at a previous time, a previous server is identified at the client. A first request is sent to the current server to perform an operation using the identifier. A second request is sent to the previous server to perform the operation using the identifier. When a first data in response to the first request is invalid and a second data in response to the second request is valid, the second data is migrated to the current server in a migration request from the client to the current server.

Abstract: Data to be exchanged in a transaction is protected by using a hashing function to transform sensitive data into a scrambled form. The hashing function can use as its inputs data derived entirely or in part from portions of the data to be exchanged. The hashing function can also use as its inputs data derived entirely or in part from other locally available data. The output of the hash function is then transmitted to a destination that attempts to recreate the output of the hash function in order to validate the data transmission.

Abstract: A processor includes a hash register and a hash generating circuit. The hash generating circuit includes a novel programmable nonlinearizing function circuit as well as a modulo-2 multiplier, a first modulo-2 summer, a modulor-2 divider, and a second modulo-2 summer. The nonlinearizing function circuit receives a hash value from the hash register and performs a programmable nonlinearizing function, thereby generating a modified version of the hash value. In one example, the nonlinearizing function circuit includes a plurality of separately enableable S-box circuits. The multiplier multiplies the input data by a programmable multiplier value, thereby generating a product value. The first summer sums a first portion of the product value with the modified hash value. The divider divides the resulting sum by a fixed divisor value, thereby generating a remainder value. The second summer sums the remainder value and the second portion of the input data, thereby generating a hash result.

Abstract: A system and machine-implemented method including receiving a request for a set of content items within a session, identifying, in response to the request, one or more content items, determining if the session is associated with a continuation token, wherein the continuation token provides information regarding the content items provided for display during the session, generating a continuation token for the session when the session is not associated with a continuation token, providing at least one of the one or more content items for display and updating the continuation token with information regarding the at least one of the one or more content items provided for display and a time stamp.

Abstract: Described is a technique for securely verifying access rights to a media file stored on a user device. By verifying the access rights to the media file, a server may provide access to a licensed version of the media from a media library in a remote location such as server for a cloud-based service. When a media file is confirmed to be available in a media library, a cryptographic a hash function that incorporates a random value or “salt” is used to verify that a user is in possession of an entire media file. Accordingly, the techniques described herein improve the security for verifying that a user is in possession of a media file by preventing the use of pre-calculated hashes.

Abstract: A computer-implemented method for generating an identifier for a device includes identifying one or more applications from a plurality of applications installed on a device; generating an identifier for the device based on the one or more identified applications; and providing the generated identifier as identification for the device. Another computer-implemented method for identifying a device includes: receiving information that indicates one or more applications of a plurality of applications installed on a device; and identifying the device using the received information.

Abstract: The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.