Abstract: Methods, systems, and apparatus, including computer program products, for creating test directories and files in a logical storage in a data storage system. The directories include a root directory subdirectories in the root directory. Each directory has a directory key name that has a prefix value; a subdirectory value, the subdirectory value defining a number of immediate subdirectories in the directory; a subdirectory prefix value, the subdirectory prefix value defining a prefix value of each directory key name of each immediate subdirectory in the directory; and a file value, the file value defining a number of files stored in the directory. The contents of any directory can be checked against the directory key name to determine if a data storage error has occurred during testing.

Abstract: Techniques for creating and using a hierarchical data structure, in accordance with embodiments of the present invention include storing received data as records in a first level of the hierarchical data structure. One or more parameters for each block of records in the first level are summarized and stored in a second level of the hierarchical data structure. The techniques may also include querying a given level of a hierarchical data structure. One or more blocks of records one level below the given level are accessed. Each of the accessed blocks correspond to records that are summarized by records in the given level that satisfies the query.

Abstract: A method and system for managing consistent data objects are included herein. The method includes detecting an operation to store a consistent data object. Additionally, the method includes detecting an attribute for the consistent data object. Furthermore, the method includes storing the consistent data object based on the attribute. In addition, the method includes determining an additional format of the consistent data object is to be stored. The method also includes generating a second consistent data object based on the additional format and storing the second consistent data object.

Abstract: A cluster file system comprises a metadata server and a plurality of object storage servers each having one or more object storage targets. The object storage servers and their corresponding object storage targets are arranged in a first storage tier of the cluster file system. The cluster file system further comprises a second storage tier that includes a plurality of nodes of a scale-out network attached storage cluster. The cluster file system is configured to control movement of data between the first and second storage tiers. For example, in some embodiments, the object storage servers comprise one or more components of parallel log-structured file system (PLFS) middleware configured to control migration of data between the first and second storage tiers.

Abstract: A content lifespan management device assigns a life to a first content based on a user preference for the first content. The content lifespan management device extends or shortens a lifespan of the first content based on the user preference of the first content that is calculated after the life is assigned to the first content.

Abstract: Implementations described and claimed herein provide systems and methods for resumable replication. In one implementation, an interruption in a transfer between a source and a target is identified. The transfer includes a send stream generated by the source and configured for receipt by the target. The send stream includes one or more snapshots of the source. A receive checkpoint generated by the target is received at the source. The receive checkpoint indicates data successfully received into the target during the transfer prior to the interruption. The transfer is resumed based on the received checkpoint. The resumed transfer skips any of the one or more snapshots and all data in a resumable snapshot already existing on the target. A complete transfer from the source to the target is verified using a chain of custody.

Abstract: Method and apparatus for locating data on disk storage, wherein multiple instances of data can be stored at different locations to satisfy different use requirements such as read access, write access, and data security. The method allows a data storage system, such as a file system, to provide both read optimized and write optimized performance on disk storage of different types (e.g., sizes and speed).

Abstract: Methods and apparatus for secure data modification using segmented hashing are disclosed. An intermediate device on a data path between a storage service and a client receives a modification request for a data segment of a data chunk of a storage object. The device generates a new chunk hash value for the chunk based on an original chunk hash value of the chunk, an original segment hash value of the segment to be modified, and a new segment hash value computed based on the modification. The device generates a modified version of the chunk based on the modification request, and uploads the modified version and the new chunk hash value to the storage service.

Abstract: Described are computer-based methods and apparatuses, including computer program products, for metadata enhanced filenames. A filename is received, the filename including a first set of characters that represents a name for the electronic file, and a second set of additional characters. The filename is parsed to identify the second set of additional characters. Metadata associated with the electronic file is determined based on the second set of additional characters without downloading additional data of the electronic file. The electronic file is processed based on the determined metadata.

Abstract: A method is used in managing log transactions in storage systems. A first portion of a transaction log space is allocated from a persistent journal associated with a file system to a first core of a multi-core storage processor and a second portion of the transaction log space is allocated from the persistent journal to a second core of the multi-core storage processor. A set of per-core bitmaps is used for allocating a set of transaction log entries from the persistent journal. Each per-core bitmap is associated with a respective core of the multi-core storage processor. A per-core bitmap of the set of per-core bitmaps associated with a core of the multi-core storage processor is used for allocating a transaction log entry for a write request performed by the core.

Abstract: A system and method can support metadata compression in a distributed data grid. The system can store a plurality of keys in compact data structure (such as a radix tree) in a heap, wherein each said key is associated with an entry in a journal off-heap. Furthermore, the system can look up the tree for a key, wherein the key is associated with a journal ticket that represents a virtual address of an entry in the journal. Then, the system can use the journal ticket to manage the entry in the journal that is associated with said key.

Abstract: A method, apparatus, and system for OZIP, a data compression and decompression codec, is provided. OZIP utilizes a fixed size static dictionary, which may be generated from a random sampling of input data to be compressed. Compression by direct token encoding to the static dictionary streamlines the encoding and avoids expensive conditional branching, facilitating hardware implementation and high parallelism. By bounding token definition sizes and static dictionary sizes to hardware architecture constraints such as word size or processor cache size, hardware implementation can be made fast and cost effective. For example, decompression may be accelerated by using SIMD instruction processor extensions. A highly granular block mapping in optional stored metadata allows compressed data to be accessed quickly at random, bypassing the processing overhead of dynamic dictionaries. Thus, OZIP can support low latency random data access for highly random workloads, such as for OLTP systems.

Abstract: A deduplication storage system and a backup application create a synthetic backup. Metadata instructions are provided to the deduplication storage system. Each of the metadata instructions specifies the data segment of an originating backup and a designated location of the data segment in the synthetic backup. Each of the metadata instructions are processed by locating those data sub-segments in the deduplication storage system specified by the data segment in each of the metadata instructions, and creating metadata references to each of the data sub-segments and adding the metadata references to metadata of the synthetic backup being created.

Abstract: Data file in the data deduplication system are associated with a file identifier defined to have a first part identifier for denoting a location of the data file in a storage, and a second part identifier for uniquely identifying the data file in the data deduplication system over time.

Abstract: Aspects of the present invention include a method, system and computer program product for performing data deduplication for eventually consistent distributed data storage (DDS) system. The method includes receiving data content from one or more clients by a DDS system, wherein the one or more clients do not coordinate transmitting of the data content. The method also includes calculating a hash for the data content by the distributed data storage system, writing the data content to an object used for data deduplication, wherein a name of the object is based on the hash and determining whether the data content is present in the distributed data storage system based on the name of an object previously stored on the DDS system. The method further includes keeping track of a number of references to the data content and delaying deletion of the data content for a predetermined period of time.

Abstract: Provided is a method of file synchronizing. The method includes: determining whether there is a changed matter in a file which is divided into a plurality of chunk files; calculating a hash value for each of plurality of divided chunk files when it is determined that there is a changed matter in file; transmitting a hash value list configured by calculated hash values to a server; receiving a transmission request for a chunk file corresponding to a hash value which is not stored in server, among hash values included in hash value list; and transmitting chunk file to the server in response to the transmission request.

Abstract: A method of storing a file in a storage system that includes a plurality of memory-storage hosts includes: providing unique chunk identifiers for memory chunks included in the file; using a hash mapping to identify one or more storage locations for each chunk identifier, each storage location corresponding to a portion of a memory-storage host; and storing each memory chuck at the corresponding one or more storage locations identified by the hash mapping.

Abstract: Embodiments described herein provide techniques for maintaining consistency in a distributed system (e.g., a distributed secondary storage system). According to one embodiment of the present disclosure, a first set of file system objects included in performing the requested file system operation is identified in response to a request to perform a file system operation. An update intent corresponding to the requested file system operation is inserted into an inode associated with each identified file system object. Each file system object corresponding to the inode is modified as specified by the update intent in that inode. After modifying the file system object corresponding to the inode, the update intent is removed from that inode.

Abstract: A computer implemented method and a secure relational file system (SRFS) for storing and managing data for backup and restore are provided. The SRFS receives data, generates first metadata including file-to-sector mapping information, splits the data into fixed sized data chunks (FSDCs), generates second metadata including logical boundaries used for splitting, creates fixed sized data blocks by prepending the second metadata to the FSDCs, splits each FSDC into variable sized data chunks (VSDCs), generates third metadata including unique identifiers (UIDs) for the VSDCs, creates variable sized data blocks (VSDBs) by prepending the third metadata and the second metadata to each VSDC, identifies unique variable sized data chunks (UVSDCs) of the VSDBs using the UIDs, stores the UVSDCs in chunk files, and stores the first metadata, the second metadata extracted from the VSDBs, and storage locations of the UVSDCs with the third metadata of the UVSDCs and duplicate VSDCs in databases.

Abstract: One embodiment of the present disclosure is a method of creating metadata during object development. The method comprises receiving a change to an object during its development that results in a changed version of the object, identifying information about the change, and creating metadata comprising the information about the change. The information about the change may include a unique instance identifier identifying and unique to the changed version of the object. As an object is changed multiple times during development, the created metadata may include a series of information segments each relating to a particular change and each uniquely identified by its unique instance identifier. The information about the change may also include, as examples, an identification of a unique instance identifier of a prior version of the object, the time of the change to the object, and/or identification of the software used to make the change.

Abstract: A computerized method and apparatus is disclosed for dynamic presentation of advertising, factual, informational content and combinations thereof. In particular, the advertising content is dynamically presented according to the playback of corresponding segments identified within a media file or stream.

Abstract: A computerized method and apparatus for providing a virtual media channel based on media search is featured. The method and apparatus features the steps of, or structure for, obtaining a set of rules that define instructions for obtaining media content that comprise the content for a media channel, the set including at least one rule with instructions to include media content resulting from a search; searching for candidate media content according to a search query defined by the at least one rule; and merging one or more of the candidate media content resulting from the search into the content for the media channel. The candidate media content can include segments of the media content resulting from the search. The set of rules can additionally include a rule with instructions to add media content from a predetermined location.

Abstract: A method includes obtaining from an online social media site a plurality of instances of images of objects associated with a person; analyzing with a data processor the plurality of instances of the images with a plurality of predetermined style classifiers to obtain a score for each image for each style classifier; and determining with the data processor, based on the obtained scores, a likely preference of the person for a particular style of object. The plurality of instances of images of objects associated with the person can be images that were posted, shared or pinned by person, and images that the person expressed a preference for. In a non-limiting embodiment the object is clothing, and the style can include a fashion style or fashion genre including color preferences. A system and a computer program product to perform the method are also disclosed.

Abstract: A configured machine performs image matching and retrieval of natural images that may depict logos. The machine generates and uses color-localized spatial masks, which may be computationally less expensive than spatial verification techniques. Key points are detected within images that form a reference database of images. Local masks are defined by the machine around each key point based on the scale and orientation of the key point. To utilize color information presented in logo images, ordered color histograms may be extracted by the machine from locally masked regions of each image. A cascaded index may then be constructed for both visual descriptors and color histograms. For faster matching, the cascaded index maps the visual descriptors and color histograms to a list of relevant or similar images. This list may then be ranked to generate relevant matches for an input query image.

Abstract: A computing device can determine visual words for images in a first set of images associated with a description that includes a term. Each visual word can describe pixels in an image. The device can determine visual words for images in a second set of images that are associated with a description that does not include the term. The device can determine a set of visual words that are associated with the term. The device can determine regions in images corresponding to visual words in the set of visual words. The device can associate the respective determined regions for the images with the term in a database. When a search query is received that includes an image that is determined to be associated with the term, the database can be utilized to determine images responsive to the query by identifying images that are also associated with the term.

Abstract: An example of a method of keyword identification and content overlay is described. A mobile device identifies a keyword within a file containing one or more images. The identified keyword is compared to a collection of pre-stored keywords. If the identified keyword matches one of the pre-stored keywords, content corresponding to the matched pre-stored keyword is overlaid onto the image and the image with the overlaid content is displayed. A user of the mobile device can select the overlaid content to retrieve additional information related to the identified keyword.

Abstract: Architecture that addresses page information lost as part of a selection process in a search engine framework. An aggregation process collects all page or document information from the same image cluster and uses the aggregated page information to annotate one or more selected image-page pairs within the same image cluster. Once the entire set of descriptive terms is received, the entire set of descriptive terms or only an optimum set of top N descriptive terms of the entire set is for annotation of one or more of the representative images in the cluster.

Abstract: Various embodiments are generally directed to creating and using an index based on eye movements of the human eye to store and retrieve images in an image database. An apparatus comprises a processor circuit and a storage communicatively coupled to the processor circuit and storing instructions operative on the processor circuit to receive a first eye movement data associated with a first image provided by the apparatus from an image database stored in the storage; determine a first identity of a first object at a first focus region in the first image indicated by the first eye movement data; search the image database for an image depicting the first object; and provide a second image depicting the first object from the image database. Other embodiments are described and claimed herein.

Abstract: A drift condition, or change, in a data structure can be detected and communicated to one or more subscribers. Data structure can be monitored by periodic configurable polling of a data source or on demand polling. Upon detection of a change in the in the data structure, subscribers can be notified of the change and optionally other information such as the identity of the object that changed and nature of the change.

Abstract: Provided are a token-based database system and method of interfacing with the token-based database system. In accordance therewith, a token database is accessed in a memory. The token database includes a core model, which includes one or more core model tokens that provide default functionality. An extended model is generated in the token database. The extended model includes one or more non-core extended model tokens, which conform to the core model and reference the core model tokens. Thereafter, one or more non-core instance tokens are generated in the token database. The instance tokens conform to the extended model and reference the extended model tokens, wherein each of the instance tokens is configured to have zero or more connections connecting other instance tokens, and wherein the token database comprises a plurality of tokens that includes the core model tokens, the extended model tokens, and the instance tokens.

Abstract: A method, system, and/or computer program product associates new contextual state information about a changed data structure with a beneficial target data. Event data, which describes an event that modifies a data structure, is incorporated into the data structure to create a changed data structure state. New contextual state information, which describes the changed data structure state, is generated and transmitted to multiple beneficial target data stores, wherein applying the new contextual state information to a particular beneficial target data affects an activity that utilizes the particular beneficial target data.

Abstract: A processing platform implemented using one or more processing devices comprises a data fabric layer and a storage infrastructure layer underlying the data fabric layer. The data fabric layer comprises at least one database application having a plurality of nodes. The storage infrastructure layer comprises a plurality of storage servers and is configured to implement one or more virtual storage volumes for each of the nodes of the database application of the data fabric layer using the storage servers. In some embodiments, responsive to detection of a failure of a given one of the nodes of the database application, the one or more virtual storage volumes associated with the given node are utilized to facilitate instantiation of a replacement node for the given node. For example, a virtual storage volume may be unmounted from the given node and attached to the replacement node.

Abstract: Embodiments relate to database systems. An aspect includes deferring row insert operations until occurrence of a triggering event. One method includes receiving a row insert for a tuple into a column group store table, where the tuple includes one or more tuplets and each of the tuplets corresponds to a column group in the column group store table. The method also includes copying at least one of the tuplets into an insert buffer that is specific to one of the column groups in the column group store table. The method also includes deferring the row insert into the column group store table until an occurrence of one or more triggering events. The method also includes flushing the row insert into storage associated with the column group store table, in response to the occurrence of the one or more triggering events.

Abstract: A node search method, comprising: setting a current time value as a traversing time value; searching descendent nodes to perform a traversing from a root node to a leaf node according to a tree index data structure; determining whether a preset traversing condition is satisfied by comparing an update time value, which indicates a time when a change to a searched descendent node is made, with the traversing time value; depending on whether the traversing proceeding condition is satisfied, re-performing a traversing from a parent node or continuing traversing to the searched descendent node; and terminating the traversing at a leaf node where no descendent node exists.

Abstract: Record addressing information retrieval is achieved using a plurality of user data descriptors. When a threshold number of user data descriptors from a set of user data descriptors are received, the threshold number of user data descriptors can be converted into a computed record index that is compared to a list of record indexes associated with a plurality of records. When the computed record index compares favorably to a record index in the list of record indexes, the record addressing information for a particular record is retrieved based on the record index.

Abstract: Embodiments include method, systems and computer program products for data-dependent clustering of geospatial words. In some embodiments, a matrix of words and locations may be obtained. A plurality of locality-sensitive hash (LSH) functions may be generated. A plurality of sparse row vectors may be generated. A plurality of sparse LSH functions may be generated based on the plurality of LSH functions. A clustering bucket may be determined from a plurality of clustering buckets for each of the plurality of sparse row vectors using the sparse LSH functions.

Abstract: An embodiment of the method of processing communication data to identify one or more themes within the communication data includes identifying terms in a set of communication data, wherein a term is a word or short phrase, and defining relations in the set of communication data based on the terms, wherein the relation is a pair of terms that appear in proximity to one another. The method further includes identifying themes in the set of communication data based on the relations, wherein a theme is a group of one or more relations that have similar meanings, and storing the terms, the relations, and the themes in the database.

Abstract: The disclosure is directed to storing data in different tiers of a database based on the access pattern of the data. Immutable data, e.g., data that does not change or changes less often than a specified threshold, is stored in a first storage tier of the database, and mutable data, e.g., data that changes more often than immutable data, is stored in a second storage tier of the database. The second storage tier of the database is more performant than the first storage tier, e.g., the second storage tier has a higher write endurance and a lower write latency than the first storage tier. All writes to the database are performed at the second storage tier and reads on both storage tiers. The storage tiers are synchronized, e.g., the set of data is copied from the second to the first storage tier based on a trigger, e.g., a specified schedule.

Abstract: A system, method, and computer program includes a communications interface configured to receive a set of industry reports from multiple industry sources, and circuitry to compare one or more attributes of at least two trade lines to identify whether the at least two trade lines are duplicates. The circuitry characterizes as a binary indication whether the comparing indicates the one or more attributes are a match, and display a representation of the binary indication and receive a user-identified indication whether the at least two trade lines are duplicates. The circuitry trains a classifier, records the indication whether the at least two trade lines are duplicates and removes at least one of the at least two trade lines from the set of industry reports, and runs the classifier. Subsequently, a supervised machine learning classifier is trained to fit on the training data and is evaluated for accuracy of the testing data.

Abstract: An information retrieval system includes a query revision architecture that integrates multiple different query revisers, each implementing one or more query revision strategies. A revision server receives a user's query, and interfaces with the various query revisers, each of which generates one or more potential revised queries. The revision server evaluates the potential revised queries, and selects one or more of them to provide to the user. A session-based reviser suggests one or more revised queries, given a first query, by calculating an expected utility for the revised query. The expected utility is calculated as the product of a frequency of occurrence of the query pair and an increase in quality of the revised query over the first query.

Abstract: A binary data file embodies an inline tree data structure storing fields of a hierarchical dataset. The inline tree comprises first-level binary string segments, each comprising substantially contiguous second-level binary string segments, corresponding to subranges of first and second subsets of data fields. Size is reduced by substituting: binary string indices for alphanumeric strings; a data clump index for a set of correlated/anticorrelated strings; field masks for unoccupied data fields. A dedicated conversion program generates the inline tree from conventional database formats, which is read entirely into RAM to be searched/filtered by a dedicated search/filter program. Small size (<2 bytes/field/record) and contiguous arrangement enables searching/filtering of >106 records (>100 data fields) in <500 nanoseconds/record/core. Recursive subdivision of selection field ranges can guide searches that include those selection fields.

Abstract: A computer-implemented method for determining a query execution plan based on transaction state may include determining that a first set of database values in a database table are in a committed transaction state and that a second set of database values in the database table are in an uncommitted transaction state. In response to the determining, the first set of database values may have a first set of statistics generated and the second set of database values may have a second set of statistics generated. The first and second set of statistics may be for use in determining a query execution plan.

Abstract: A first iterative Grover Search process is performed by causing at least one quantum computer to perform a first series of Grover Searches according to a characteristic function for varying numbers of iterations to identify a first search target. The characteristic function is modified according to the identified first search target. A second iterative Grover Search process is performed by causing the at least one quantum computer to perform a second series of Grover Searches according to the modified characteristic function beginning with a number of iterations determined based on a number of iterations required for at least one prior iterative Grover Search process to identify a second search target.

Abstract: A method, system, and computer program product is disclosed for caching results in a client-side cache. Embodiments of a method, a system, and a computer program product are disclosed that associate a first snapshot of a database with a client that indicates a state of the database after a last database request by the client, and indicate any number of invalid cached results in the client cache for the client based upon the first snapshot. In some embodiments, the method further includes receiving a second snapshot that indicates a state of the database upon receipt of a database server request by the client, and updating the first snapshot with the second snapshot.

Abstract: A system, computer-implemented method, and a computer-readable storage medium for a data graph traversal are provided. The input parameters for traversing the data graph are received. The data graph having a set of vertices and a set of edges are stored in a column based format in a memory cache of a computer device based on the input parameters is traversed. The traversal generates a set of traversed vertices that are the result of the graph traversal.

Abstract: Certain example embodiments described herein relate to techniques for introducing semantic concepts into complex event processing (CEP) systems and/or methods. In an event processing system, for example, an event bus is configured to receive a stream of events, each having a predefined event type. An event processing agent is configured to execute predefined queries on the events, with each query conforming to a query language, and with the query language being enhanced via a semantic extension corresponding to an ontology. An ontology management component in communication with the event processing agent stores mappings (optionally specified at design time) between concepts of the query language and concepts of the ontology that enhances the query language.

Abstract: A server device is configured to identify sub-pages corresponding to a particular webpage, where the particular webpage and the identified sub-pages may be associated with a same website; determine titles for the identified sub-pages; process the identified sub-pages to select a set of sub-pages, of the identified sub-pages, to use to generate a snippet for the particular webpage, where each webpage, in the set of sub-pages, may satisfy particular criteria, and a quantity of the set of sub-pages may be based on a length of a concatenation of the titles for the set of sub-pages; generate the snippet, for the particular webpage, using the titles for the set of sub-pages; and store the generated snippet in association with the particular webpage, where the generated snippet may be presented in a search result document when the particular webpage is identified as being relevant to a search query.

Abstract: The disclosed embodiments relate to techniques for supporting enhanced content-item searches in an online content-management system. These techniques generally operate by gathering information that is available in the online content-management system (such as how often each content item has been updated, how many users have accessed the content item, and how many identical copies of the content item are stored in the online content-management system) and using this information to identify relevant content items that are likely to be important to a user. This enables the online content-management system to perform searching operations that return relevant content items to the user.

Abstract: A computer-implemented method for processing query information includes receiving data representative of a search query from a user search session. The method also includes identifying a plurality of search results based upon the search query. Each search result is associated with a plurality of user characteristics and data that represents requestor behavior relative to previously submitted queries associated with the respective search result. The method also includes ordering the plurality of user characteristics based upon the data that represents requestor behavior relative to previously submitted queries and the respective search result. The method also includes adjusting the ordered plurality of user characteristics based upon at least one predefined compatibility associated with the user characteristics. The method also includes ranking the search results based upon the adjusted plurality of user characteristics.

Abstract: In one embodiment, a social-networking system accesses a social graph with a plurality of user nodes and a plurality of location nodes, receives a search query with location parameters, identifies a set of location nodes based on the search query, and determines a value for each location nodes in the set based on the edges connected to the location nodes in the social graph.