Abstract: A deduplication system and method involves interaction between a software-based system that manages the overall background deduplication process and a hardware-based system that includes a hardware-based filesystem manager and a hash generator. A filesystem checkpoint mechanism is leveraged to manage the background deduplication process and also to reduce processing complexity with respect to identifying chunks that are candidates for deduplication and processing such chunks.

Abstract: A computer system can generate a materialized query table (MQT) for data from a base table that has a plurality of rows and columns that each contain stored data. The computer system can receive a query to data stored in a row from the base table. The system can identify a column in the row and determine that data from a cell in the row and the column is above a threshold memory size. The system can generate, in response to determining that the data being above the threshold memory size, an MQT that includes data for the row and that includes a reference pointer that points to a location shared with another table.

Abstract: The information processing device 1 processes document collections having tags permitting semantic class identification appended to each document and comprises a search unit 2, which creates multiple semantic class units containing one, two, or more semantic classes based on a taxonomy that identifies relationships between semantic classes, and a frequency calculation unit 3 which, for each of the semantic class units, identifies documents that match that semantic class unit in the document collections and, for these matching documents, calculates a first frequency that represents the frequency of occurrence in a designated document collection and a second frequency that represents the frequency of occurrence in non-designated document collections. Once the calculations have been performed, the search unit 2 identifies any of the semantic class units based on the first frequency and the second frequency of the matching documents.

Abstract: Disclosed are a method and system for recovering a distributed system from a failure of a data storage unit. The distributed system includes a plurality of computer systems, each having a read-write computer and a data storage unit. Data is replicated from a particular data storage unit to other data storage units using publish-subscribe model. A read-write computer receives the replicated data, processes the data for any conflicts and stores it in the data storage unit. If a data storage unit fails, another data storage unit that has latest data corresponding to the failed data storage unit is determined and the latest data is replicated to other data storage units. Accordingly, the distributed system continues to have the data of the failed data storage unit. The failed data storage unit may be reconstructed using data from one of the other data storage units in the distributed system.

Abstract: A mutable multilevel data structure representing a graph structure may include multiple read-only levels and a single writable level. Each read-only level may include a vertex table (with references to edge tables on the same level or a different level containing elements of adjacency lists for some vertices) and an edge table (with elements of adjacency lists that changed since the previous read-only level). A hybrid variant may switch between a performance-optimized variant (whose edge tables include complete adjacency lists for vertices whose edge sets were modified) and a space-optimized variant (whose edge tables include only newly added adjacency list elements). The vertex tables and/or the writable level may be implemented using copy-on-write arrays, each including an indirection table and multiple fixed-sized data pages. Computations may be run on the read-only levels or on the writable level and read-only levels.

Abstract: Embodiments of the present invention relate to run-length encoded sequences and supporting efficient offset-based updates of values while allowing fast lookups. In an embodiment of the present invention, an indexing scheme is disclosed, herein called count indexes, that supports O(log n) offset-based updates and lookups on a run-length sequence with n runs. In an embodiment, count indexes of the present invention support O(log n) updates on bitmapped sequences of size n. Embodiments of the present invention can be generalize to be applied to block-oriented storage systems.

Abstract: Methods and systems for providing a search engine capability for large datasets are disclosed. These methods and systems employ a Partition-by-Query index containing key-values pairs corresponding to keys reflecting concept-ordered search phrases and values reflecting ordered lists of document references that are responsive to the concept-ordered search phrase in a corresponding key. A large Partition-by-Query index may be partitioned across multiple servers depending on the size of the index, or the size of the index may be reduced by compressing query-references pairs into clusters. The methods and systems described herein may to provide suggestions and spelling corrections to the user, thereby improving the user's search engine experience while meeting user expectations for search quality and responsiveness.

Abstract: A computer system can generate a materialized query table (MQT) for data from a base table that has a plurality of rows and columns that each contain stored data. The computer system can receive a query to data stored in a row from the base table. The system can identify a column in the row and determine that data from a cell in the row and the column is above a threshold memory size. The system can generate, in response to determining that the data being above the threshold memory size, an MQT that includes data for the row and that includes a reference pointer that points to a location shared with another table.

Abstract: Systems and methods for replicating data from a production server to a backup server include recording at least one operation on one or more data items stored in a volume of a production server. The operation may be recorded as at least one journal event in a memory. A determination may then be made regarding whether a system malfunction incident has occurred in the production server and if so, a first set of journal events may be transferred from the memory to an auxiliary storage at a first time instant. At a second time instant, a second set of journal events recorded in the memory between the first and second time instants may be transferred to the auxiliary storage. At one journal event stored in the auxiliary storage unit may then be sent for replication to a backup server.

Abstract: One or more processing platforms are configured to implement at least a first site of a multi-site storage system. The first site comprises a first file system resident at the first site, and a replicated version of a second file system. The second file system is resident at a second site of the multi-site storage system, and the replicated version of the second file system is resident at the first site. Virtual block storage appliances running on respective virtual machines of the first site interact with one another and with similar appliances at the second site to allow the replicated version of the second file system to be synchronized at a cache level with the second file system, and to allow a replicated version of the first file system at the second site to be synchronized at a cache level with the first file system at the first site.

Abstract: Systems, methods, and other embodiments associated with clustering using tri-point arbitration are described. In one embodiment, a method includes selecting a data point pair and a set of arbiter points. A tri-point arbitration similarity is calculated for data point pairs based, at least in part, on a distance between the first and second data points and the arbiter points. In one embodiment, similar data points are clustered.

Abstract: The efficient query processing on ordered views includes: obtaining a view definition for a view for database data; obtaining a query to be executed on the database data; identifying any columns in the view that are unnecessary for the query; after processing any predicates of the query, identifying any sorts in the view definition that are unnecessary or redundant in the query; rewriting the query to eliminate the unnecessary columns and unnecessary and redundant sorts; and executing the rewritten query without materializing the view. Thus, the query is rewritten in a way that avoids materializing the view and eliminates duplicate or unnecessary sorts. This reduces the number of rows to be sorted and eliminates unnecessary columns from materialization. These reductions lead to resource savings, including memory, temporary table space required for sorting, and CPU resources.

Abstract: Techniques for returning estimated value of search keywords of an entire account include, for the entire account, obtaining one or more selected search keywords and their respective forecast periods and parameter settings. An estimated value of a respective search keyword in the respective forecast period is forecasted. Based on stored historical data and parameter settings of the respective search keyword, the estimated value of the respective search keyword is modified to obtain a modified estimated value. The modified estimated value of each search keyword is added up to generate an estimated value of the entire account. The estimated value of the entire account is returned to a client terminal from which the entire account is logged in. The present disclosure modifies the respective search keyword's estimated value so that the estimated value of the entire account satisfies the expected value of the client.

Abstract: A system, method, and/or computer program product for automatic topic identification using a hypertext corpus may include a) receiving a content document(s); b) identifying or lexically scoring candidate topic(s) in the received content document based on label(s) used in a corpus to link to or relate to the candidate topics; c) evaluating or semantically scoring the candidate topic(s) of the received document based on a relationship between two or more candidate topics in the corpus; and d) weighting candidate topics for relevance based on algorithmic or statistical analysis of links or relationships in the corpus.

Abstract: A system, method, and/or computer program product for automatic topic identification using a hypertext corpus may include a) receiving a content document(s); b) identifying or lexically scoring candidate topic(s) in the received content document based on label(s) used in a corpus to link to or relate to the candidate topics; c) evaluating or semantically scoring the candidate topic(s) of the received document based on a relationship between two or more candidate topics in the corpus; and d) weighting candidate topics for relevance based on algorithmic or statistical analysis of links or relationships in the corpus.

Abstract: A system, method, or computer program product for translating non-quantitative, text-based data into a quantitative risk management tool(s) including: receiving, by a computer processor(s), non-quantitative data relating to cross-border public policy; receiving, by the processor(s), at least one tag relating to said non-quantitative data; storing, by the processor(s), said non-quantitative data and said at least one tag in a database; and providing, by the processor(s), quantitative risk management tools designed to provide customized, automatic daily graphical illustrations of policy activity levels on a cross-border basis using concepts and other meta-tagging tools to generate graphs. Tools may mining data to extract quantitative and graphical information from stored, tagged non-quantitative data and may semantically search those documents as well as assess correlations and covariances of cross-border policy processes, and deliver quantitative and/or graphical output results.

Abstract: The author ranking technique described herein is a technique to rank authors in social media systems along various dimensions, using a variety of statistical methods for utilizing those dimensions. More particularly, the technique ranks authors in social media systems through a combination of statistical techniques that leverage usage metrics, and social and topical graph characteristics. In various exemplary embodiments, the technique can rank author authority by the following: 1) temporal analysis of link sharing in which authority is computed based on a user's propensity to provide early links to web pages that subsequently become popular; 2) topical authority based on the author's links and content updates in specific topic areas; and 3) popularity and influence based on nodal properties of authors.

Abstract: Systems and methods are provided for analyzing through one-pass of unstructured time stamped data of a physical process. A distribution of time-stamped unstructured data is analyzed to identify a plurality of potential hierarchical structures for the unstructured data. A hierarchical analysis of the potential hierarchical structures is performed to determine an optimal frequency and a data sufficiency metric for the potential hierarchical structures. One of the potential hierarchical structures is selected as a selected hierarchical structure based on the data sufficiency metrics. The unstructured data is structured according to the selected hierarchical structure and the optimal frequency associated with the selected hierarchical structure, where said structuring of the unstructured data is performed via a single pass though the unstructured data. The identified statistical analysis of the physical process is performed using the structured data.

Abstract: A method for searching multiple documents on a computer system includes steps for sending a query to a system core where the query is passed to a search component for searching the documents. The system core in turn receives results from the search component indicating related documents to the query and passes to a summarization component a specified number of the results. The summarization component processes related documents corresponding to the specified number of results to produce a multi-document summary. The system core receives the summary from the summarization component. The multi-document summary is received from the system core.

Abstract: Techniques are disclosed for effectively reflecting a manipulation of an object in graph data. In one example, a method enables a manipulation performed on an object as an image of a mapping of at least a part of graph data including nodes and edges to be reflected in the graph data. The method includes the step of specifying, from among nodes in the graph data that are related to the object being modified by the manipulation, a shared node that may be related to another object as well, and the step of reflecting the manipulation in the graph data while preventing the shared node from being modified.