Abstract: A big data processing method based on direct computation of compressed data. The method includes 1) compressing, based on a modified Sequitur compression method, original input data according to a smallest compression granularity given by an user, and transforming them into a directed acyclic graph, DAG, consisting of digits; and 2) determining an optimal traversal pattern, and performing a top-downward traversal or a bottom-upward traversal on the DAG in the step 1) based on the determined optimal traversal pattern so as to enable direct processing of the compressed data. By providing a modified Sequitur algorithm and top-downward and bottom-upward traversal strategies in the disclosure, direct processing of compressed data is enabled, significant improvement in time and space has been gained with broad applicability, and certain representations with respect to more advanced document analytics can still be derived on the basis of these.

Abstract: The invention relates to a contribution incentive-based blockchain collaboration method and system, which is characterized in, comprising: 1) applying from an ordinary node to a qualification assessment node for registered qualification; 2) dividing a task into a number of subject tasks and creating corresponding subject task ledgers respectively, and creating a number of sub-tasks corresponding to the subject task in each of the subject task ledgers; 3) applying from the ordinary node to the qualification assessment node for receiving the sub-tasks; 4) by the qualification assessment node, auditing all sub-tasks which are applied for reception by the ordinary node; 5) submitting completed sub-tasks from the ordinary node to a task quality audit node; 6) by the task quality audit node, calculating contribution incentives and qualification credits for the sub-tasks completed by the ordinary node; 7) by the task quality audit node, traversing through each subject task ledger respectively, for finding out a bloc

Abstract: The invention relates to a contribution incentive-based blockchain collaboration method and system, which is characterized in, comprising: 1) applying from an ordinary node to a qualification assessment node for registered qualification; 2) dividing a task into a number of subject tasks and creating corresponding subject task ledgers respectively, and creating a number of sub-tasks corresponding to the subject task in each of the subject task ledgers; 3) applying from the ordinary node to the qualification assessment node for receiving the sub-tasks; 4) by the qualification assessment node, auditing all sub-tasks which are applied for reception by the ordinary node; 5) submitting completed sub-tasks from the ordinary node to a task quality audit node; 6) by the task quality audit node, calculating contribution incentives and qualification credits for the sub-tasks completed by the ordinary node; 7) by the task quality audit node, traversing through each subject task ledger respectively, for finding out a bloc

Abstract: An OLAP query processing method oriented to a database and Hadoop hybrid platform is described. When OLAP query processing is performed, the processing is executed first on a main working copy, and a query processing result is recorded in an aggregate result table of a local database; when a working node is faulty, node information of a fault-tolerant copy corresponding to the main working copy is searched for through namenode, and a MapReduce task is invoked to complete the OLAP query processing task on the fault-tolerant copy. The database technology and the Hadoop technology are combined, and the storage performance of the database and the high expandability and high availability of the Hadoop are combined; the database query processing and the MapReduce query processing are integrated in a loosely-coupled mode, thereby ensuring the high query processing performance, and ensuring the high fault-tolerance performance.

Abstract: An access optimization method for a main memory database based on page-coloring is described. An access sequence of all data pages of a weak locality dataset is ordered by page-color, and all the data pages are grouped by page-color, and then all the data pages of the weak locality dataset are scanned in a sequence of page-color grouping. Further, a number of memory pages having the same page-color are preset as a page-color queue, in which the page-color queue serves as a memory cache before a memory page is loaded into a CPU cache; the data page of the weak locality dataset first enters the page-color queue in an asynchronous mode, and is then loaded into the CPU cache to complete data processing. Accordingly, cache conflicts between datasets with different data locality strengths can be effectively reduced.

Abstract: A concurrent on-line analytical processing (OLAP)-oriented database query processing method is described, for performing, on the basis of predicate vector-based memory OLAP star-join optimization, concurrent OLAP query processing based on a batch query predicate vector bit operation. The concurrent query processing optimization technology is implemented for I/O performance and parallel OLAP processing performance in a database management system, and setting of concurrent OLAP processing load in an optimized way catering to the I/O performance is supported, thereby improving predictable processing performance oriented to diversified OLAP queries and implementing concurrent query star-join bitmap filtering processing based on predicate vector arrays.

Abstract: A multi-dimensional OLAP query processing method oriented to a column store data warehouse is described. With this method, an OLAP query is divided into a bitmap filtering operation, a group-by operation and an aggregate operation. In the bitmap filtering operation, a predicate is first executed on a dimension table to generate a predicate vector bitmap, and a join operation is converted, through address mapping of a surrogate key, into a direct dimension table tuple access operation; in the group-by operation, a fact table tuple satisfying a filtering condition is pre-generated into a group-by unit according to a group-by attribute in an SQL command and is allocated with an increasing ID; and in the aggregate operation, group-by aggregate calculation is performed according to a group item of a fact table filtering group-by vector through one-pass column scan on a fact table measure attribute.

Abstract: An OLAP query processing method oriented to a database and Hadoop hybrid platform is described. When OLAP query processing is performed, the processing is executed first on a main working copy, and a query processing result is recorded in an aggregate result table of a local database; when a working node is faulty, node information of a fault-tolerant copy corresponding to the main working copy is searched for through namenode, and a MapReduce task is invoked to complete the OLAP query processing task on the fault-tolerant copy. The database technology and the Hadoop technology are combined, and the storage performance of the database and the high expandability and high availability of the Hadoop are combined; the database query processing and the MapReduce query processing are integrated in a loosely-coupled mode, thereby ensuring the high query processing performance, and ensuring the high fault-tolerance performance.

Abstract: A concurrent on-line analytical processing (OLAP)-oriented database query processing method is described, for performing, on the basis of predicate vector-based memory OLAP star-join optimization, concurrent OLAP query processing based on a batch query predicate vector bit operation. The concurrent query processing optimization technology is implemented for I/O performance and parallel OLAP processing performance in a database management system, and setting of concurrent OLAP processing load in an optimized way catering to the I/O performance is supported, thereby improving predictable processing performance oriented to diversified OLAP queries and implementing concurrent query star-join bitmap filtering processing based on predicate vector arrays.

Abstract: An access optimization method for a main memory database based on page-coloring is described. An access sequence of all data pages of a weak locality dataset is ordered by page-color, and all the data pages are grouped by page-color, and then all the data pages of the weak locality dataset are scanned in a sequence of page-color grouping. Further, a number of memory pages having the same page-color are preset as a page-color queue, in which the page-color queue serves as a memory cache before a memory page is loaded into a CPU cache; the data page of the weak locality dataset first enters the page-color queue in an asynchronous mode, and is then loaded into the CPU cache to complete data processing. Accordingly, cache conflicts between datasets with different data locality strengths can be effectively reduced.

Abstract: A multi-dimensional OLAP query processing method oriented to a column store data warehouse is described. With this method, an OLAP query is divided into a bitmap filtering operation, a group-by operation and an aggregate operation. In the bitmap filtering operation, a predicate is first executed on a dimension table to generate a predicate vector bitmap, and a join operation is converted, through address mapping of a surrogate key, into a direct dimension table tuple access operation; in the group-by operation, a fact table tuple satisfying a filtering condition is pre-generated into a group-by unit according to a group-by attribute in an SQL command and is allocated with an increasing ID; and in the aggregate operation, group-by aggregate calculation is performed according to a group item of a fact table filtering group-by vector through one-pass column scan on a fact table measure attribute.