Abstract: A computer-implemented method for partition table data writing includes obtaining a first data row to be written into a first partition table and a current partitioning rule indicating at least that the first partition table includes a partitioning key that is a basis for dividing data rows to be written into logical partitions and a first quantity of partitions. A target logical partition corresponding to the first data row is determined. A target physical partition is determined, where the target physical partition has a globally unique partition identifier. The globally unique partition identifier of the target physical partition and an auto-increment identifier of the first data row to the first data row are added as a composite primary key value. The first data row with the composite primary key value is stored to the first partition table.

Abstract: The specification provides an IO request scheduling method. The method includes: obtaining a target IO request to be scheduled, and determining whether the target IO request is a traffic-limited IO request: when the target IO request is a traffic-unlimited first-type IO request, determining a current moment as a scheduling moment of the target IO request: or when the target IO request is a traffic-limited second-type IO request, determining a scheduling moment of the target IO request based on a current resource traffic-limiting proportion; and adding the target IO request to an IO request set, where several IO requests in the IO request set are sequentially sent to a disk for processing in ascending order of scheduling moments.

Abstract: Embodiments of this specification provide data filtering methods, computer-readable media and apparatuses for data queries. The filtering methods can be performed by a target device that serves as a storage node of a database. As an example, a target filtering operator involved in a query is determined. A target data block stored in the target device is obtained. Further, a pre-built operator function set is queried by using a data feature and an encoding feature of the target data block to obtain a target execution function of the target filtering operator. The operator function set is used to record filtering operator execution functions that are suitable for use and that correspond to various data types and various encoding manners. The target data block can be filtered by using the target execution function matched by the query to obtain a filtering result.

Abstract: Storage management by a target node for a distributed database includes sending, except the target node, a block query request to n?1 nodes of n nodes of a distributed jointly using a target memory storing some data blocks. N?1 to-be-reclaimed data block sets are received can combined with a to-be-reclaimed data block set maintained by the target node to obtain a combined data block set which is sent to the n?1 nodes. Each of the n?1 nodes removes a data block from the combined data block set and provides an obtained filtered data block set to the target node. The obtained filtered data block set and an n?1 filtered data block set sent by the n?1 nodes as n target data block sets is used by the target node. An overlapping data block of the n target data block sets is determined. The overlapping data block is reclaimed.

Abstract: Implementations of the present specification describe evicting a memory block in a cache, including determining a temperature index value of a memory block in a cache, where the memory block is a storage unit of the cache. Based on the temperature index value of the memory block in the cache, determining a to-be-evicted memory block that needs to be cleared in the cache. A target element in the to-be-evicted memory block is migrated to a target memory block in the cache, where the target memory block is a memory block that is retained in the cache and whose available storage space is greater than storage space required by the target element. The to-be-evicted memory block is deleted from the cache.

Abstract: A data compression method includes: obtaining a to-be-compressed object; searching a recommendation record for a recommended compression coding rule that meets a compression rate condition, the recommendation record being configured to record a compression coding rule of a historical compressed object and corresponding compression rate information, and the historical compressed object being of a same type as the to-be-compressed object; and if the recommended compression coding rule that meets the compression rate condition is found, compressing the to-be-compressed object by using the recommended compression coding rule; and if the recommended compression coding rule that meets the compression rate condition is not found, starting a regular compression coding process to obtain estimated compression rates of a plurality of compression coding rules for the to-be-compressed object, selecting a target compression coding rule based on at least the estimated compression rates, and compressing the to-be-compressed

Abstract: The present specification discloses a database migration method, including: copying metadata from a first server of a database, the metadata including identification information of synthetic data; receiving a service instruction sent by a client after the copying the metadata from the first server of the database; determining, based on the service instruction, whether the synthetic data needs to be retrieved from the first server of the database; and in response to it is determined that the synthetic data needs to be retrieved from the first server of the database, copying the synthetic data from the first server of the database based on the identification information of the synthetic data. According to the present disclosure, the service instruction sent by the client is received in advance, and the synthetic data is copied from the first server of the database based on the identification information of the synthetic data, thereby reducing time taken for database migration.

Abstract: Implementations of the present application provide a partition splitting method. Data files in a target source partition are divided into at least two data groups. A first data group from the at least two data groups is selected as a target splitting data group. A first splitting step is performed on the target splitting data group, including: splitting the target splitting data group into a plurality of data sub-groups based on a predetermined splitting rule, wherein each data sub-group of the plurality of data sub-groups corresponds to a new partition of at least two new partitions for the target source partition, and assigning each data sub-group to the new partition corresponding to the data sub-group. A second data group from the at least two data groups is selected as the target splitting data group, wherein the second data group is different from the first data group.

Abstract: One or more implementations of the present specification disclose a database analysis information generation method, including: reading data in a to-be-merged data level and data in a merging data level; generating a new merging data level by merging the data in the to-be-merged data level and the data in the merging data level in a merging process; and performing an operation of generating new database analysis information based on data in the new merging data level and data in existing database analysis information in the process of generating the new merging data level.

Abstract: A data compression method includes: obtaining a to-be-compressed object; searching a recommendation record for a recommended compression coding rule that meets a compression rate condition, the recommendation record being configured to record a compression coding rule of a historical compressed object and corresponding compression rate information, and the historical compressed object being of a same type as the to-be-compressed object; and if the recommended compression coding rule that meets the compression rate condition is found, compressing the to-be-compressed object by using the recommended compression coding rule; and if the recommended compression coding rule that meets the compression rate condition is not found, starting a regular compression coding process to obtain estimated compression rates of a plurality of compression coding rules for the to-be-compressed object, selecting a target compression coding rule based on at least the estimated compression rates, and compressing the to-be-compressed

Abstract: Implementations of the present application provide a partition splitting method. Data files in a target source partition are divided into at least two data groups. A first data group from the at least two data groups is selected as a target splitting data group. A first splitting step is performed on the target splitting data group, including: splitting the target splitting data group into a plurality of data sub-groups based on a predetermined splitting rule, wherein each data sub-group of the plurality of data sub-groups corresponds to a new partition of at least two new partitions for the target source partition, and assigning each data sub-group to the new partition corresponding to the data sub-group. A second data group from the at least two data groups is selected as the target splitting data group, wherein the second data group is different from the first data group.

Abstract: The present specification discloses a database migration method, including: copying metadata from a first server of a database, the metadata including identification information of synthetic data; receiving a service instruction sent by a client after the copying the metadata from the first server of the database; determining, based on the service instruction, whether the synthetic data needs to be retrieved from the first server of the database; and in response to it is determined that the synthetic data needs to be retrieved from the first server of the database, copying the synthetic data from the first server of the database based on the identification information of the synthetic data. According to the present disclosure, the service instruction sent by the client is received in advance, and the synthetic data is copied from the first server of the database based on the identification information of the synthetic data, thereby reducing time taken for database migration.

Abstract: One or more implementations of the present specification disclose a database analysis information generation method, including: reading data in a to-be-merged data level and data in a merging data level; generating a new merging data level by merging the data in the to-be-merged data level and the data in the merging data level in a merging process; and performing an operation of generating new database analysis information based on data in the new merging data level and data in existing database analysis information in the process of generating the new merging data level.

Abstract: This disclosure relates to bitmap index writing, dumping, merging, and querying. In one aspect, a method includes determining, based on update data, a to-be-updated row of a bitmap index that indexes data of a database using a bitmap data structure. The to-be-updated row of the bitmap index is locked. The locking includes adding a row lock to the to-be-updated row, thereby preventing the to-be-updated row from being updated by other update operations. At least one other row of the bitmap index remains unlocked for updating by other update operations. Data in the to-be-updated row of the bitmap index is updated based on the update data.

Abstract: Implementations of the present application provide a partition splitting method. Data files in a target source partition are divided into at least two data groups. A first data group from the at least two data groups is selected as a target splitting data group. A first splitting step is performed on the target splitting data group, including: splitting the target splitting data group into a plurality of data sub-groups based on a predetermined splitting rule, wherein each data sub-group of the plurality of data sub-groups corresponds to a new partition of at least two new partitions for the target source partition, and assigning each data sub-group to the new partition corresponding to the data sub-group. A second data group from the at least two data groups is selected as the target splitting data group, wherein the second data group is different from the first data group.

Abstract: This disclosure relates to bitmap index writing, dumping, merging, and querying. In one aspect, a method includes determining, based on update data, a to-be-updated row of a bitmap index that indexes data of a database using a bitmap data structure. The to-be-updated row of the bitmap index is locked. The locking includes adding a row lock to the to-be-updated row, thereby preventing the to-be-updated row from being updated by other update operations. At least one other row of the bitmap index remains unlocked for updating by other update operations. Data in the to-be-updated row of the bitmap index is updated based on the update data.

Abstract: Embodiments provide a data block storage method applied to a database employing an LSM tree. According to the method, data in the 0th layer of the LSM tree is organized in MacroBlocks. When the data in the LSM tree is transferred from an upper layer to a lower layer, the data is transferred to the lower layer in the form of MacroBlocks, so that all layers of the LSM tree are organized in the form of MacroBlocks. When inter-layer data merger is performed, the merger may be performed based on the MacroBlocks, which can avoid rewriting of all data on an entire layer and can effectively reduce disk IO and temporary space consumption. Moreover, as each MacroBlock comprises a plurality of variable-length MicroBlocks, the data may be stored in a storage space in a more compact manner.

Abstract: This disclosure relates to bitmap index writing, dumping, merging, and querying. In one aspect, a method includes determining, based on update data, a to-be-updated row of a bitmap index that indexes data of a database using a bitmap data structure. The to-be-updated row of the bitmap index is locked. The locking includes adding a row lock to the to-be-updated row, thereby preventing the to-be-updated row from being updated by other update operations. At least one other row of the bitmap index remains unlocked for updating by other update operations. Data in the to-be-updated row of the bitmap index is updated based on the update data.

Abstract: This disclosure relates to bitmap index writing, dumping, merging, and querying. In one aspect, a method includes determining, based on update data, a to-be-updated row of a bitmap index that indexes data of a database using a bitmap data structure. The to-be-updated row of the bitmap index is locked. The locking includes adding a row lock to the to-be-updated row, thereby preventing the to-be-updated row from being updated by other update operations. At least one other row of the bitmap index remains unlocked for updating by other update operations. Data in the to-be-updated row of the bitmap index is updated based on the update data.

Abstract: One example method includes receiving, by the one or more processors, a request to update data stored on a distributed data storage system that comprises a primary server and a plurality of secondary servers, the request comprising data updates; sending, by the one or more processors, an update log instruction to the primary server and the plurality of secondary servers so that each of the primary server and the plurality of secondary servers records a log associated with the request; determining, by the one or more processors, that the log has been recorded; and in response to determining that the log has been recorded, sending, by the one or more processors, a data storage instruction to the primary server and a secondary server of the plurality of secondary servers so that each of the primary server and the secondary server stores the data updates.