Abstract: A database management system configured to perform a computer-implemented method is provided. The method involving configuring a first set of nodes to determine an instantaneous state of the database management system, generate system configuration data, and provide the system configuration data to a second set of nodes of the database management system, wherein the configuration data is replicated amongst the first set of nodes according to a consensus-based replication protocol so as to maintain the system configuration data. The method also involves configuring the second set of nodes to receive the system configuration data and implement a state machine that is arranged to modify a configuration of the second set of nodes according to the system configuration data, wherein the second set of nodes are arranged to replicate user data according to a non-consensus-based replication protocol.

Abstract: Disclosed herein is a method of managing data in a database management system comprising a first, master storage, node and a second node for performing operations on data stored in the database. The method comprises configuring the second node to perform a set of steps. The second node is configured to write data indicative of one or more operations on data to a set of log files. The second node segments the set of log files into a set of log file chunks For each of the log file chunks, the second node may transmit data from the database associated with a log file chunk to the first node and thereafter transmit the log file chunk to the first node. The segmenting and transmitting of the set of log files and their associated data may be performed asynchronously to the updating of the set of log files.

Abstract: There is provided a method of using a bitmap filter to filter a data source as part of a join operation. A set bit at a bit location of the bitmap filter is associated with an entry of the data source that comprises an identifier that matches a filter parameter. A row identifier of a row in another data source that corresponds to the identifier is determined based on a sum of a number of bits set in bit locations that are between the bit location of the set bit associated with the entry of the data source and a bit location in the bitmap filter corresponding to a different, known, row identifier of a different row in the another data source. The row of the another data source corresponding to the determined row identifier is interrogated to obtain information defining the identifier that matches the filter parameter.

Abstract: In an example there is provided a computer-implemented method which comprises generating an execution plan for a received user query, converting the execution plan into bytecode, compiling to unoptimized machine code using the bytecode and beginning execution of the execution plan by executing the unoptimized machine code, compiling optimized machine code using the bytecode whilst executing the unoptimized machine code; and switching to executing the optimized machine code in order to execute the execution plan, when the optimized machine code has been compiled.

Abstract: A method for managing log files for recording operations on data stored in a database is provided, wherein a set of log files having an allocated first portion of storage is updated, the allocated first portion of storage is monitored, and a second portion of storage is allocated in dependence on a determination that an available portion of storage is below a predetermined size. A method for generating a snapshot is provided, wherein data entries are included in the snapshot in dependence on a determined relative order of log records in a set of log files. A method of replicating a binary large object is provided, wherein the binary large object is sent to a second database in response to identifying a log record comprising data indicating the binary large object.

Abstract: A database system and a computer implemented method for managing functions in the database system is provided. The method, implemented using instructions that may be stored in the database system, involves obtaining first data representing a set of one or more operations to be performed on data in the database system. Second data, representing a function definition, is derived from the first data and the function definition includes the set of one or more operations and a set of state variables. A function is generated according to the second data. Generating the function includes storing a set of one or more values for respective ones of the state variables. The set of values are stored in an isolated computing environment in the database system and the function is configured to operate on the values in the isolated computing environment.

Abstract: Disclosed herein is a computer-implemented method of processing a query using a columnstore comprising a plurality of segments, the method comprising: receiving a query comprising a parameter; determining a key corresponding to the parameter; identifying a mapping structure, from a plurality of mapping structures, relating to said key, wherein each mapping structure corresponds to a respective segment of the columnstore; interrogating the identified mapping structure to determine a value corresponding to the key, wherein the value identifies an entry of an index table of a plurality of index tables that corresponds to the respective segment, wherein the entry of the index table identifies a row of the respective segment of the columnstore relating to the parameter; and retrieving data relating to the parameter from a data source based on data stored in the entry of the index table.

Abstract: A method for causing a processor to perform a query on a column-store table of encoded values is provided. The method includes configuring the processor to receive the query, comprising a filter to be applied to at least a first column vector of the encoded values. The processor processes the query for the encoded values in the first column vector, to generate a first vector indicative of respective encoded values passing or failing the filter. The processor determines, from the first vector, an indicator of encoded values passing the filter and encoded values failing the filter, relative to the encoded values in the first column vector. The processor determines a bit length of the encoded values. The processor selects an algorithm for processing the query based on the indicator and the determined bit length of the encoded values. A system and a non-transitory, computer-readable medium are also provided.

Abstract: There is provided a system of generating a bitmap filter as part of a join operation. A query comprising a filter parameter is received and a first data source associated with the filter parameter is queried to identify an entry therein with an identifier corresponding to the filter parameter. Each entry of the first data source is identified by a corresponding identifier to differentiate the entries from one another. The system generates a bitmap filter corresponding to the filter parameter by identifying a single bit location in the bitmap filter that corresponds to an identifier of the entry corresponding to the filter parameter. Each identifier has a numerical value used to identify the bit location. A single bit location is assigned to the identifier and a bit is set at the assigned bit location such that there is one-to-one mapping between the identifier and the assigned bit location.

Abstract: A non-transitory computer-readable medium is provided. The medium comprises a set of instructions, which, when executed by a processing system associated with a database or data warehouse, causes the processing system to retrieve data from a data source in accordance with a mapping between a first set of partitions and a second set of partitions, the first set of partitions being associated with the data source and the second set of partitions being associated with the database. The set of instructions, when executed by the processing system, further causes the processing system to load the retrieved data into the database. Retrieving the data and loading the retrieved data comprise a single logical unit of work. A database system and a method executed by a processing system associated with a database are also provided.

Abstract: A method for managing log files for recording operations on data stored in a database is provided, wherein a set of log files having an allocated first portion of storage is updated, the allocated first portion of storage is monitored, and a second portion of storage is allocated in dependence on a determination that an available portion of storage is below a predetermined size. A method for generating a snapshot is provided, wherein data entries are included in the snapshot in dependence on a determined relative order of log records in a set of log files. A method of replicating a binary large object is provided, wherein the binary large object is sent to a second database in response to identifying a log record comprising data indicating the binary large object.

Abstract: A computer-implemented method of evaluating a set of filter parameters being represented by a filter tree comprising a plurality of nodes. The method can include identifying whether a node is a root of a sub-tree comprising other nodes of the filter tree; generating a cost for said node by processing a sample input comprising a plurality of data items of a data source using the filter parameter and measuring the time taken for the plurality of data items to be processed; and determining a selectivity of said node based on an output of its filter parameter as a result of processing the sample input using the filter parameter; then ordering at least some of the plurality of nodes of the filter tree having the same parent node based on their relative costs and selectivities, for use in generating an ordered filter tree.

Abstract: In an example there is provided a computer-implemented method comprising generating an execution plan for a received user query in a structured query language, the execution plan having a number of operators for operating on data from a content database; compiling the operators into corresponding byte code and machine code; executing the operators by first interpreting the corresponding byte code; switching to executing the operators by executing the corresponding machine code when compiled.

Abstract: A method for causing a processor to perform a query on a column-store table of encoded values is provided. The method includes configuring the processor to receive the query, comprising a filter to be applied to at least a first column vector of the encoded values. The processor processes the query for the encoded values in the first column vector, to generate a first vector indicative of respective encoded values passing or failing the filter. The processor determines, from the first vector, an indicator of encoded values passing the filter and encoded values failing the filter, relative to the encoded values in the first column vector. The processor determines a bit length of the encoded values. The processor selects an algorithm for processing the query based on the indicator and the determined bit length of the encoded values. A system and a non-transitory, computer-readable medium are also provided.

Abstract: Disclosed herein is a computer-implemented method of processing a query using a columnstore comprising a plurality of segments, the method comprising: receiving a query comprising a parameter; determining a key corresponding to the parameter; identifying a mapping structure, from a plurality of mapping structures, relating to said key, wherein each mapping structure corresponds to a respective segment of the columnstore; interrogating the identified mapping structure to determine a value corresponding to the key, wherein the value identifies an entry of an index table of a plurality of index tables that corresponds to the respective segment, wherein the entry of the index table identifies a row of the respective segment of the columnstore relating to the parameter; and retrieving data relating to the parameter from a data source based on data stored in the entry of the index table.

Abstract: A database is partitioned into a plurality of sorted runs, wherein each sorted run includes a set of sorted segments of data records. One of the sorted runs preferably includes more than half of the records of the database, and the other sorted runs are progressively smaller. A query is processed by searching each of the sorted runs. Writes are effected by appending a new sorted run to the database. Sorted merges are used to combine the smaller sorted runs. Deletions are effected by marking the deleted record in the sorted run. Modifications are effected by deleting the original record and writing the modified record to the database. The larger sorted runs are only re-sorted or merged when the sorted run includes a substantial number of deletions. Two merge queues are maintained to enable rapid merges of the smaller sorted runs while a merger of larger sorted runs are occurring.

Abstract: There is provided a computer-implemented method of generating a bitmap filter. A filter parameter is received, and a first data source associated with the filter parameter is queried to identify at least one entry in the first data source with an identifier corresponding to the filter parameter. A first process is performed where zero or one single bit location of a plurality of bit locations in a bitmap filter is identified that corresponds to an identifier of an entry of the first data source corresponding to the filter parameter. Each identifier has a numerical value and the bit location is identified based on the numerical value of the corresponding identifier. The single bit location is assigned to the identifier, such that there is one-to-one mapping between each identifier corresponding to the filter parameter and an assigned bit location in the bitmap filter. A bit is set at the assigned bit location.