Abstract: A database system includes a plurality of computing devices. Each computing device includes a plurality of processing modules, a computing device operating system, and an application specific operating system. The computing device operating system includes a computing device operating system file system management instruction set. The application specific operating system includes at least one custom file system management instruction set operable to configure operation of a configurable set of processing modules of the plurality of processing modules based on generating a corresponding file system management configuration signal for each processing module of the configurable set of processing modules indicating a selected file system management instruction set of the computing device operating system or the application specific operating system.

Abstract: A database system is operable to determine a set of segments that include a set of rows required to execute at least one query. At least one virtual segment in the set of segments is determined. At least two data portions in a first segment of the at least one virtual segment are identified. A locally rebuilt set of rows for the at least two data portions is generated based on retrieving at least two corresponding data portions of at least one other segment from storage and rebuilding at least two data portions utilizing the retrieved at least two corresponding data portions. Query resultant data is generated based on performing execution of the at least one query by utilizing the locally rebuilt set of rows.

Abstract: A database system operates by: generating an initial query operator execution flow for a corresponding query for execution that includes an extend operation for execution serially before an aggregation operation indicating grouping performance of a corresponding aggregation by new column values of a new column generated via the extend operation via performance of a corresponding extend function of at least one input column; converting the initial query operator execution flow into an updated query operator execution flow semantically equivalent to the initial query operator execution flow based on: generating an updated aggregation operation to group by input column values of the at least one input column, and pushing the updated aggregation operation for execution serially before the extend operation; and executing the corresponding query by applying the updated query operator execution flow based on: executing the updated aggregation operation upon a plurality of rows to generate a plurality of aggregation

Abstract: A database system is operable to generate an initial query operator execution that includes an IO operator for execution serially before an aggregation operator indicating performance of an aggregation. The initial query operator execution flow is converted into an updated query operator execution flow semantically equivalent to the initial query operator execution flow based on updating the IO operator to include the performance of the aggregation and adding a re-aggregation operator serially after the IO operator. A corresponding query is executed by applying the updated query operator execution flow based on executing, via each of a plurality of parallelized resources, the IO operator generate corresponding sub-aggregation output via performance of the aggregation, and executing the re-aggregation operator upon data blocks generated based on execution of the IO operator across the plurality of parallelized resources.

Abstract: A database system is operable to determine to generate a segment rebuild plan for rebuilding a first segment based on accessing first segment metadata for the first segment. A plurality of replicated part location data is determined for a plurality of parts of the first segment based on processing the first segment metadata and based on applying a deterministic segment part replica dispersal algorithm previously utilized to store the segment group. Nodes of the database system storing a corresponding plurality of sibling parts required to rebuild the plurality of parts of the first segment are determined based on the plurality of replicated part location data and based on storage location-to-node mapping data. The first segment is rebuilt based on applying the segment rebuild plan.

Abstract: A database system is operable to generate an initial query operator execution that includes an IO operator for execution serially before an aggregation operator indicating performance of an aggregation. The initial query operator execution flow is converted into an updated query operator execution flow semantically equivalent to the initial query operator execution flow based on updating the IO operator to include the performance of the aggregation and adding a re-aggregation operator serially after the IO operator. A corresponding query is executed by applying the updated query operator execution flow based on executing, via each of a plurality of parallelized resources, the IO operator generate corresponding sub-aggregation output via performance of the aggregation, and executing the re-aggregation operator upon data blocks generated based on execution of the IO operator across the plurality of parallelized resources.

Abstract: A database system is operable to generate an initial query operator execution flow for a corresponding query for execution that includes an extend operation serially before a column-based filtering operation indicating filtering by new column values of a new column generated via the extend operation as a function of an existing column. The initial query operator execution flow is converted into an updated query operator execution based on pushing the column-based filtering operation serially before the extend operation, and updating the column-based filtering operation to filter by existing column values of the existing column via applying an inverse function of the function. The corresponding query is executed by applying the updated query operator execution flow based on executing the column-based filtering operation upon a plurality of rows to generate a filtered subset of the plurality of rows, and executing the extend operation upon only rows in the filtered subset.

Abstract: A node of a database system is operable to determine, at a first time, to prepare for a shutdown. The shutdown of the node is performed at a second time that is a period of time after the first time based on the node delaying the shutdown until a plurality of currently running processes being run by the node that initiated prior to the first time are determined to be complete. During the period of time after the first time and prior to performing the shutdown, a set of new processing requests are rejected by the node.

Abstract: A database system is operable to generate a query operator execution flow for a query that includes a plurality of operators implementing a plurality of parallelized optimization processes configured to facilitate generating of a machine learning model. The query operator execution flow is executed in conjunction with executing the query based on executing the plurality of operators. Executing each of the plurality of parallelized optimization processes includes generating a corresponding set of candidate model coefficients of a plurality of sets of candidate model coefficients. A corresponding set of candidate model coefficients is based on processing the set of best positions generated via the second type of optimization algorithm. The machine learning model is generated in executing the query based on selection of a most favorable set of candidate model coefficients from a plurality of sets of candidate model coefficients outputted via the plurality of parallelized optimization processes.

Abstract: A database system operates by: determining a set of assigned segments based on data ownership information; generating a plurality of segment handles and a corresponding plurality of segment metadata for the set of assigned segments based on performing a segment activation step for the set of assigned segments; determining a first query for execution requiring access to a first segment of the set of assigned segments; and executing an IO operator of the first query based on loading the first segment by utilizing a corresponding one of the plurality of segment handles.

Abstract: A database system is operable to receive a plurality of records of a dataset for storage. A plurality of segment row data is generated from the plurality of records and a plurality of parity data corresponding to the plurality of segment row data is generated. The plurality of segment row data is stored via a first storage mechanism. The plurality of parity data is stored via a second storage mechanism. Execution of a plurality of queries against the dataset is facilitated by accessing the plurality of segment row data via the first storage mechanism. A storage failure of one of the plurality of segment row data via the first storage mechanism is detected. The one of the plurality of segment row data is recovered for storage via the first storage mechanism based on accessing at least one parity data via the second storage mechanism.

Abstract: A database system is operable to cache first resultant data generated via executing a first query in cache memory resources during a first temporal period. An instruction to re-execute the first query is processed during the first temporal period by accessing the first resultant data in the cache memory resources. A cached resultant invalidation instruction indicating the first relational database table is received. The first resultant data from the cache memory resources is removed during a second temporal period based on processing the cached resultant invalidation instruction. A third instruction to re-execute the first query is processed after the second temporal period by re-executing the first query via access to the first relational database table in the first storage resources to re-generate corresponding first resultant data for the first query.

Abstract: A database system is operable to read a plurality of rows from memory of a relational database stored in memory resources. Each of the plurality of rows are assigned to a corresponding one of a plurality of training data subsets of the plurality of rows based on performing a row dispersal process. A plurality of sets of candidate model coefficients are generated based on executing a plurality of parallelized optimization processes. Each set of candidate model coefficients is generated based on executing a corresponding parallelized optimization process upon a corresponding training data subset independently from executing other ones of the plurality of parallelized optimization processes upon other ones of the plurality of training data subsets. A most favorable set of candidate model coefficients is selected from the plurality of sets of candidate model coefficients generated via the plurality of parallelized optimization processes.

Abstract: A database system is operable to execute a first to generate the machine learning model from a training set of rows based on accessing and processing the training set of rows via a plurality of operators. system configuration data tracking a plurality of objects of a database system is updated to further track the machine learning model as a corresponding first object tracked via the system configuration data. A query output for a second query indicating applying of the machine learning model via execution of the second query based on applying the machine learning model to a set of rows in accordance with at least one property of the corresponding first object based on accessing the system configuration data.

Abstract: A node of a computing system includes a main memory and a plurality of processing core resources. The main memory includes a computing device section and a database section. The computing device section includes a computing device operating system area and a computing device general area. The database section includes a database section that includes a database operating system area, a disk area, a network area, and a database general area. The database operating system area allocates at least one portion of the main memory for database operations that is locked from access by the computing device operating system area.

Abstract: A database system is operable to store a plurality of column values for a first column of a plurality of rows. A query having a query predicate indicating a negation of a condition for the first column based on a first value is executed. A first index element is applied to identify a first proper subset of the plurality of rows having values for the first column meeting the condition based on accessing index data. At least one second index element is applied to identify a second proper subset of the plurality of rows having values for the first column meeting at least one missing data-based condition based on accessing the index data. A set difference between the plurality of rows and a union of the first proper subset and the second proper subset is identified. A query resultant for the query is generated based on the set difference.

Abstract: A database system is operable to generate query output based on executing a plurality of operators of a query operator execution flow for a query. Executing each operator can include generating a set of output query rows based on processing a set of input query rows, maintaining execution tracking data for the each operator that includes at least one counter tracking at least one event type, and generating a set of execution tracking rows based on sampling the execution tracking data. The query output includes a final set of rows corresponding to a query resultant and a final set of execution tracking rows generated by at least one final operator based on an aggregation of a plurality of sets of execution tracking rows generated by the plurality of operators.

Abstract: A database system is operable to determine a query for execution. The database system can execute a left join operation for the query based on receiving a plurality of left input rows, and forwarding all of the plurality of left input rows are forwarded. It is determined whether any right input rows match with each given left input row based on a hash map. When one right input row is identified as matching with the given left input row, a value of right input row is copied and emitted in conjunction with the given left input row. When no right input rows are identified as matching with the given left input row, a null value is emitted in conjunction with the given left input row.

Abstract: A record processing and storage system is operable to receive a set of records for storage. The set of records are included in a plurality of pages stored by a page storage system, and each page of the plurality of pages includes a plurality of records in the set of records. Key value-based record distribution data is generated for the set of records based on a plurality of cluster key values of the set of records. A cluster key domain spanned by the plurality of cluster key values is divided into a plurality of key space sub-intervals based on the key value-based record distribution data. The set of records are segregated into a plurality of row subsets corresponding to the plurality of key space sub-intervals. A plurality sets of segments are generated by processing the plurality of row subsets in parallel.

Abstract: A database system is operable to write to a file buffer corresponding to geospatial index data for a plurality of rows based on processing each given row of the plurality of rows. A new leaf node of a set of leaf nodes in a temporary leaf node buffer when the given row includes a geospatial object. When the temporary leaf node buffer is determined to have a number of leaf nodes meeting a predetermined threshold number of leaf nodes, a new tree-based index structure of a set of tree-based index structures of the geospatial index data is built via processing the temporary leaf node buffer. The geospatial index data is stored based on writing the file buffer to disk memory resources. A query is executed against a relational database table based on accessing the geospatial index data in the disk memory resources.