Abstract: A processing module of a database system is operable to determining a query for execution. A plurality of row sets are processed over a temporal period in conjunction with execution of the query based on automatically selecting a requested number of rows to be included in each row set based on row scheduling parameters, generating a scheduling request for the requested number of rows, and receiving each row set for processing in response to the scheduling request via access to one segment of a set of segments required for execution of the query. A query resultant for the query is generated based on processing of the plurality of row sets, and further based on processing of an additional plurality of rows included in the set of segments via a set of other processing modules implemented in parallel with the processing module.

Abstract: A method for execution by at least one processor of a database system includes determining whether a first operator of a query execution plan is an input to a second operator of the query execution plan such that the second operator is trapped at a non-leaf level of the query execution plan. When the second operator is trapped at the non-leaf level of the query execution plan, the method further includes generating an updated query execution plan, wherein the updated query execution plan includes the first operator being distributed to produce a plurality of distributed first operators such that the second operator executes at a lower level than the non-leaf level of the query execution plan. The method further includes executing the updated query execution plan to produce a query resultant.

Abstract: A database system is operable to determine a query for execution. A disjunction of a plurality of conjunctive normal form (CNF) filtering predicates is determined based on filtering predicates of the query. An IO pipeline is generated based on the disjunction of the plurality of CNF filtering predicates that includes, based on applying at least one type of optimization, at least one shared IO element common to multiple ones of plurality of the CNF filtering predicates. The IO pipeline is executed in accordance with execution of the query to generate a filtered subset of a plurality of rows that each meet at least one of the plurality of CNF filtering predicates.

Abstract: A processing module of a database system is operable to determining a query for execution. A plurality of row sets are processed over a temporal period in conjunction with execution of the query based on automatically selecting a requested number of rows to be included in each row set based on row scheduling parameters, generating a scheduling request for the requested number of rows, and receiving each row set for processing in response to the scheduling request via access to one segment of a set of segments required for execution of the query. A query resultant for the query is generated based on processing of the plurality of row sets, and further based on processing of an additional plurality of rows included in the set of segments via a set of other processing modules implemented in parallel with the processing module.

Abstract: A processing module of a database system is operable to determining a query for execution. A plurality of row sets are processed over a temporal period in conjunction with execution of the query based on automatically selecting a requested number of rows to be included in each row set based on row scheduling parameters, generating a scheduling request for the requested number of rows, and receiving each row set for processing in response to the scheduling request via access to one segment of a set of segments required for execution of the query. A query resultant for the query is generated based on processing of the plurality of row sets, and further based on processing of an additional plurality of rows included in the set of segments via a set of other processing modules implemented in parallel with the processing module.

Abstract: A database system is operable to generate a first segment that includes a first subset of a plurality of rows. A first secondary indexing scheme for indexing the first segment is selected from a plurality of secondary indexing options, and first index data for the first segment is generated in accordance with the first secondary indexing scheme. A second segment is generated that includes a second subset of the plurality of rows. A second secondary indexing scheme for indexing the second segment is selected from the plurality of secondary indexing options, and second index data is generated for the second segment in accordance with the second secondary indexing scheme. Execution of a query is facilitated by utilizing the first index data and the second index data.

Abstract: A method for execution by one or more computing devices of a storage network includes dispersed storage error encoding a data segment of data into a set of encoded data slices in accordance with an information dispersal algorithm width number and a decode threshold number, where the decode threshold number of encoded data slices of the set of encoded data slices are needed to reconstruct the data segment. The method further includes storing the set of encoded data slices in primary storage slots of a set of storage units deployed across multiple sites of the storage network in accordance with a mapping of the primary storage slots to storage units of the set of storage units, where a number of the primary storage slots is equal to or greater than the information dispersal algorithm width number.

Abstract: A database system is operable to maintain lock state data that includes a mapping of a plurality of lock state structures to a plurality of lock scope identifiers. Each of the plurality of lock state structures maintains a set of lock requests based on storing a queue of pending lock requests and a set of granted lock requests. A lock management function is executed to update the lock state data for one of the plurality of lock state structures. In response to executing the lock management function, an evaluation function is executed to further update the one of the plurality of lock state structures based on moving at least one lock request from the front of the queue of pending lock requests to the set of granted lock requests based on determining to move the at least one lock request.

Abstract: A method for execution by one or more computing devices of a massive data ingestion system includes receiving data for storage in the massive data ingestion system. The method further includes storing a data segment of the data in a cache memory of the massive data ingestion system. The method further includes retrieving, based on one or more retrieval parameters, the data segment from the cache memory. The method further includes error encoding, in accordance with error encoding parameters, the data segment to produce a set of encoded data slices, wherein the error encoding parameters include a pillar width number and a decode threshold number, and wherein the decode threshold number is a minimum number of encoded data slices needed to reconstruct the data segment. The method further includes storing the set of encoded data slices in a set of storage units of the massive data ingestion system.

Abstract: A data storage system is operable to generate an operation energy utilization estimation function based on historic energy utilization data and historic operation execution data. An operation for execution is determined and energy utilization estimation input data is determined for the operation. The operation energy utilization estimation function is performed upon the energy utilization estimation input data to generate energy utilization estimate data for the operation. An energy efficiency strategy for the data storage system is applied based on the energy utilization estimate data generated for the operation.

Abstract: A data storage system is operable to generate an energy utilization-based operation scheduling function based on power usage policy data for a data storage system. An operation for execution is determined and energy utilization-based operation scheduling input data is determined for the operation. The operation energy utilization-based operation scheduling function is performed upon the energy utilization-based operation scheduling input data to generate energy utilization-based operation scheduling data for the operation. The operation is executed in accordance with the energy utilization-based operation scheduling data.

Abstract: A data storage system is operable to perform a power supply strategy selection function based on first energy utilization-based operation optimizer input data to generate first energy utilization-based power supply strategy data that includes first power supply resource type proportion data denoting, for each power supply type of a plurality of power supply types, a corresponding first proportion of power supply resources that be implemented as the each power supply type. Based on the first energy utilization-based power supply strategy data, a first set of power supply resources is selected to execute a first operation is selected. The power supply strategy selection function is performed based on second energy utilization-based operation optimizer input data to generate second energy utilization-based power supply strategy data. Based on the second energy utilization-based power supply strategy data, a second set of power supply resources is selected to execute a second operation.

Abstract: A data storage system is operable to generate operation requirement data based on power usage policy data. An energy utilization-based operation limitation enforcement function is performed upon energy utilization-based operation restriction enforcement input data to generate energy utilization-based operation restriction enforcement data for an operation based on applying the operation requirement data. The operation is executed when the energy utilization-based operation restriction enforcement data indicates execution of the operation is allowed, and execution of the operation is foregone when the energy utilization-based operation restriction enforcement data indicates execution of the operation is disallowed.

Abstract: A data processing system is operable to execute a query based on generating a request for rows and sending the request to a data storage system that stores a plurality of records via a plurality of files in memory resources of an object storage system and further stores configuration data that includes table metadata mapping storage of the plurality of records of a plurality of tables via the plurality of files. A response is received from the data storage system indicating a filtered row set generated by the data storage system as the proper subset of the plurality of records. The filtered row set indicated in the response is processed to produce a query resultant.

Abstract: A method for execution by a storage network begins by receiving data for storage by the storage network and continues by determining data preparation tasks for the data. The method continues by indexing the data in accordance with the data preparation tasks to generate a data index and processing the data in accordance with the data index to produce indexed data. The method then continues by determining distribution criteria for the data based on the data index and distributing the data and the data index to a set of distributed storage units in accordance with the distribution criteria, Finally, the method establishes criteria for analyzing found data of the data in the storage network.

Abstract: A data storage system is operable to storing a first plurality of files and a second plurality of files in memory resources of an object storage system of a data storage system. The first plurality of files store a plurality of records of at least one table, and the second plurality of files store a set of index structures indexing the plurality of records. Table metadata is generated for storage, mapping the first plurality of files and the second plurality of files to the at least one table, via a metadata processing system of the data storage system. A filtered row set identifying a proper subset of the plurality of records meeting filtering parameter data is generated based on accessing the table metadata, and based on further accessing at least one file of the second plurality of files in the object storage system storing at least one index structure.

Abstract: A data storage system is operable to store a plurality of records of a plurality of tables via a plurality of files in memory resources of an object storage system of a data storage system and to store configuration data indicating table metadata, mapping storage of the plurality of records of the plurality of tables via the plurality of files, via a metadata processing system of the data storage system. A record identification pipeline is generated for execution based on filtering parameter data and the configuration data. A filtered row set identifying a proper subset of the plurality of records meeting the filtering parameter data is generated based on executing the record identification by accessing at least one file of the plurality of files.

Abstract: A database system is operable to store a plurality of segment row data via a first storage mechanism corresponding to a first durability level and facilitate storage of the plurality of segment row data via at least one file stored in a second storage mechanism corresponding to a second durability level, where table metadata corresponding to the at least one file is further stored via the second storage mechanism. A failure of storage of one of the plurality of segment row data via the first storage mechanism is detected. The one of the plurality of segment row data is recovered for storage via the first storage mechanism based on accessing of the at least one file based on accessing the table metadata corresponding to the at least one file based on communications between at least one storage system interface and a metadata processing system of the second storage mechanism.

Abstract: A method for execution by a storage network begins by receiving a data object for storage, segmenting the data according to a data segmenting protocol to generate a set of data segments and executing a deterministic function on the set of data segments to generate scoring information for respective data segments of the set of data segments. The method continues by comparing the scoring information for a respective data segment to scoring information for previously stored data segments in the storage network and based on the comparison, facilitating storage of a first portion of the set of data segments and not storing a second portion of the set of data segments.

Abstract: A processing system is operable to encode data to produce a first set of data slices based on a value of a width parameter. The data is stored based on maintaining storage of the first set of data slices across a set of storage units of a storage pool. Storage of the first set of data slices is maintained in the set of storage units of the storage pool after addition of an additional set of storage units added to the storage pool. The value of the width parameter is increased to an increased value to produce an updated width parameter. The data is re-encoded in accordance with the updated width parameter to produce a second set of data slices. The data is re-stored based on maintaining storage of the second set of data slices across the expanded set of storage units of the storage pool.