Abstract: A record processing and storage system operates by: generating a set of pages from a plurality of row data via a plurality of processing core resources, wherein each processing core resource in the plurality of processing core resources generate a corresponding subset of the set of pages, independently from and in parallel with processing of other subsets of the set of pages via other ones of the plurality of processing core resources; facilitating performance of a single storage transaction to store the set of pages; identifying a page set interval based on a plurality of row number intervals of the set of pages; generating, based on completing the single storage transaction, row durability data indicating a least favorably ordered row number of a plurality of row numbers corresponding to the plurality of row data; and transmitting the row durability data to a computing device associated with the plurality of row data.

Abstract: A method includes generating a first lock requests, each for transmission to a storage unit, where each first lock request indicates a first encoded data slice. First ballots are received, each of the first ballots corresponds to a storage unit, each of the first ballots indicates a first ranking of first requesting entities that includes the DST processing unit. First election result data is generated based on the first ballots. The first election result data indicates a first winner. When the first winner is the DST processing unit, slice access requests are generated in response to the first election result data indicating that the first winner is the DST processing unit, each for transmission to one of the storage units, where each of the slice access requests indicates the first encoded data slice.

Abstract: A method includes generating a plurality of pages from a plurality of records received from a plurality of data sources. Deduplication of the plurality of pages is facilitated based on a plurality of page metadata of the plurality of pages based on, for the each page of the plurality of pages. A filtered set of potentially-intersecting pages is identified for each given page as a proper subset of the plurality of pages stored in the page storage system based on first comparison parameters, and an intersecting set of pages that include a row number intersection with the given page is identified as a proper subset of the filtered set of potentially-intersecting pages based on second comparison parameters. Records with records with row numbers included in row number intersections with other pages in the intersecting set of pages are removed from the each page.

Abstract: A method for execution on a storage network begins with receipt of a data object for storage by a plurality of users and continues with a storage network processing unit determining whether the data object is associated with a replica storage node of a plurality of replica storage nodes associated with the storage network. When the data object is associated with a replica storage node, the method continues by facilitating storing the data object in each replica storage node of the plurality of replica storage nodes and receiving a plurality of access responses from at least some replica storage nodes of the plurality of replica storage nodes. The method then continues by determining whether an access response threshold number of access responses have been received and finally, in response to a determination that an access response threshold number of access responses have been received, transmitting an acknowledgement to the plurality of users.

Abstract: Methods and apparatus for asynchronous replication of data in a storage network. In an embodiment, a processing module(s) of a computing device identifies at least a first storage set and a second storage set for replicated storage of data. The processing module maintains a synchronization schedule for the first storage set and the second storage set. After initiating storage of a data object in the first storage set (e.g., using first error encoding parameters), the processing module determines, based at least in part on the synchronization schedule, to synchronize the first storage set and the second storage set. In response to determining to synchronize the first and second storage sets, the processing module determines that the second storage set requires the data object to maintain synchronization with the first storage set and facilitates storage of the data object in the second storage set (e.g., using second error encoding parameters).

Abstract: A storage system is operable to transition from operation in accordance with a first storage scheme to operation in accordance with a second storage scheme by, for each storage device of the plurality of storage devices, expanding a storage size of a single storage structure of the corresponding plurality of data storage structures identified to implement the second storage scheme from an initial size to a final size consuming full storage resources of the each storage device based on writing all new data of a plurality of write requests to the single storage structure in accordance with the second storage scheme, performing at least one expansion of the single storage structure within the each storage device in response to an expansion condition being met, and completing transition based on the single storage structure reaching the final size consuming the full storage resources of the each storage device.

Abstract: A method for execution in a storage network begins with a storage network entity receiving a write request for a data object, where the data object is dispersed error encoded in accordance with dispersed error encoding parameters to produce a plurality of sets of encoded data slices. The method continues with the storage network entity determining whether to store a plurality of copies of the data object; and in response to a decision to store a plurality of copies of the data object, determining a plurality of vaults for storage of the data object. The method then continues by identifying a multicast IP address associated with the plurality vaults for storage of the plurality of copies. The method then continues by generating a multicast IP message to a plurality of storage units associated with each vault of the plurality of vaults, where the storage units of the plurality of storage units are configured to determine a multicast IP address from the multicast IP message.

Abstract: A database system operates by: generating a plurality of pages that include a plurality of records in accordance with a row-based format; storing the plurality of pages via a page storage system; generating segment generation determination data based on storage utilization data of the page storage system; generating a plurality of segments from the plurality of pages that include the plurality of records in a column-based format based on the segment generation determination data indicating segments be generated; storing the plurality of segments via a segment storage system; receiving a query; generating an operator execution flow for execution of the query that includes an IO operator, generating a modified operator execution flow for execution of the query by replacing the IO operator of the operator execution flow with a segment IO operator in parallel with a page IO operator, and facilitating execution of the query.

Abstract: A processing system of a storage network operates by: sending, to at least one storage unit of the storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to data, wherein the data is coded in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of encoded data slices that is required to decode the data; receiving, at the at least one processing circuit and from the at least one storage unit, a first subset of the set of encoded data slices, wherein at least one missing encoded data slice was not included in the first subset and wherein a number of encoded data slices in the first subset is less than the read threshold number; generating, via the at

Abstract: A method for execution by one or more computing devices of a storage network includes obtaining performance impact information regarding a data reconstruction operation associated with a computing device of the one or more computing devices, where the data reconstruction operation is regarding reconstructing data at a first reconstruction rate of a plurality of reconstruction rates, and where the performance impact information includes performance metrics of the storage network affected by the data reconstruction operation. The method further includes determining a second reconstruction rate of the plurality of reconstruction rates to utilize for the data reconstruction operation based on the performance impact information to achieve a first performance metric of the performance metrics. The method further includes executing the data reconstruction operation in accordance with the second reconstruction rate.

Abstract: A method for execution by one or more processing modules of a storage network begins by receiving an access request for a set of encoded data slices, where the data object is segmented into a plurality of data segments, a data segment of which is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of encoded data slices. The method continues by determining whether a revision for one or more encoded data slices in a first storage module is a most recent revision level for the one or more encoded data slices and when the one or more encoded data slices is a most recent revision level, determining whether the revision level for the one or more encoded data slices in a second storage module is the most recent revision level.

Abstract: A method for execution by one or more computing devices includes generating current slice integrity check value information for a rebuilt appended encoded data slice of a set of appended encoded data slices stored in a set of storage units of the storage network, where an appended encoded data slice of the set of appended encoded data slices includes an encoded data slice of a set of encoded data slices and slice integrity check value information for the encoded data slice. The method further includes determining the current slice integrity check value information compares favorably to the slice integrity check value information to produce a trusted rebuilt appended encoded data slice. When the current slice integrity check value information compares favorably to the slice integrity check value information, the method further includes storing the rebuilt appended encoded data slice as the trusted rebuilt appended encoded data slice.

Abstract: A database system operates to: receive, via a communication network, a record stream from a data source that includes a plurality of row data, wherein each of the plurality of row data includes a set of records for storage as relational database rows of at least one database table and further indicates a batch number corresponding to the set of records; add each of the plurality of row data to a pending row data pool; and perform parallelized generation of a plurality of pages from the plurality of row data to each include multiple records of the plurality of row data via a plurality of processing core resources based on each processing core resource in the plurality of processing core resources including power, processing a corresponding subset of the plurality of row data, independently from and in parallel with processing of other subsets of the plurality of row data via other ones of the plurality of processing core resources.

Abstract: A method for execution by one or more computing devices of a storage network includes dispersed storage error encoding a data segment of data into a set of encoded data slices in accordance with an information dispersal algorithm width number and a decode threshold number, where the decode threshold number of encoded data slices of the set of encoded data slices are needed to reconstruct the data segment. The method further includes storing the set of encoded data slices in primary storage slots of a set of storage units deployed across multiple sites of the storage network in accordance with a mapping of the primary storage slots to storage units of the set of storage units, where a number of the primary storage slots is equal to or greater than the information dispersal algorithm width number.

Abstract: A method for execution by a distributed storage network begins by receiving a request to transfer a copy of a set of encoded data slices from at least some associated virtual storage vaults to a destination virtual storage vault and continues by determining whether the destination storage unit supports a source virtual storage vault of the at least some source virtual storage vaults. When the destination storage unit supports the source virtual storage vault the method continues by determining a sub-set of encoded data slices of the set of encoded data slices for transfer and finally, by facilitating sending the sub-set of encoded data slices to the destination storage unit.

Abstract: A processing system of a storage network operates by: sending, to at least one storage unit of the storage network, at least one read request corresponding to at least a read threshold number of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to data, wherein the data is coded in accordance with dispersed error coding parameters that include a write threshold number and the read threshold number, wherein the write threshold number is a number of encoded data slices in the set of encoded data slices and wherein the read threshold number is a number of the set of encoded data slices that is required to decode the data; receiving, via the at least one processing circuit and from the at least one storage unit, a first subset of the set of encoded data slices, wherein the first subset is missing at least one missing encoded data slice that was not received from the at least one storage unit in response to the at least one read request and wherein a number of encoded

Abstract: A storage network method begins by determining to initiate periodic integrity checks for a plurality of data objects stored in the storage network, where a data object is dispersed error encoded in accordance with dispersed error encoding parameters to produce a plurality of sets of encoded data slices. The method continues by determining a frequency for performing the periodic integrity checks and determining, based on the frequency for performing the periodic integrity checks, a portion of a data object for a first integrity check, where the portion includes one or more encoded data slices of the plurality of sets of encoded data slices. The method then continues by issuing a list data request for the portion of the data object, receiving a list data response for the portion of the one or more data objects and determining whether the list data response indicates a data error for the portion of the one or more data objects.

Abstract: A computing device includes an interface configured to interface and communicate with a storage network, a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. A computing device receives a data access request for an encoded data slice associated with a data object, determines whether the encoded data slice is stored in the first memory and in response to a determination that the encoded data slice is not stored in the first memory, issues another data access request for the encoded data slice to a second memory, where the first memory includes access characteristics that are faster than the second memory. When a data access response including the encoded data slice is received from the second memory, a response including the encoded data slice is transmitted.

Abstract: A record processing and storage system operates by: automatically determining a buffer interval for a mapping function of batch size to estimated memory utilization; automatically selecting a first batch size for performance of a first data format conversion based a first intersection between an upper bound of the buffer interval and a current memory availability; performing the first data format conversion upon a first batch of data having the first batch size; automatically updating the buffer interval based on a difference between an actual amount of memory utilized in performing the first data format conversion and an estimated memory utilization mapped to the first batch size via the mapping function; automatically selecting a second batch size for performance of a second data format conversion based a second intersection between an upper bound of the updated buffer interval and the current memory availability; and performing the second data format conversion upon a second batch of data having the second

Abstract: A method includes rebuilding an appended encoded data slice of a set of appended encoded data slices stored in a set of storage units of a storage network to produce a rebuilt appended encoded data slice, where the appended encoded data slice includes an encoded data slice of a set of encoded data slices and slice integrity check value information, where the slice integrity check value information includes a slice integrity check value for the encoded data slice, and where the encoded data slice is hashed to produce the slice integrity check value. The method further includes generating current slice integrity check value information for the rebuilt appended encoded data slice. The method further includes verifying the current slice integrity check value information corresponds to the slice integrity check value information. When verified, the method includes storing the rebuilt appended encoded data slice.