Abstract: A hash based data processing system generates an output dataset based on combining a plurality of input datasets. Input datasets may be divided into a plurality of portions using a hashing algorithm that hashes based on a data attribute of at least one of the input datasets. Pairs of the portions may then be combined through join operations that may be performed at least partly in parallel, based on the data attribute used in the hashing algorithm. The combined dataset portions resulting from the join operations may then be combined to generate an output dataset to be stored in a location that is accessible to user devices. In some cases, one or more step datasets may be generated and stored prior to the output dataset, the step dataset(s) including subset(s) of the data in the output dataset.

Abstract: Hash based data processing generates an output dataset based on combining a plurality of input datasets. Input datasets may be divided into a plurality of portions using a hashing algorithm that hashes based on a data attribute of at least one of the input datasets. Pairs of the portions may then be combined through join operations that may be performed at least partly in parallel, based on the data attribute used in the hashing algorithm. The combined dataset portions resulting from the join operations may then be combined to generate an output dataset to be stored in a location that is accessible to user devices. In some cases, one or more step datasets may be generated and stored prior to the output dataset, the step dataset(s) including subset(s) of the data in the output dataset.