Abstract: Technology disclosed herein includes systems and methods for transforming large amounts of transactional or system data with no downtime and little to no impact on the end user experience. More specifically, systems and methods for performing crawling data transforms are disclosed in which data transforms run as timed and throttled background processes to reduce the required processing resources and eliminate the need for downtime. A method for performing crawling data transforms includes generating transformation plans having multiple steps based on a transform definition, determining that a number of running plans is below a threshold, determining that a number of running steps is below a threshold, identifying an uncompleted table within a step of a transformation plan, and transforming records in the uncompleted table until all records in the table are transformed or the step is terminated.

Abstract: Technology disclosed herein includes systems and methods for transforming large amounts of transactional or system data with no downtime and little to no impact on the end user experience. More specifically, systems and methods for performing crawling data transforms are disclosed in which data transforms run as timed and throttled background processes to reduce the required processing resources and eliminate the need for downtime. A method for performing crawling data transforms includes generating transformation plans having multiple steps based on a transform definition, determining that a number of running plans is below a threshold, determining that a number of running steps is below a threshold, identifying an uncompleted table within a step of a transformation plan, and transforming records in the uncompleted table until all records in the table are transformed or the step is terminated.