Abstract: A computer-implemented method includes obtaining at least one metric from a framework processing large-scale data, generating at least one prompt from the at least one metric, providing the at least one prompt to a selected large language model (LLM), the LLM being trained with data related to the framework, converting a recommendation from the LLM into a first configuration for the framework, forming a second configuration by applying at least one guiderail to the first configuration, applying the second configuration to the framework for processing a first data processing job, evaluating a cause of failure of the first data processing job, automatically adjusting, based on the cause of failure, the second configuration to form a third configuration, and apply the third configuration to the framework for processing a second data processing job, the second data processing job being a resubmitted first data processing job.