Abstract: The invention relates to a method and system for predicting optimal configuration parameters for a data analytics engine. The method and system utilizes Supervised Machine Learning (ML) techniques for predicting optimal configuration parameters for the data analytics engine to run a specific application in a given time frame, wherein the optimal configuration includes estimating an optimal number of executor instances in real-time. The invention analyzes input parameters such as data metrics, software metrics, and hardware metrics to determine if the input parameters comprise a new dataset or a continuous dataset and deploy one or more models such as a Forecasting Model and a Regression Model. The one or more models then derive and allocate the possible number of executor instances. The executor instances allocated can be a number of executors, executor cores, the memory of executors, driver memory, and parallelism.