Abstract: A Federated learning server and a method are provided. The Federated learning server is configured to aggregate a plurality of received model updates to update a master machine learning model. Once a pre-defined threshold or interval for received model updates is reached, a set of current hyper-parameter values and corresponding validation set performance metrics obtained from the updated master machine learning model are sent to a hyper-parameter optimization model. The optimization model infers the next set of optimal hyper-parameters using pairwise history of hyper-parameter values and the corresponding performance metrics. The inferred hyper-parameter values are sent to the Federated Learning server which updates the master machine learning model with the updated set of hyper-parameter values and redistributes the updated master machine learning model with the updated set of hyper-parameter values.