Abstract: This application discloses a system and method for federated collaborative machine learning model development using local training datasets that are not shared. An adaptive and evolutionary approach is used to select local training nodes that are most fit from one training round to the next training round to optimize an overall cost and performance function for the federated learning, to cross-over model architecture between local training nodes, and to perform model architecture mutation within local training nodes. The local training nodes are further clustered to account for the inhomogeneity in the local datasets. Such adaptive, evolutionary, and collaborative federated learning thus provides cost-effective and high-performance model development.