Abstract: Methods for improving neural networks by addressing the vanishing gradient include obtaining seed topologies in a deep neural network and iterating over the seed topologies using neuroevolution, with mutations to adjust the topologies or weights of the neural network. The performance of the various mutated models of the neural network is identified or modeled. An ideal, or champion, topology or model is thereby generated based on the neuroevolution. The path taken to arrive at the champion is monitored and stored, such that the series of evolutions along the evolutionary path from the seed model to the champion model is identified. After identifying the champion model and the associated mutation steps, the model may be further iterated by re-traversing the series of topological steps that led the champion model, while providing mutations or randomized weights for the various steps, which can identify further advancements or improvements to the neural network.

Abstract: A method may include receiving a graph-based model in a first format, including a static topology of the graph-based model. The method may also include encoding the graph-based model from the first format into a neural network topology optimizer (NNTO) readable format such that the topology of the encoded graph-based model is configured to be altered; creating a first group of entities based on at least a same portion of the encoded graph-based model; and performing a learning operation by tuning parameters of the first group of entities to produce an optimization score for each entity. Additionally, the method may include performing a validation operation; determining that an improvement in validation performance for at least one entity is within a threshold amount of improvement; selecting a solution entity; and adding the selected solution entity into the graph-based model in place of the same portion.