Abstract: Methods and systems for segmenting a medical image into classes are described. A system to segment a medical image includes a processor and memory with instructions that upon execution cause the system to perform a method for segmenting the image. The method includes using initial segmentation methods to derive at least one set of probabilities of belonging to the classes for each pixel of the image. The at least one set of probabilities and the image are input into a neural network which segments the image based on both the probabilities and the image provided. This system can also use patches or sub-sections of the original image and the at least one set of probabilities as inputs to the final neural network. The patch based method enables segmentation of larger images, which usually require large amounts of time and memory to segment, and can produce a highly trained neural network.