Abstract: Presented herein are systems and methods using reinforcement learning to perform localization, segmentation, and classification on biomedical images. A computing system may identify a biomedical image. The biomedical image may have a structure of interest (SOI) corresponding to a condition. The computing system may apply one or more models to perform (i) localization to determine a location of the SOI in the biomedical image; (ii) segmentation to identify a segment corresponding to the SOI on the biomedical image; and (iii) classification to determine a class identifying the condition of the biomedical image. The models may be established using reinforcement learning, such as deep Q-learning.