Abstract: The present disclosure provides a method and a system for recognizing medical image, the present disclosure utilizes the image data with markers of different diseases for calculating and analyzing to build a pre-trained model, the present disclosure has significant improvements to improve the accuracy of image recognition under the general situation of insufficient effective data in the field of medical image recognition technology, the present disclosure can be applied to the field of medical image recognition technology, including X-ray, CT, MRI, ultrasonic, pathological slice photography or fundus photography.