Abstract: There is disclosed a method and system for clinical effectiveness evaluation of artificial intelligence based medical devices. The clinical effectiveness evaluation system includes a diagnosis result receiving unit receiving diagnosis results for arbitrary clinical data from a plurality of specialists and the artificial intelligence based medical device; a correspondence degree calculation unit calculating a degree of correspondence between the received diagnosis results; a statistical value derivation unit deriving a predetermined statistical value on the basis of the calculated degree of correspondence; and a comparison test unit performing a comparison test on the artificial intelligence based medical device using the derived statistical value.