Abstract: A traffic hand signal detection system includes: an imaging unit configured to acquire a photographed image from a camera photographing a predetermined range; an image classifier configured to classify an arm motion from the photographed image provided from the imaging unit by imparting a class; a detection module configured to detect the arm motion from the photographed image classified by the image classifier and generate a traffic hand signal sequence converted into a number; and an analysis module configured to receive the traffic hand signal sequence converted into the number from the detection module and determine a type of a traffic hand signal.