Abstract: A video surveillance system for reducing false alarms. The video surveillance system includes a camera and an electronic processor. The electronic processor is configured to, when a moving object is detected in a video captured by the camera, perform object detection on the video to determine a score associated with a class of objects, using metadata associated with the video, determine a feature associated with the moving object detected in the video, and, using a machine learning algorithm, analyze the score associated with the class of objects and the feature associated with the moving object detected in the video, to determine whether the moving object detected in the video is a false alarm or a true alarm. The electronic processor is also configured to, when the moving object detected in the video is a true alarm, generate an alert.