Abstract: Described is a system for multi-object detection and recognition in cluttered scenes. The system receives an image patch containing multiple objects of interest as input. The system evaluates a likelihood of existence of an object of interest in each sub-window of a set of overlapping sub-windows. A confidence map having confidence values corresponding to the sub-windows is generated. A non-maxima suppression technique is applied to the confidence map to eliminate sub-windows having confidence values below a local maximum confidence value. A global maximum confidence value is determined for a sub-window corresponding to a location of an instance of an object of interest in the image patch. The sub-window corresponding to the location of the instance of the object of interest is removed from the confidence map. The system iterates until a predetermined stopping criteria is met. Finally, detection information related to multiple instances of the object of interest is output.