Abstract: A pattern outline tracking system provides the functions that include scanning an object to be recognized and providing a series of analog video signals, comparing a video signal level with the specific, variable threshold level and providing a binary signal consisting of a sequence of serial bits, holding the binary signal bits as the input image pattern in an image memory, deriving partial patterns sequentially from the serial binary signal bits, each of the partial patterns consisting of a matrix of vertical and horizontal lines each containing specific numbers of pixels, matching the partial patterns against the corresponding reference patterns containing the previously defined starting points to be tracked, holding the address in the image memory for the certain pixel within the partial pattern identified in the matching step as the possible point address for the tracking operation, and tracking the outline of the input image pattern on the image memory in accordance with the possible point address.