Abstract: The invention is a method and apparatus for quickly marking object trajectories in an image sequence. The traditional method of manually tracking an object is by marking the location of the object in each image in the image sequence. In the present invention, the object is first tracked in one direction by one quick stroke of a spatial input device across the subsampled image sequence that is laid out in that direction. The object is then tracked in an orthogonal direction by another quick stroke of the spatial input device across the same subsampled image sequence that is laid out in the orthogonal direction. The intersections of the two tracks will then form the points of the 2-D neo-trajectory, which is then interpolated onto the image sequence. The invention is useful not only for quick manual annotation of tracks, but also for performance evaluation and optimization of automatic tracking algorithms.