Abstract: Blocks in frames of a video are matched to similar blocks across frames using qualitative characteristics of the pixels in the block. Pixels in a frame are categorized into two or more categories. In one embodiment, three pixel categories are: 1) quasi-constant (QC) pixels having intensity values that vary slowly (e.g., pictorial portions), 2) textured pixels characterized by many small edges and thin-line signals (e.g., texture portions), and 3) dominant-edge (DE) pixels characterized by few sharp and dominant edges (e.g., edge portions). Blocks in different frames are matched by minimizing a cost function that measures the mismatch between a block in a first frame and a plurality of candidate blocks in a second frame. In one embodiment, a cost function comprises a contrast component and a brightness component. The brightness component is determined as a function of the pixel categories. Block matching in sequential frames of a video enables a calculation of displacement or motion estimation in the video.