Abstract: A system and method for scaling image streams that use motion vectors is disclosed. The system combines an error term with a predicted term in order to produce a display value. The system operates on image components represented in the spatial and frequency domains. The system processes motion vectors in the spatial domain. The motion vectors are scaled. The integral part of the scaled motion vector addresses a framestore. The fractional portion of the scaled motion vector is input to a nonlinear filter which determines the value of image components for a location that does not correspond with an image location in the framestore. The output of the nonlinear filter comprises the predicted terms. Data in the frequency domain is processed more efficiently by reducing the size of a block of data by appropriate filtering. The resulting data is transformed to the spatial domain.