Abstract: A decoder for a video signal encoded according to the MPEG-2 standard processes either interlace scan signals or progressive scan signals by dynamically reconfiguring a single high-bandwidth memory. The memory is used to hold 1) the input bit-stream, 2) first and second reference frames used for motion compensated processing, and 3) image data representing a field that is currently being decoded. The decoder includes circuitry which stores and fetches the bit-stream data, fetches the reference frame data, stores the image data for the field that is currently being decoded in block format and fetches this image data for conversion to raster-scan format. This circuitry also detects whether the signal is in interlace or progressive format from the input data stream. When an interlace format signal is being decoded, the memory is partitioned in one configuration and when a progressive format signal is being decoded, the memory is partitioned in another configuration.