Abstract: A multi-dimensionally layered encoding and decoding system having reduced the bandwidth requirements for a channel that carries an image stream encoded by the system. A source image stream is partitioned multi-dimensionally into a lower resolution layer and higher resolution layer image stream. The lower resolution image stream is encoded and decoded in accordance with certain accepted standards such as MPEG-2, MPEG-4 and JPEG. In the encoding of the lower layer image stream, a number of indicator signals are derived that indicate where a compression loss and resolution loss occurs. Additionally, the lower layer image stream is non-linearly processed to create an improved image stream and non-linear indicator for use in the upper-layer encoder. The upper-layer encoder uses the non-linear processed image stream and a decoded, reconstructed upper-layer encoded stream to derive the motion vectors and prediction frames in a motion compensation process.