Method and an apparatus for reordering a decoded picture sequence using virtual picture

Abstract

Under some circumstances, it is necessary to contiguously displaying pictures stored in one display buffer. In order to solve the problem mentioned above, the present invention discloses a method for reordering a decode order into a display order of images by inserting virtual pictures to minimize hardware and software cost.

Description

FIELD OF INVENTION

[0001] The present invention is related to video display systems, and more particularly, to the reordering of decode order into display order in video display systems.

BACKGROUND OF INVENTION

[0002] The CCITT/ISO committee has standardized a set of compression and decompression algorithms for still and motion digital video. These standards, including the JPEG, MPEG and H.261 compression schemes, are commonly applied in video conferencing, CD-ROM or DVD-ROM based interactive videos for education and entertainment, video, video on demand (VOD) applications, satellite video transmission applications and many other applications which require communication of motion digital video. The MPEG standard was drafted by the Moving Picture Coding Experts Group (MPEG) which operates within the framework of the Joint ISO/IEC Technical Committee (JCCI) on Information Technology. The draft provided a standard for coded representation of moving pictures, audio and their combination.

[0003] In General, motion estimation is a technique for compressing data in image processing method by estimating a motion vector. The motions within the pixels of current frame (or fields) and pixels of previous frame, in successive image signals, are transmitted as a motion vector and a difference of data between macro-blocks of the previous frame and the current frame.

[0004] In motion estimation, frames of input image signals are divided into I-pictures (Intra-coded picture), P-pictures (Predictive-coded picture), and B-pictures (Bi-directionally predicted-coded picture). The I-pictures are frames (or fields) which are coded without using motion estimation. The P-pictures are frames that perform only the forward motion estimation by using I-pictures or other P-pictures as references. The B-pictures are frames that perform the forward and backward estimation by using I-pictures or P-pictures as references. In standard MPEG 2, the so-called Group of Picture is formed from a series of image frames comprising of I-pictures, P-pictures and B pictures, such as IBBPBBPBBPBBI . . . , the pictures are coded and transmitted to the display system for display.

[0005] The input coded (or compressed) sequence has to be decoded into decoding sequence (i.e. decode order), the order of the decode sequence is not the same as the order of display sequence (i.e. display order), hence the decode order has to be re-ordered into display order for correct display. In a common display and decode control flow, reordering is done by storing pictures of decode order into different decoding buffer, then the display system retrieves the pictures stored in different buffers according to the display order.

[0006] But under some circumstances, it is necessary to display pictures stored in one of the decoding buffer continuously, therefore the above-mentioned reordering method is not applicable. For example, the digest view function provides users with the ability to quickly preview the content of the disc(s), it is implemented by displaying simultaneously the scaled down version of previously-decoded picture with the current decoding picture in the same sequence (/or different sequences). In order to support the preview function, current decoding picture and previous-decoded picture must be stored in the one single buffer for the video output module to display the content in the display buffer continuously. Furthermore, current decoding pictures must be stored into the display buffer in accordance with the display order, which is illustrated in Table. 1. 1 TABLE 1 decode order and display order time 1 2 3 4 5 6 7 8 9 10 11 12 13 14 decode order IO P3 B1 B2 P6 B4 B5 I9 B7 B8 P12 B10 B11 display order I0 B1 B2 P3 B4 B5 P6 B7 B8  I9 B10 B11 P12

[0007] Assume the size of all buffers are the same, and the display system can scale down the decoded picture by a given scale factor such as one third or one fourth. In Table.1, after I0 is decoded and stored in one decoded buffer at time T1, P3 should be decoded and the scaled down I0 should be stored into the display buffer at time T2. When the display system finishes decoding B1 and B2, the scaled down version of P3 should be stored into display buffer thereafter. During the decoding and displaying process, the I-pictures and the P-pictures, which are used as reference pictures, will be stored twice, that is one normal size into decode buffer and the other scaled down size into display buffer. In the meantime, B-pictures will be decoded and sent to display buffer directly.

[0008] Conventional technique tackles the problem by decoding the reference pictures twice, one stored in the decoded buffer in normal size and the other stored in display buffer in scaled down size. However, it will require the system to retrieve the picture again for second decoding, which makes the decoding control scheme complicated. Another conventional technique provides additional DMA hardware with scaling function to move the pictures stored in decoded buffer to display buffer, but it requires additional hardware cost. Still another conventional technique uses system RISC to read pictures stored in the decoded buffer and write them into display buffer, the price is to cost excessive RISC I/O cycles and bring down RISC performance.

[0009] To reorder the decode order into display order, the system has to either decode the reference pictures twice, or add additional DMA hardware, or increase I/O cycles of system RISC, and sometimes the system has to contiguously display the pictures stored in one buffer. The present invention discloses a method for reordering a decode order into a display order by inserting a virtual picture, therefore minimum hardware and software cost can be achieved.

SUMMARY OF INVENTION

[0010] The present invention discloses a method and a circuit for reordering a decode order into a display order of images by inserting a virtual picture in the decode order, the decode order comprises an I-picture, a P-picture, and a B-picture. Upon receiving a compressed picture sequence, the present invention performs the following steps: (1) determining a first picture of the compressed picture sequence is an I-picture, a P-picture, or a B-picture; (2) if the first picture is I-picture, decoding the first picture and storing the decoded first picture into a first buffer; and (3) obtaining a first virtual picture according to a predetermined manner, sending the first virtual picture to a second buffer for display.

DESCRIPTION OF DRAWINGS

[0011] FIGS. 1(a) and 1(b) schematically illustrate the circuit of present invention.

[0012] FIGS. 2(a), 2(b), 3(a), and 3(b) are the flow diagrams of present invention.

[0013] FIG. 4 schematically illustrates one preferred embodiment of present invention.

DETAILED DESCRIPTION OF PRESENT INVENTION

[0014] Please refer to FIG. 1(a), the present invention discloses an apparatus for reordering a decode order into a display order, comprises a parameter generator 100 for generating a parameter 105; a motion compensator 110, in response to the parameter 105, for outputting a virtual picture 115 using the first decode picture stored in the first buffer 131; a bus 120 for connecting every component in the present invention.

[0015] Please refer to FIG. 1(b) for detailed drawing of the memory 130, the memory 130 comprises a first buffer 131 for storing a first decoded picture of the decode order, a second buffer 132 for storing the virtual picture, and a third buffer 133 for storing a second decoded picture of the decode order.

[0016] Continuing with FIG. 1(a), the present invention comprises a display controller 140 connecting to the second buffer 132 for displaying the virtual picture 115. All the pictures are transmitted via bus 120. The display controller 140 obtains the virtual picture 115 and the original decoded pictures, performs OSD, subpicture, highlight mixing function and outputs pictures to video display machine such as TV.

[0017] It is noted that pictures in the compressed picture sequence are either frame or field type, virtual pictures are also frame or field type, the corresponding parameter 105 comprising (in MPEG2 format):

[0018] a. the picture coding type (picture_coding_type) of virtual picture is P-picture, every macro-block in a virtual picture is a skipped macro-block;

[0019] b. the prediction of the macro-blocks in a virtual frame picture should be made as if frame_motion_type is “Frame-based”;

[0020] c. the prediction of the macro-blocks in a virtual field picture should be made as if field_motiontype is “Field-based”;

[0021] d. motion vector is user-defined, preferably 0;

[0022] e. coded-block pattern is 0;

[0023] f. as to virtual field picture, the prediction should be made from the field of the same parity.

[0024] Preferably, the parameter 105 also comprises a scale factor, so the width and height of a virtual picture can be different from the original picture.

[0025] In the following descriptions and illustrations, blocks having the same number mean the same operation.

[0026] In one aspect of the present invention, please refer to FIG. 2(a), step 200 obtains a compressed picture sequence, step 205 determines a first picture in the compressed picture sequence is an I-picture, a B-picture or a P-picture, if the first picture is I-picture, step 210 decodes the first picture and stores the decoded first picture into a first buffer 131. Step 215 uses a decoded picture pre-stored in a third buffer 133, responsive to a parameter 105, generates the first virtual picture 115 and send it to a second buffer 132 for display.

[0027] The present invention further performs the following steps: step 220 determines a second picture, if the second picture is P-picture, step 222 decodes the second picture and stores the decoded second picture into a third buffer 133. Step 223 uses a decoded picture pre-stored in the first buffer 131, responsive to a parameter 105, generates the second virtual picture and send it to the second buffer 132 for display.

[0028] In step 220, if the second picture is I-picture, step 225 decodes the second picture and stores the decoded second picture into the third buffer 133. Step 226 uses a decoded picture pre-stored in the first buffer 131, responsive to the parameter 105, generates the second virtual picture and send it to the second buffer 132 for display.

[0029] In step 205, if the first picture is P-picture as shown in FIG. 2(b), the rest follows the case in which the first picture is I-picture.

[0030] Besides, it should be noted that in FIG. 2(a), the step 210 and 215 are interchangeable, which means the present invention can generate a virtual picture 115 first then decodes the first picture, shown as step 305 and 310 of FIG. 3(a). The rest is the same as FIG. 2(a). Also in FIG. 2(a), after step 220 is done, the following step 222 and step 223 are interchangeable, step 225 and step 226 are interchangeable. The same can be applied in FIG. 3a, wherein step 222 and 223 are interchangeable, step 225 and step 226 are interchangeable.

[0031] While in FIG. 2(b), step 210 and step 215 are interchangeable, shown as step 305 and 310 of FIG. 3(b). And the step 222 and 223 in FIG. 2(b) are interchangeable, step 225 and step 226 are interchangeable. In the same way, step 222 and step 223 of FIG. 3(b) are interchangeable, whereas step 225 and step 226 in FIG. 3(b) are interchangeable.

[0032] Please refer to FIG. 4, in one preferred embodiment of the present invention, upon receiving picture P6, decode picture P6 and store the decoded picture P6 into first buffer 131. In response to the parameter 105, generate a virtual picture using P3 stored in the third buffer 133, send the virtual picture to the second buffer 132. Subsequently, decode the received B4 and B5, send the decoded B4 and B5 directly to the second buffer 132, the display controller 140 will use the pictures stored in the second buffer 132 for display. The next picture received is I9, decode I9 and store the decoded I9 in the third buffer 133, also in response to the parameter 105, generate a virtual picture using P6 stored in the third buffer 131, send the virtual picture to the second buffer 132. The following B7 and B8 will be decoded, and the decoded B7 and B8 will be sent to the second buffer 132 directly, the display controller 140 uses the pictures stored in the second buffer 132 for display thereafter.

[0033] Continuing with FIG. 4, when P12 is received, decode picture P12 and store the decoded picture P12 into first buffer 131. In response to the parameter 105, generate a virtual picture using I9 stored in the third buffer 133, send the virtual picture to the second buffer 132. Subsequently, decode the received B10 and B11, send the decoded B10 and B11 directly to the second buffer 132, the display controller 140 will use the pictures stored in the second buffer 132 for display. The next picture received is P15, decode P15 and store the decoded P15 in the third buffer 133, then in response to the parameter 105, generate a virtual picture using P12 stored in the third buffer 131, send the virtual picture to the second buffer 132.

[0034] Please refer to Table.2, which illustrates how to use virtual pictures to reorder the decode order and the display order. 2 TABLE 2 decode order and display order (with virtual pictures inserted) time 1 2 3 4 5 6 7 Decode order IO (IO_vp) B1 B2 (P3_vp) B4 B5 P3 P6 Display order IO B1 B2 P3 B4 B5 time 8 9 10 11 12 13 14 Decode order (P6_vp) B7 B8 (I9_vp) B10 B11 (P12_vp) I9 P12 P15 Display order P6 B7 B8 I9 B10 B11 P12

[0035] While in Table.2, the motion compensator 110 is treated as a special DMA channel by moving data directly between different buffers to create virtual picture. Besides, it should be noted that whenever the present invention starts to decode a reference picture (I-picture or P-picture), the present invention will stored the received picture in the first or the third buffer and override the older data stored in the first or the third buffer. Please refer to FIG. 4, when P6 is received, it will be stored in the buffer where originally I0 is stored, not the buffer where P3 is stored.

[0036] The present invention uses the motion compensator 110 to generate the virtual picture, all operations can be completed by motion compensator 110 itself. Thus avoiding the need for additional DMA hardware, or increasing I/O cycles of the RISC, and the complexity and the cost of hardware and software design can be reduced.

[0037] It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention covers the modification and variations of this invention provided they come within the scope of the appended claims and their equivalents.

Claims

1. A method for reordering a decode order into a display order of an image, the decode order comprising an I-picture (intra-coded picture), a P-picture (predictive-coded picture), and a B-picture (bi-directionally-predictive coded picture), the method obtaining a compressed picture sequence and comprising the following steps:

(1.1) determining a first picture of the compressed picture sequence;

(1.2) if the first picture is I-picture, decoding the first picture and storing a decoded first picture into a first buffer; and

(1.3) obtaining a first virtual picture according to a predetermined manner, sending the first virtual picture to a second buffer for display.

2. The method of claim 1, wherein the predetermined manner, responsive to a parameter, generates the first virtual picture using a decoded picture pre-stored in the third buffer.

3. The method of claim 1,wherein the step (1.3) further comprises:

(3.1) determining a second picture;

(3.2) if the second picture is P-picture, decoding the second picture and storing a decoded second picture into a third buffer; and

(3.3) obtaining a second virtual picture according to the predetermined manner, sending the second virtual picture to the second buffer for display.

4. The method of claim 3, wherein the predetermined manner, responsive to a parameter, generates the second virtual picture using a decoded picture pre-stored in the first buffer.

5. The method of claim 1,wherein the step (1.3) further comprises:

(5.1) determining a second picture;

(5.2) if the second picture is I-picture, decoding the second picture and storing a decoded second picture into a third buffer; and

(5.3) obtaining a second virtual picture according to the predetermined manner, sending the second virtual picture to the second buffer for display.

6. The method of claim 5, wherein the predetermined manner, responsive to a parameter, generates the second virtual picture using a decoded picture pre-stored in the first buffer.

7. A method for reordering a decode order into a display order of an image, the decode order comprising an I-picture (intra-coded picture), a P-picture (predictive-coded picture), and a B-picture (bi-directionally-predictive coded picture), the method obtaining a compressed picture sequence and comprising the following steps:

(7.1) determining a first picture of the compressed picture sequence;

(7.2) if the first picture is P-picture, decoding the first picture and storing a decoded first picture into a first buffer; and

(7.3) obtaining a first virtual picture according to a predetermined manner, sending the first virtual picture to a second buffer for display.

8. The method of claim 7, wherein the predetermined manner, responsive to a parameter, generates the first virtual picture using a decoded picture pre-stored in the third buffer.

9. The method of claim 7, wherein the step (7.3) further comprises:

(9.1) determining a second picture;

(9.2) if the second picture is P-picture, decoding the second picture and storing a decoded second picture into a third buffer; and

(9.3) obtaining a second virtual picture according to the predetermined manner, sending the second virtual picture to the second buffer for display.

10. The method of claim 9, wherein the predetermined manner, responsive to a parameter, generates the second virtual picture using a decoded picture pre-stored in the first buffer.

11. The method of claim 7, wherein the step (7.3) further comprises:

(11.1) determining a second picture;

(11.2) if the second picture is I-picture, decoding the second picture and storing a decoded second picture into a third buffer; and

(11.3) Obtaining a second virtual picture according to the predetermined manner, sending the second virtual picture to the second buffer for display.

12. The method of claim 11, wherein the predetermined manner, responsive to a parameter, generates the second virtual picture using a decoded picture pre-stored in the first buffer.

13. A method for reordering a decode order into a display order of an image, the decode order comprising an I-picture (intra-coded picture), a P-picture (predictive-coded picture), and a B-picture (bi-directionally-predictive coded picture), the method obtaining a compressed picture sequence and comprising the following steps:

(13.1) determining a first picture of the compressed picture sequence;

(13.2) if the first picture is I-picture, obtaining a first virtual picture according to a predetermined manner, sending the first virtual picture to a first buffer for display; and

(13.3) decoding the first picture and storing a decoded first picture into a second buffer.

14. The method of claim 13, wherein the predetermined manner, responsive to a parameter, generates the first virtual picture using a decoded picture pre-stored in a third buffer.

15. The method of 13, wherein the step (13.3) further comprises:

(15.1) determining a second picture;

(15.2) if the second picture is P-picture, decoding the second picture and storing a decoded second picture into a third buffer; and

(15.3) obtaining a second virtual picture according to the predetermined manner, sending the second virtual picture to the first buffer for display.

16. The method of claim 15, wherein the predetermined manner, responsive to a parameter, generates the second virtual picture using a decoded picture pre-stored in the second buffer.

17.The method of claim 13, wherein the step (13.3) further comprises:

(17.1) determining a second picture;

(17.2) if the second picture is I-picture, decoding the second picture and storing a decoded second picture into a third buffer; and

(17.3) obtaining a second virtual picture according to the predetermined manner, sending the second virtual picture to the first buffer for display.

18. The method of claim 17, wherein the predetermined manner, responsive to a parameter, generates the second virtual picture using a decoded picture pre-stored in the second buffer.

19. A method for reordering a decode order into a display order of an image, the decode order comprising an I-picture (intra-coded picture), a P-picture (predictive-coded picture), and a B-picture (bi-directionally-predictive coded picture), the method obtaining a compressed picture sequence and comprising the following steps:

(19.1) determining a first picture of the compressed picture sequence;

(19.2) if the first picture is P-picture, obtaining a first virtual picture according to a predetermined manner, sending the first virtual picture to a first buffer for display; and

(19.3) decoding the first picture and storing a decoded first picture into a second buffer.

20. The method of claim 19, wherein the predetermined manner, responsive to a parameter, generates the first virtual picture using a decoded picture pre-stored in a third buffer.

21. The method of claim 19, wherein the step (19.3) further comprises:

(21.1) determining a second picture;

(21.2) if the second picture is P-picture, decoding the second picture and storing a decoded second picture into the third buffer; and

(21.3) obtaining a second virtual picture according to the predetermined manner, sending the second virtual picture to the first buffer for display.

22. The method of claim 21, wherein the predetermined manner, responsive to a parameter, generates the second virtual picture using a decoded picture pre-stored in the second buffer.

23. The method of claim 21, wherein the step (21.3) further comprises:

(23.1) determining a second picture;

(23.2) if the second picture is I-picture, decoding the second picture and storing a decoded second picture into a third buffer; and

(23.3) obtaining a second virtual picture according to the predetermined manner, sending the second virtual picture to the first buffer for display.

24. The method of claim 23, wherein the predetermined manner, responsive to a parameter, generates the second virtual picture using a decoded picture pre-stored in the second buffer.

25. An apparatus for reordering a decode order into a display order of an image, the decode order comprising an I-picture (intra-coded picture), a P-picture (predictive-coded picture), and a B-picture (bi-directionally-predictive coded picture), the apparatus receiving a compressed picture sequence, comprising:

a parameter generator for generating a parameter;

a motion compensator for outputting a virtual picture according to a predetermined manner;

a memory comprising:

a first buffer for storing a first decoded picture of the decode order of an image;

a second buffer for storing the virtual picture;

a third buffer for storing a second decoded picture of the decode order of an image;

a display controller, connected to the second buffer, for displaying the virtual picture.

26. The apparatus of claim 25, wherein the motion compensator, in response to the parameter, generates the virtual picture using the first decode picture stored in the first buffer.