Interpolation method for binary image
An improved interpolation method in which a threshold value used for determining a pixel value of a pixel generated by interpolation according to a context which is a state value of adjacent pixels. In the interpolation method, the ambiguity between the interpolation value and the threshold value is removed by using the context, thereby reducing the blocking and smoothing phenomena in the restored binary image.
Latest Samsung Electronics Patents:
This is a continuation of U.S. patent application Ser. No. 08/979,107, filed Nov. 26, 1997, now U.S. Pat. No. 6,002,812, and claims priority to Korean Patent Application Nos. 97-32102, 97-51105, and 97-54869 filed in Korea on Jul. 10, 1997, Oct. 4, 1997, and Oct. 24, 1997, respectively, all of which are herein incorporated by reference.
BACKGROUND OF THE INVENTION1. Field of the Invention
The present invention relates to an interpolation method for a binary image, and more particularly, to an improved interpolation method in which a variable threshold value used for determining a pixel value to be generated by interpolation is determined according to a context (the state value of adjacent pixels). This invention has been adopted in ISO/IEC JTC1/SC29/WG11 N1903 (ISO/IEC 14496-2 (Subpart 7 Committee Draft).
2. Description of the Related Art
Recently, a function for processing shape information has been added to MPEG-4. The shape information, which refers to object information of the image, is expressed as a binary image. To code such a binary image, MPEG-4 adopts a context-based arithmetic encoder (CAE). For lossy shape coding a down sampling method and an up sampling method are performed in each macro block of binary shape. A shape image is divided into shape blocks having an M×M size.
The down sampling refers to a method for reducing the binary image block according to a given conversion ratio. The reduced image block is transmitted together with the conversion ratio. Here, the conversion ratio is determined such that an error between the original binary image block and an binary image block restored later is within a predetermined range.
The reduced image block obtained by the down sampling is coded by a CAE and then transmitted.
The up sampling method is used to restore such a reduced image block. Up sampling is for restoring the reduced image block into block with the size of the original binary image block, by interpolation.
In such an up sampling process, an effective interpolation method must not cause excessive blocking and smoothing effects in the restored binary image.
SUMMARY OF THE INVENTIONTo satisfy the above requirement, it is an object of the present invention to provide an improved interpolation method in which a context (state value of reference pixels adjacent to a pixel (interpolated pixel) generated by interpolation) is used for interpolation, thereby reducing blocking and smoothing effects.
To achieve the above object, there is provided an interpolation method for a binary image, for restoring a binary image block reduced through a down sampling into the block with the size of original binary image block, the method comprising the steps of: (a) preparing a threshold table showing various threshold values corresponding to a context Cp (state value) of pixels (reference pixels) of the reduced image, around an interpolated pixel; (b) calculating an interpolation value based on the pixel values of pixels (object pixels) adjacent and/or around the interpolated pixel; (c) calculating the context Cp which is the state value of the reference pixels around the interpolated pixel; (d) obtaining a threshold value corresponding to the calculated context from the threshold table; and (e) comparing the interpolation value with the threshold value of the step (d), and setting the pixel value of the interpolated pixel as “1” if the interpolation value is greater than the threshold value, and setting the pixel value of the interpolated pixel as “0” if the interpolation value is equal to or less than the threshold value.
Preferably, the step (a) comprises the sub-steps of: (a1) calculating possible sums of the reference pixels; (a2) calculating possible interpolation values with respect to the possible sums of the reference pixels; (a3) setting candidate threshold values with respect to the possible sum of the reference pixels; and (a4) selecting a threshold value from the candidate threshold values with respect to each context corresponding to the possible sum of the reference pixels, resulting in the threshold table.
The invention may be embodied in a general purpose digital computer that is running a program from a computer usable medium including but not limited to storage media such as magnetic storage media (e.g., ROM's, floppy disks, hard disks, etc.), optically readable media (e.g., CD-ROMs, DVDs, etc.), hybrid formats (magneto optical disks) and carrier waves (e.g., transmissions over the Internet). For instance, a part of the present invention can be computer usable medium having computer readable program code means embodied therein for processing by a machine, the computer usable medium having embodied thereon a computer program for restoring a binary image block reduced through a down sampling into the block with the size of the original binary image block, the computer program being executable by the machine to perform:
-
- (a) preparing a threshold table showing various threshold values corresponding to a context Cp (state value) of pixels (reference pixels) of the reduced image, around an interpolated pixel;
- (b) calculating an interpolation value based on the pixel values of pixels (object pixels) adjacent to and/or around the interpolated pixel;
- (c) calculating the context Cp which is the state value of the reference pixels around the interpolated pixel;
- (d) obtaining a threshold value corresponding to the calculated context from the threshold table; and
- (e) comparing the interpolation value with the threshold value of the step (d), and setting the pixel value of the interpolated pixel as “1” if the interpolation value is greater than the threshold value, and setting the pixel value of the interpolated pixel as “0” if the interpolation value is equal to or less than the threshold value.
The above object and advantages of the present invention will become more apparent by describing in detail a preferred embodiment thereof with reference to the attached drawings in which:
In
In the down sampling step 100, an M×N binary image block is converted into an (M×Cr)×(N×CR) binary image block. Here, CR is a conversion ratio indicating the ratio in size of the reduced image block obtained by the down sampling with respect to the original binary image block.
In the coding step 102, the reduced image is encoded. In order to code the binary image including objects, MPEG-4 adopts a context-based arithmetic encoder (CAE). The CAE is adopted as a coding method in the verification model of MPEG-4, due to its simplicity and comparatively high coding efficiency.
The reduced image coded through the coding step 102 is transmitted via a transmission route. In the reverse coding step 104, the coded reduced image is restored into the reduced image. In the up sampling step 106, the interpolation is performed on the reduced image block to obtain the block with the size of the original binary image.
In the down sampling step, if half or more pixels of four pixels (indicated by “O”) are equal to “1”, the pixel value of the converted pixel (indicated by “X”) becomes “1”. Here, pixels having value “1” represent a part of the picture containing an object, and pixels having value “O” represent a part of the picture outside an object. The macro block including both pixels having value “1” and pixels having value “0” represents the boundary block of the object information. The pixel values are assigned arbitrarily and though convention. The values can, of course, be reversed.
Basically, pixels (indicated by “X”) of a restored shape block 300 are obtained from the neighboring pixels (indicated by “O”) of the reduced shape macro block as shown in the circle 303 of FIG. 1C.
For example, pixels of an upper border 301 and a left border 302 are referred to as down-sampled pixels which belong to the upper macro block and left macro block, respectively.
When the pixels of lower boundary and right boundary of macro block 300 refer to the neighboring pixels outside the block 300, the pixels outside the block can be obtained by extending the outermost pixels inside the block as shown in FIG. 1C.
Referring to
In
First, an interpolation value INP[P] is obtained as follows:
INP[P1]=r*A+s*(B+C+D)+t*(E+F+G+H+I+J+K+L)
INP[P2]=r*B+s*(A+C+D)+t*(E+F+G+H+I+J+K+L)
INP[P3]=r*C+s*(A+B+D)+t*(E+F+G+H+I+J+K+L)
INP[P4]=r*D+s*(A+B+C)+t*(E+F+G+H+I+J+K+L)
where r, s and t are weights according to the distance from the interpolated pixel, that is, r is the weight of the object pixel which is the closest to the interpolated pixel, s is the weight of object pixels which are the next closest to the interpolated pixel, and t is the weight of object pixels around the next closest pixels. Also, the condition of r>s>t is satisfied.
Next, the obtained interpolation value INP[P]and a threshold value THR are compared. Here, the threshold value THR is set to a value which is half of the largest possible interpolation value. For example, if r, s and t are 4, 2 and 1, respectively, then the threshold value is set to 9. If the interpolation value INP[P] is greater than the threshold value THR, the pixel value of the interpolated pixel becomes “1”. Otherwise, the pixel value of the interpolated pixel becomes “0”.
Here, only one threshold THR is applied while a plurality of interpolation values INP may be applied. Thus, if the interpolation value INP[P] and the threshold value THR are close to each other, it is unclear whether the pixel value of the interpolated pixel is correct or not. Accordingly, serious blocking or smoothing phenomenon can be present in the restored binary image.
To reduce the blocking or smoothing effects, a threshold value to be compared with the interpolation value INP[P] is adaptively determined according to the context (state values of the pixels (reference pixels) around the interpolated pixel), thereby reducing ambiguity in determination of the pixel value of the interpolated pixel.
An improved interpolation method according to a preferred embodiment of the present invention will be described with reference to
First, an interpolation value INP[P] is obtained by the object pixels as follows:
INP[P1]=r*A+s*(B+C+D)+t*(E+F+G+H+I+J+K+L)
INP[P2]=r*B+s*(A+C+D)+t*(E+F+G+H+I+J+K+L)
INP[P3]=r*C+s*(A+B+D)+t*(E+F+G+H+I+J+K+L)
INP[P4]=r*D+s*(A+B+C)+t*(E+F+G+H+I+J+K+L)
where r, s and t are weights according to the distance from the interpolated pixel, that is, r is the weight of the object pixel which is the closest to the interpolated pixel, s is the weight of object pixels which are the next closest to the interpolated pixel, and t is the weight of object pixels around the next closest pixels. Also, the condition of r>s>t is satisfied. Here, r, s and t are 4, 2 and 1, respectively.
Then, the context Cp which is state value of the reference pixels is calculated by the following equation (1):
where P represents the position of the interpolated pixel, O represents the reference pixel, and k is an index of the reference pixels and also a weight. Here, the index k is varied depending on the relative position between the interpolated pixel and the adjacent object pixels.
Next, a threshold value THRc according to the calculated context is determined with reference to the threshold value table shown in FIG. 4. The threshold value of
In the threshold table of
Since the number of the reference pixels for calculating context is 8, the number of potential combinations become 256.
In
In
The interpolation value INP[P] and the threshold value THRc according to the context of the object pixels are a compared to determine a pixel value of the interpolated pixel.
If the interpolation value INP[P] is greater than the threshold value THRc, the pixel value of the interpolated pixel is set to “1”. Otherwise, the pixel value of the interpolated pixel is set to “0”.
The threshold table used in the present invention is effectively determined in consideration of reference pixel values. The procedure for preparing the threshold table in consideration of the reference pixel values will be described with reference to
-
- 1) Possible sums of the reference pixels are calculated.
Here, the sum of the reference pixels represents the sum of pixel values of the reference pixels which contributes to calculate the context. For example, it refers to the sum of the pixel values of the reference pixels indicated by E, F, G, H, I, J, K and L in FIG. 3.
In
-
- 2) Possible interpolation values with respect to the possible sums of the reference pixels are calculated.
For example, if the sum of the reference pixels is equal to “0”, that is, all the pixel values of the reference pixels are “0”, the interpolation value is one of 0, 2, 4, 6, 8 and 10. If all the pixel values of the adjacent object pixels A, B, C and D are “0”, the interpolation value becomes “0”. Also, if all the pixel values of the adjacent object pixels A, B, C and C are 37 1”, the interpolation value becomes “10”.
In the same manner, if the sum of the reference pixels is equal to “8”, that is, if all the pixel values of the reference pixels are “1”, the interpolation value becomes one of 8, 10, 12, 14, 16 and 18.
-
- 3) Possible candidate threshold values with respect to the possible sum of the reference pixels are determined.
-
- 4) A threshold value with respect to each context value corresponding to the possible sums of the reference pixels is selected among the possible candidate threshold values. Here, the threshold value may be the median among the possible threshold values.
- 5) The table is prepared by matching the contexts with the selected threshold values. Here, each context matches with the threshold value with respect to the sum of the corresponding reference pixels.
In
If the context value is equal to “0”, that is, if the sum of the reference pixels is “0” in
If the context value is equal to “1”, that is, if the sum of the reference pixels is “1” in
If the context value is equal to “2”, that is, if the sum of the reference pixels is “2” in
If the context value is equal to “255”, that is, if the sum of the reference pixels is “8” in
Also,
In the interpolation method of the present invention, the ambiguity in the comparison between the interpolation value and the threshold value is removed by using the state value (context) of the reference pixels around the interpolated pixel, thereby reducing the blocking and smoothing phenomena in the restored binary image.
The invention may be embodied in a general purpose digital computer that is running a program or program segments originating from a computer readable or usable medium, such medium including but not limited to magnetic storage media ((e.g., ROM's, floppy disks, hard disks, etc.), optically readable media (e.g., CD-ROMs, DVDs, etc.) and carrier waves (e.g., transmissions over the Internet). A functional program, code and code segments, used to implement the present invention can be derived by a skilled computer programmer from the description of the invention contained herein.
As can be seen from
As described above, in the interpolation method of the present invention, the ambiguity in the comparison between the interpolation value and the threshold value is removed by using the state value (context) of the reference pixels around the interpolated pixel, thereby reducing the blocking and smoothing phenomena in the restored binary image.
In the interpolation method of the present invention, since variable threshold values according to the context combination are quickly obtained from the threshold table, the interpolation can be performed rapidly.
Claims
1. A computer usable medium having embodied thereon a computer program for restoring a binary image block reduced through a down sampling into the block with the size of the original binary image block, the computer program being executable by a machine to perform:
- (a) preparing a threshold table showing various threshold values corresponding to a context Cp (state value) of pixels (reference pixels) of the reduced image, around an interpolated pixel;
- (b) calculating an interpolation value based on the pixel values of pixels (object pixels) adjacent to and/or around the interpolated pixel;
- (c) calculating the context Cp which is the state value of the reference pixels around the interpolated pixel;
- (d) obtaining a threshold value corresponding to the calculated context from the threshold table; and
- (e) comparing the interpolation value with the threshold value of the step (d), and setting the pixel value of the interpolated pixel as “1” if the interpolation value is greater than the threshold value, and setting the pixel value of the interpolated pixel as “0” if the interpolation value is equal to or less than the threshold value.
2. The computer usable medium of claim 1, wherein the computer program at (b) calculates by an interpolation method using the following equations: where P1, P2, P3 and P4 represent an interpolated pixel, A, B, C and D are the objective pixels adjacent to the interpolated pixel, E, F, G, H, I, J, K and L are the objective pixels around the pixels A, B, C and D, INP[P] represents the interpolation value of the interpolated pixel P, r is the weight of the object pixel which is the closest to the interpolated pixel, s is the weight of three object pixels which are the next closest to the interpolated pixel, and t is the weight of eight object pixels around the next closest pixels, and the condition of r>s>t is satisfied.
- INP[P1]=r*A+s*(B+C+D)+t*(E+F+G+H+I+J+K+L)
- INP[P2]=r*B+s*(A+C+D)+t*(E+F+G+H+I+J+K+L)
- INP[P3]=r*C+s*(A+B+D)+t*(E+F+G+H+I+J+K+L)
- INP[P4]=r*D+s*(A+B+C)+t*(E+F+G+H+I+J+K+L)
3. The computer usable medium of claim 1, wherein the computer program at (c) is performed using the following equation: C p = ∑ 0 k O k · 2 k where P represents the position of the interpolated pixel, O represents the reference pixel, and k is an index of the reference pixels.
4. The computer usable medium of claim 3, wherein the index k of the computer program at (c) is varied according to the location of the interpolated pixels with respect to the adjacent pixels.
5. The computer usable medium of claim 1, wherein the computer program at (a) preforms:
- (a1) calculating possible sums of the reference pixels;
- (a2) calculating possible interpolation values with respect to the possible sums of the reference pixels;
- (a3) setting possible candidate threshold values with respect to the possible sum of the reference pixels; and
- (a4) selecting a threshold value from the candidate threshold values with respect to each context corresponding to the possible sum of the reference pixels, resulting in the threshold table.
6. The computer usable medium of claim 5, wherein the candidate threshold values of the computer program at (a3) are the median values of every adjacent pair of the possible interpolation values which are arranged in size.
7. The computer usable medium of claim 5, wherein the computer program at (a4) selects the median values of the candidate threshold values with respect to each context as the threshold value.
8. The computer usable medium of claim 7, wherein the computer program being further executable by a machine to perform adjusting the threshold values of the table by allowing a predetermined variable range for the selected threshold values and then selecting a threshold value with respect to each context, showing the least error in reproduction when the threshold values are applied to an actual binary image.
9. An apparatus comprising:
- (a) means for preparing a threshold table showing various threshold values corresponding to a context Cp (state value) of pixels (reference pixels) of the reduced image, around an interpolated pixel;
- (b) means for calculating an interpolation value based on the pixel values of pixels (object pixels) adjacent to and/or around the interpolated pixel;
- (c) means for calculating the context Cp which is the state value of the reference pixels around the interpolated pixel;
- (d) means for obtaining a threshold value corresponding to the calculated context from the threshold table; and
- (e) means for comparing the interpolation value with the threshold value of the step (d), and setting the pixel value of the interpolated pixel as “1” if the interpolation value is greater than the threshold value, and setting the pixel value of the interpolated pixel as “0” if the interpolation value is equal to or less than the threshold value.
10. The apparatus of claim 9, wherein the interpolation calculating means calculates by an interpolation method using the following equations: where P1, P2, P3 and P4 represent an interpolated pixel, A, B, C and D are the objective pixels adjacent to the interpolated pixel, E, F, G, H, I, J, K and L are the objective pixels around the pixels A, B, C and D, INP[P] represents the interpolation value of the interpolated pixel P, r is the weight of the object pixel which is the closest to the interpolated pixel, s is the weight of three object pixels which are the next closest to the interpolated pixel, and t is the weight of eight object pixels around the next closest pixels, and the condition of r>s>t is satisfied.
- INP[P1]=r*A+s*(B+C+D)+t*(E+F+G+H+I+J+K+L)
- INP[P2]=r*B+s*(A+C+D)+t*(E+F+G+H+I+J+K+L)
- INP[P3]=r*C+s*(A+B+D)+t*(E+F+G+H+I+J+K+L)
- INP[P4]=r*D+s*(A+B+C)+t*(E+F+G+H+I+J+K+L)
11. The apparatus of claim 9, wherein the context calculating means is performed using the following equation: C p = ∑ 0 k C k · 2 k where P represents the position of the interpolated pixel, O represents the reference pixel, and k is an index of the reference pixels.
12. The apparatus of claim 11, wherein the index k used in the context calculating means is varied according to the location of the interpolated pixels with respect to the adjacent pixels.
13. The apparatus of claim 9, wherein the threshold table preparing means includes:
- (a1) means for calculating possible sums of the reference pixels;
- (a2) means for calculating possible interpolation values with respect to the possible sums of the reference pixels;
- (a3) means for setting possible candidate threshold values with respect to the possible sum of the reference pixels; and
- (a4) means for selecting a threshold value from the candidate threshold values with respect to each context corresponding to the possible sum of the reference pixels, resulting in the threshold table.
14. The apparatus of claim 13, wherein the candidate threshold values set by the possible candidate threshold value setting means are the median values of every adjacent pair of the possible interpolation values which are arranged in size.
15. The apparatus of claim 13, wherein the threshold value selecting means selects the median values of the candidate threshold values with respect to each context as the threshold value.
16. The apparatus of claim 15, further comprising means for adjusting the threshold values of the table by allowing a predetermined variable range for the selected threshold values and then selecting a threshold value with respect to each context, showing the least error in reproduction when the threshold values are applied to an actual binary image.
4771471 | September 13, 1988 | Kitamura |
5054100 | October 1, 1991 | Tai |
5754706 | May 19, 1998 | Schweid et al. |
5799113 | August 25, 1998 | Lee |
5805305 | September 8, 1998 | Abe |
5818964 | October 6, 1998 | Itoh |
5832143 | November 3, 1998 | Suga et al. |
6088489 | July 11, 2000 | Miyake |
4199477 | July 1992 | JP |
7182503 | July 1995 | JP |
A-7-182503 | July 1995 | JP |
8116430 | May 1996 | JP |
A-8-116430 | May 1996 | JP |
- Sikora, T., The MPEG-4 Video Standard Verification Model, IEEE Transactions on Circuits and Systems for Video Technology, vol. 7, No. 1, Feb. 1997, pp. 19-31.
- Office Action issued by the Japanese Patent Office on Feb. 24, 2004 in connection with corresponding application.
Type: Grant
Filed: Dec 12, 2001
Date of Patent: Aug 15, 2006
Assignee: Samsung Electronics Co., Ltd. (Suwon-si)
Inventors: Dae-sung Cho (Seoul), Jae-seob Shin (Seoul)
Primary Examiner: Bhavesh M. Mehta
Assistant Examiner: Gregory Desire
Attorney: Buchanan Ingersoll PC
Application Number: 10/012,527
International Classification: G06K 9/32 (20060101);