Method for detecting bisection pattern in deinterlacing
A method for detecting the Bisection pattern for use in conjunction with the deinterlacing is to improve low angle direction detection capability. A method of Bisection pattern detection operates in an interlaced video containing both top and bottom fields. For each missing pixel in a current input field, a window W is constructed whose center pixel is at the considered missing pixel. A binary map is generated which includes rows of values corresponding to pixels in the window, wherein the values indicate if each element of the window is greater than the sample mean of the area surrounding the missing pixel. The number of value changes in the values in each row of the binary map is counted. It is then determined whether or not the missing pixel is within the Bisection pattern based on said counts. In order to maintain both the low angle edge direction detection capability and low misdetection probability, complicated areas are identified by the Bisection pattern detection method. Then deinterlacing is performed.
Latest Samsung Electronics Patents:
- CLOTHES CARE METHOD AND SPOT CLEANING DEVICE
- POLISHING SLURRY COMPOSITION AND METHOD OF MANUFACTURING INTEGRATED CIRCUIT DEVICE USING THE SAME
- ELECTRONIC DEVICE AND METHOD FOR OPERATING THE SAME
- ROTATABLE DISPLAY APPARATUS
- OXIDE SEMICONDUCTOR TRANSISTOR, METHOD OF MANUFACTURING THE SAME, AND MEMORY DEVICE INCLUDING OXIDE SEMICONDUCTOR TRANSISTOR
The present invention relates to the field of image processing, and more particularly, to the application of Bisection detection algorithm in conjunction with edge direction detection for video deinterlacing in order to improve the misdetection probability of the low angle direction.
BACKGROUND OF THE INVENTIONDue to the recent advances of digital television (DTV), deinterlacing plays an important role in standard conversion between interlaced and progressive video scanning formats. For an image display of size M×N, where M and N denote the number of scan lines and number of pixels in each scan line, respectively, in an interlaced video scanning format, the picture formed by the stack of all even scan line m=0, 2, 4, . . . , M−2 is called top field (or even field). Each even scan line of the top field is called existing scan line while its odd counterpart whose pixels do not exist is called missing scan line. In a similar fashion, the bottom field (or odd field) is formed by the stack of all odd scan lines m=1, 3, . . . , M−1 which are the existing scan lines in this case. These two fields are displayed in an alternating manner at different time instants to form a complete image on the display.
The object of deinterlacing is to convert each M/2×N field (either top or bottom field) to a complete M×N image by interpolating each pixel in the missing scan lines from its neighboring (above and below) existing scan lines. Let d (in pixel unit) denote the detected direction obtained by the edge direction detection process. Referring to the example in
where m is the scan line number of the missing pixel, and n is the pixel number in the missing scan line.
In deinterlacing, conventional edge direction detection techniques typically misdetect when direction |d| is large (the very low angle direction). For example, in
The present invention addresses the above shortcomings. An object of the present invention is to provide a method for detecting the Bisection pattern for use in conjunction with the deinterlacing to improve low angle direction detection capability.
A method for detecting the Bisection pattern for use in conjunction with the deinterlacing is to improve low angle direction detection capability. A method of Bisection pattern detection operates in an interlaced video. For each missing pixel in a current input field, a window W is constructed whose center pixel is at the considered missing pixel. A binary map is generated which includes rows of values corresponding to pixels in the window, wherein the values indicate if each element of the window is greater than the sample mean of the area surrounding the missing pixel. The number of value changes in the values in each row of the binary map is counted. It is then determined whether or not the missing pixel is within the Bisection pattern based on said counts (there are two sets of value change counts corresponding to two rows). In order to maintain both the low angle edge direction detection capability and low misdetection probability, complicated areas are identified by the Bisection pattern detection method. Then deinterlacing is performed.
In one example according to the present invention, a method of deinterlacing with Bisection pattern detection for an interlaced field f of pixels, comprising the steps of: (a) for each considered missing pixel of the current input field f, detecting a direction d; (b) performing Bisection pattern detection to determine a Bisection pattern decision β; (c) determining a refined detected direction d′ by multiplying the Bisection pattern decision β with the detected direction d; and (d) interpolating the missing pixel by directional interpolation based on the refined detected direction d′.
Other embodiments, features and advantages of the present invention will be apparent from the following specification taken in conjunction with the following drawings.
In one embodiment, the present invention applies Bisection pattern detection in conjunction with deinterlacing. The general idea of the Bisection pattern is briefly discussed while its strict definition will be later delineated mathematically in the detailed description of the preferred embodiment section. Conceptually, the pattern of a collection of pixels within some window (or area) is called Bisection pattern if those pixels can be clearly separated into two regions (or sections): (1) a region containing pixels whose gray levels are greater than a sample mean in that area, and (2) a region containing pixels whose gray levels are not greater than a sample mean in that area. The areas containing Bisection patterns are very well-structured areas and are considered uncomplicated areas. An example of the Bisection pattern within a specified window is shown in
By contrast, the areas which do not contain Bisection patterns are considered as complicated areas. Examples of the areas not containing Bisection pattern are such as texture and thin line (foreground) in the uniform background are shown in
In the portion of field 500B in
An example Bisection pattern detection according to the present invention includes the steps of: (a) for each missing pixel in the current input field, constructing the 2×(2K+1) window whose center pixel is at the considered missing pixel, for some integer K≧0; (b) generating the binary map by determining whether or not each element of window is greater than the sample mean calculated from the area surrounding the considered missing pixel; (c) calculating the numbers of value changes by counting the number of changes from 0 to 1 or vice versa in each row of binary map as in step (b); and (d) determining whether or not the missing pixel is within the Bisection pattern using the number of value changes in step (c). One example of value change is a sign change.
The refined detected direction d′ is input to the linear directional interpolator 108 along with the input field, where associated calculation is performed according to relation (1) above as shown by example in
After each missing pixel in the current field is linear directional interpolated using its associated refined detected direction d′, the resulting output is a complete image or frame (instead of field) corresponding to its current input field.
For arbitrary Boolean variable Aε{TRUE,FALSE}, the predicate function Pred is defined as:
After the window W is constructed in accordance with relation (3), each element of the window W is determined in a binary map generator 604, whether or not that element is greater than the sample mean
The sample mean
After the binary map B is obtained, the number of value changes s−1, and s1, of elements in binary vectors b−1, and b1, are computed by a value change counter 606 as:
Thereafter, the Bisection pattern determinator 608 determines the Bisection pattern decision β. Let S denote the maximum number of value changes allowed. Then, the pattern of pixels within the window W is called Bisection if and only if the Bisection pattern decision β of window W equals one, where β is defined as:
β=Pred(s−1≦S AND s1≦S). (8)
As such, if both numbers of value changes, s−1 and s1, are less than or equal to the maximum number of value changes allowed S, the considered missing pixel is classified to be in the Bisection pattern and the Bisection pattern decision β is set to 1; otherwise, β is set to 0.
Other values for S can be used, however, based on extensive simulation, the maximum number of value change S allowed is set to 1 and the integer K is chosen to be 4 (so that width of the window W is 2K+1=9).
The directional linear interpolation could be replaced with a more advanced interpolation such as cubic spline interpolation which requires 4 existing pixels: ƒ[m−3,n+3d], ƒ[m−1,n+d], ƒ[m+1,n−d], and ƒ[m+3,n−3d]) or higher order B-spline interpolations.
To lower the computation burden, the Bisection pattern detection can be implemented as a pre-processing step. For example, instead of detecting direction d and Bisection pattern decision β in parallel, the Bisection pattern decision β can be calculated first and its result used to determine whether edge direction detection should be performed or not. If β=0, the detected direction d is set to vertical direction without the implementation of edge direction detection; otherwise, the edge direction detection process is performed as usual.
Further, to double check the fidelity of refined detected direction d′, the Bisection pattern detection can be used in the post-processing step. Instead of the fixed window width (e.g., 2K+1), the window width of the Bisection pattern detection in the post-processing step can be adaptive to the refined detected direction d′. Based on extensive simulation, the refined detected direction d′ and its corresponding window width (2K+1) are provided in the following coordinates (d′, 2K+1) as: (1, 3), (−1, 3), (2, 5), (−2, 5), (3, 7), (−3, 7), (4, 7), (−4, 7), (5, 7), (−5, 7), (6, 7), (−6, 7), (7, 7), (−7, 7), (8, 9), and (−8, 9).
As such, the present invention provides a method for detecting the Bisection pattern for use in conjunction with the deinterlacing to improve low angle direction detection capability. In edge direction detection process of deinterlacing, it is known that there is a trade-off between low angle direction detection capability and misdetection probability. That is, if the detected direction is detected as a very low angle direction, it is very likely to be an incorrect direction detection. Such misdetection usually occurs in the complicated areas such as texture and thin line (foreground) in the uniform background. In order to maintain both the low angle edge direction detection capability and low misdetection probability, those complicated areas must be identified first. Accordingly, the present invention provides a method and system for detecting the Bisection pattern which is inversely related to those complicated areas.
The present invention has been described in considerable detail with reference to certain preferred versions thereof; however, other versions are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the preferred versions contained herein.
Claims
1. A method of deinterlacing with Bisection pattern detection for an input field f, comprising the steps of:
- (a) for each considered missing pixel of the current input field f, detecting a direction d;
- (b) performing Bisection pattern detection to determine a Bisection pattern decision β;
- (c) determining a refined detected direction d′ by multiplying the Bisection pattern decision β with the detected direction d; and
- (d) interpolating the missing pixel by interpolation based on the refined detected direction d′.
2. The method of claim 1 further comprising repeating steps (a) through (d) until each missing pixel of the current input field is interpolated.
3. The method of claim 1, wherein the step of Bisection pattern detection further comprises the steps of:
- constructing a window W whose center pixel is at the considered missing pixel f[m,n];
- generating a binary map B including rows of values corresponding to pixels in the window, wherein the values indicate if each element of the window is greater than the sample mean of the area surrounding the missing pixel;
- counting the number of value changes in each row of the binary map; and
- determining a Bisection pattern decision β based on said counts.
4. The method of claim 3 wherein the step of constructing the window W further comprises the steps of constructing a 2×(2K+1) window W whose center pixel is at the considered missing pixel f[m,n].
5. The method of claim 4 wherein the step of constructing the window W further comprises the steps of determining the window W as: W = [ ρ - 1 ρ 1 ], where ρ i = { ρ i [ j ] = f [ m + i, n + j ]: - K ≤ j ≤ K }.
6. The method of claim 4 wherein the step of generating the binary map further includes the steps of generating the binary map B as: B = [ b - 1 b 1 ], where b i = { b i [ j ] = Pred ( ρ i [ j ] > W _ ): - K ≤ j ≤ K }; W _ = 1 4 ( 2 K + 1 ) ∑ i ∈ { - 3, - 1, 1, 3 } ∑ j = - K K f [ m + i, n + j ], and Pred ( A ) = { 1 if A = TRUE; 0 if A = FALSE.
- the sample mean W is calculated as:
7. The method of claim 6 wherein the step of counting the numbers of value changes s−1 and s1 further includes the steps of determining: s i = ∑ j = - K K - 1 b i [ j ] - b i [ j + 1 ] , for i=−1 and 1, where said counts are for the numbers of value changes s−1 and s1 of elements in binary vectors b−1 and b1 in the binary map B.
8. The method of claim 7 wherein the step of calculating the Bisection pattern decision β further comprises the steps of calculating:
- β=Pred(s−1≦S AND s1≦S),
- where S denotes the maximum number of value changes allowed.
9. The method of claim 1 wherein the step of interpolation further comprises the steps of performing directional interpolation.
10. The method of claim 1 wherein the step of interpolation further comprises the steps of performing spline interpolation based on cubic spline interpolation using 4 existing pixels: ƒ[m−3,n+3d], ƒ[m−1,n+d], ƒ[m+1,n−d], and ƒ[m+3,n−3d]).
11. The method of claim 1 wherein the step of interpolation further comprises the steps of performing interpolation by B-spline interpolation.
12. The method of claim 8 wherein the maximum number of value change S allowed is 1 and the integer K is selected to be 4 such that width of the window W is 2K+1=9.
13. The method of claim 1 wherein the step of Bisection pattern detection is performed before the step of detecting the direction d, wherein the Bisection pattern decision β is used to determine whether the edge direction detection should be performed or not.
14. The method of claim 13 wherein if β=0, the detected direction d is set to vertical direction without performing the step of edge direction detection; otherwise, the edge direction detection process is performed.
15. The method of claim 3 further comprising the step of determining the fidelity of refined detected direction d′.
16. The method of claim 15, wherein the step of determining the fidelity of refined detected direction d′ further comprises the steps of performing Bisection pattern detection in a post-processing step.
17. The method of claim 16 wherein the window width of the Bisection pattern detection is adaptive to the refined detected direction d′.
18. The method of claim 17 wherein the step of constructing the window W further comprises the steps of constructing a 2×(2K+1) window W whose center pixel is at the considered missing pixel, where integer K≧0.
19. The method of claim 18 wherein the refined detected direction d′ and its corresponding window width (2K+1) are (d′, 2K+1) as: (1, 3), (−1, 3), (2, 5), (−2, 5), (3, 7), (−3, 7), (4, 7), (−4, 7), (5, 7), (−5, 7), (6, 7), (−6, 7), (7, 7), (−7, 7), (8, 9), and (−8, 9).
20. A method of Bisection pattern detection in an interlaced field of pixels, comprising the steps of:
- (a) for each missing pixel in a current input field, constructing window W whose center pixel is at the considered missing pixel;
- (b) generating a binary map including rows of values corresponding to pixels in the window, wherein the values indicate if each element of the window is greater than the sample mean of the area surrounding the missing pixel;
- (c) counting the number of changes in the values in each row of the binary map; and
- (d) determining whether or not the missing pixel is within the Bisection pattern based on said counts.
21. The method of claim 20, wherein the step of constructing the window W further includes the steps of, for each missing pixel in a current input field, constructing a 2×(2K+1) window W whose center pixel is at the considered missing pixel, where integer K≧0.
22. A video processing system for deinterlacing with Bisection pattern detection for an input field f, comprising:
- (a) a direction detector that for each considered missing pixel of the current input field f, detects a direction d;
- (b) a pattern detector that performs Bisection pattern detection to determine a Bisection pattern decision β;
- (c) a direction refiner that determines a refined detected direction d′ by multiplying the Bisection pattern decision β with the detected direction d; and
- (d) an interpolator that interpolates the missing pixel by interpolation based on the refined detected direction d′.
23. The system of claim 22 wherein each missing pixel of the current input field is interpolated.
24. The system of claim 22 wherein the pattern detector further comprises:
- a window constructor that constructs a window W whose center pixel is at the considered missing pixel f[m,n];
- a binary map generator that generates a binary map B including rows of values corresponding to pixels in the window, wherein the values indicate if each element of the window is greater than the sample mean of the area surrounding the missing pixel;
- a counter that counts the number of values changes in each row of the binary map; and
- a pattern decision calculator that determining a Bisection pattern decision β based on said counts.
25. The system of claim 24 wherein the window constructor constructs a 2×(2K+1) window W whose center pixel is at the considered missing pixel f[m,n].
26. The system of claim 25 wherein the window constructs the window W as: W = [ ρ - 1 ρ 1 ], where ρ i = { ρ i [ j ] = f [ m + i, n + j ] : - K ≤ j ≤ K }.
27. The system of claim 25 wherein the binary map generator generates the binary map B as: B = [ b - 1 b 1 ], where b i = { b i [ j ] = Pred ( ρ i [ j ] > W _ ) : = - K ≤ j ≤ K }; W _ = 1 4 ( 2 K + 1 ) ∑ i ∈ { - 3, - 1, 1, 3 } ∑ j = - K K f [ m + i, n + j ], and Pred ( A ) = { 1 if A = TRUE; 0 if A = FALSE.
- the sample mean W is calculated as:
28. The system of claim 27 wherein the counter counts the numbers of value changes s−1 and s1 by determining: s i = ∑ j = - K K - 1 b i [ j ] - b i [ j + 1 ] , for i=−1 and 1, where said counts are for the numbers of value changes s−1 and s1, of elements in binary vectors b−1 and b1 in the binary map B.
29. The system of claim 28 wherein the pattern decision calculator calculates the Bisection pattern decision β as:
- β=Pred(s−1≦S AND s1≦S),
- wherein S denotes the maximum number of value changes allowed.
30. The system of claim 22 wherein the interpolator performs interpolation by directional interpolation.
31. The system of claim 22 wherein the interpolator comprises a cubic spline interpolator using 4 existing pixels: ƒ[m−3,n+3d], ƒ[m−1,n+d], ƒ[m+1,n−d], and ƒ[m+3,n−3d]).
32. The system of claim 22 wherein interpolator comprises a B-spline interpolator.
33. The system of claim 29 wherein the maximum number of value change S allowed is 1 and the integer K is selected to be 4 such that width of the window W is 2K+1=9.
34. The system of claim 22 wherein the pattern detector performs said Bisection pattern diction before the direction detector detects the direction d, wherein the Bisection pattern decision β is used to determine whether the edge direction detection should be performed or not.
35. The system of claim 34 wherein if β=0, the detected direction d is set to vertical direction without edge direction detection, otherwise, edge direction detection is performed.
36. The system of claim 24 wherein the pattern detector determines the fidelity of refined detected direction d′.
37. The system of claim 36 wherein the pattern detector checks the fidelity of refined detected direction d′ by performing Bisection pattern detection as a post-processor.
38. The system of claim 37 wherein the window width of the Bisection pattern detection is adaptive to the refined detected direction d′.
39. The system of claim 38 wherein the window constructor constructs the window W as a 2×(2K+1)window whose center pixel is at the considered missing pixel, where integer K≧0.
5519451 | May 21, 1996 | Clatanoff et al. |
5579053 | November 26, 1996 | Pandel |
5638139 | June 10, 1997 | Clatanoff et al. |
6924844 | August 2, 2005 | Kawamura et al. |
20030076447 | April 24, 2003 | Wang |
- G. De Haan and E.B. Bellers, Deinterlacing—An Overview, Proceedings of the IEEE, Sep. 1998, pp. 1839-1857, vol. 86, No. 9.
- M.K. Park, et al., New Edge Dependent Deinterlacing Algorithm Based on Horizontal Edge Pattern, IEEE Transactions on Consumer Electronics, Nov. 2003, pp. 1508-1512, vol. 49, No. 4.
Type: Grant
Filed: May 2, 2005
Date of Patent: Jul 22, 2008
Patent Publication Number: 20060244861
Assignee: Samsung Electronics Co., Ltd. (Suwon)
Inventors: Surapong Lertrattanapanich (Santa Ana, CA), Yeong-Taeg Kim (Irvine, CA)
Primary Examiner: M. Lee
Attorney: Myers Dawes Andras & Sherman, LLP
Application Number: 11/121,815
International Classification: H04N 7/01 (20060101); H04N 11/20 (20060101);