Color interpolation method
Disclosed is a color interpolation method. The present invention decides a precise position of an edge with use of a G value in a unit pixel structure with a size of 3×3, thereby using different color interpolations according to the position of the edge. Also, the present invention provides an effect of emphasizing an edge by emphasizing a brightness and lowering colors when the edge is placed in the vertical center or the horizontal center of the unit pixel structure of 3×3 with use of a property that the edge has stronger brightness than the colors and prevents an incorrect color.
Latest Intellectual Ventures II LLC Patents:
- Device, system and method for controlling speed of a vehicle using a positional information device
- Digital cameras with direct luminance and chrominance detection
- SCHEDULING TRANSMISSIONS ON CHANNELS IN A WIRELESS NETWORK
- PAGING IN A WIRELESS NETWORK
- COMMUNICATING OVER MULTIPLE RADIO ACCESS TECHNOLOGIES (RATS)
The present invention relates to a color interpolation method of an image sensor; and more particularly, to a color interpolation method considering edges at a complementary metal oxide semiconductor (CMOS) device.
DESCRIPTION OF RELATED ARTSAn image sensor is a device producing an image by using a characteristic that a semiconductor device reacts to a light. That is, a pixel detects different brightness and wavelengths of lights coming from each different subject and produces the lights in an electrical value. Herein, the image sensor serves a role in converting this electrical value into a level capable of performing a signal processing.
When obtaining perfect red, green and blue colors from products using the image sensor, there are three methods to be used. A first method is to obtain an average value of red, green and blue values coming from the image sensor by using a peripheral pixel of 3×3. A second method is to obtain the average value by aligning the red, green and blue values according to a size of the each color. A third method is to replace the red, green and blue colors with an adjacent pixel. However, by obtaining the average value or replacing the colors, the edges can be damaged in an image or a false color can be generated. There is a conventional color interpolation method considering the edges, however, the conventional color interpolation method only detects edges in a horizontal or a vertical direction. Accordingly, the color interpolation cannot consider fine edges and correct the colors with characteristics of the edges.
SUMMARY OF THE INVENTIONIt is, therefore, an object of the present invention to provide a color interpolation method capable of keeping characteristics of edges by considering fine edges.
In accordance with one aspect of the present invention, there is provided a color interpolation method, including the steps of: defining four R-type, B-type, Gb-type and Gr-type unit pixel structures with a size of 3×3 comprised of one pixel subjected to the color interpolation and eight pixels surrounding the pixel subjected to the color interpolation, wherein the R-type unit pixel structure is comprised of a first pixel of the R-type B1, a second pixel of the R-type G2, a third pixel of the R-type B3, a fourth pixel of the R-type G4, a fifth pixel of the R-type R5, a sixth pixel of the R-type G6, a seventh pixel of the R-type B7, an eight pixel of the R-type G8 and a ninth pixel of the R-type B9, the Gb-type unit pixel is comprised of a first pixel of the Gb-type G1, a second pixel of the Gb-type R2, a third pixel of the Gb-type G3, a fourth pixel of the Gb-type B4, a fifth pixel of the Gb-type G5, a sixth pixel of the Gb-type B6, a seventh pixel of the Gb-type G7, an eight pixel of the Gb-type R8 and a ninth pixel of the Gb-type G9, the Gr-type is comprised of a first pixel of the Gr-type G1, a second pixel of the Gr-type B2, a third pixel of the Gr-type G3, a fourth pixel of the Gr-type R4, a fifth pixel of the Gr-type G5, a sixth pixel of the Gr-type R6, a seventh pixel of the Gr-type G7, an eight pixel of the Gr-type B8 and a ninth pixel of the G9 and the B-type is comprised of a first pixel of the B-type R1, a second pixel of the B-type G2, a third pixel of the B-type R3, a fourth pixel of the B-type G4, a fifth pixel of the B-type B5, a sixth pixel of the B-type G6, a seventh pixel of the B-type R7, an eight pixel of the B-type G8 and a ninth pixel of the B-type R9; defining R′, G′ and B′ that are representative values of R, G and B at the four unit pixel structures with the size of 3×3; deciding one of the four unit pixel structures with the size of 3×3 that the corresponding pixel is falling under for performing the color interpolation with respect to the corresponding pixel; judging whether an edge exists in one of a horizontal direction and a vertical direction at the unit pixel structure as the unit pixel structure is decided; judging whether the edge exists in one of a horizontal top direction and a horizontal bottom direction as the corresponding unit pixel structure has the horizontal edge and judging whether the edge exists in one of a vertical left direction and a vertical right direction as the corresponding unit pixel structure has the vertical direction; judging whether the edge exists in the center as the edge existing one of the horizontal top direction and the horizontal bottom direction and the edge existing one of the vertical left direction and the vertical right direction are checked; and performing the color interpolation according to all the discriminated edges by using the surrounding pixels except for the pixels where the edge exists, wherein the color interpolation is performed by making the values of R, G and B identically as the edge exists in the center, thereby lowering a color and emphasizing a brightness.
The above and other objects and features of the present invention will become better understood with respect to the following description of the preferred embodiments given in conjunction with the accompanying drawings, in which:
Hereinafter, a color interpolation method in accordance with embodiments of the present invention will be described with reference to the accompanying drawings. Furthermore, reference denotations of R, G and B denote a red color, a green color and a blue color, respectively.
R′=CenterC=R5
G′=MissG
B′=CatRB [Mathematics Formula 1]
Herein, C denotes a color, thereby denoting color information, R and B. That is, the representative value of R, i.e., R′, denotes R or B in the center and since the fifth pixel of the R-type R5 is placed in the center, the formula denoting R′=R5 can be formed.
Miss means missing. The fifth pixel of the R-type R5 is placed in the center and the second pixel of the R-type G2, the fourth pixel of the R-type G4, the six pixel of the R-type G6 and the eight pixel of the R-type G8 are placed in the top, the bottom, the right and the left of R5 in the center. However, there is not G in the center of the R-type, thereby denoting this structure with MissG. At this time, methods to obtain the representative value of G, i.e., G′, are various such as taking an average value of the four G pixels, i.e., the second pixel of the R-type G2, the fourth pixel of the R-type G4, the six pixel of the R-type G6 and the eight pixel of the R-type G8 or taking a mean value of the aforementioned four G pixels.
CatRB denotes colors other than the colors corresponding to the R-type or B-type unit pixel structure. Accordingly, CatRB means the representative value of B, i.e., B′ and can be obtained by using the four B pixels in the corners, i.e., the first pixel of the R-type B1, the third pixel of the R-type B3, the seventh pixel of the R-type B7 and the ninth pixel of the R-type B9.
R′=C1atG
G′=CenterG=G5
B′=C2atG [Mathematics Formula 2]
Herein, C includes both R and B. What is placed in the top and the bottom of the fifth pixel of the Gb-type G5 is C1 and what is placed in the left and the right of the fifth pixel of the Gb-type G5 is C2.
Accordingly, the representative value of R, i.e., R′, can be obtained by using the second pixel of the Gb-type R2 and the eight pixel of the Gb-type R8 denoting R that is C1 in the Gb-type unit pixel structure. For instance, the representative value of R, i.e., R′, can be obtained by using an average value of the second pixel of the Gb-type R2 and the eight pixel of the Gb-type R8. The representative value of G, i.e., G′, is G5 that is in the center denoted as CenterG.
The representative value of B, i.e., B′, can be obtained by using B4 and B6 denoting B that is C2 at the Gb-type unit pixel structure. That is, the representative value of B, i.e., B′, can be obtained by using an average value of the fourth pixel of the Gb-type B4 and the sixth pixel of the Gb-type B6.
R′=C2atG
G′=CenterG=G5
B′=C1atG [Mathematics Formula 3]
Herein, C includes both R and B. What is placed in the top and the bottom of the fifth pixel of the Gr-type G5 in the center denoted as CenterG is C1 and what is placed in the left and the right of the fifth pixel of the Gr-type G5 is C2.
Accordingly, the representative value of B, i.e., B′, can be obtained by using the second pixel of the Gr-type B2 and the eight pixel of the Gr-type R8 denoting B that is C1 in the Gr-type unit pixel structure. For instance, the representative value of B, i.e., B′, can be obtained by using an average value of the second pixel of the Gr-type B2 and the eight value of the Gr-type B8. The representative value of G, i.e., G′, is G5 in the center denoted as CenterG.
The representative value of R, i.e., R′, can be obtained by using the fourth pixel of the Gr-type R4 and the sixth pixel of the Gr-type R6 denoting R that is C2 in the Gr-type unit pixel structure. For instance, the representative value of R, i.e., R′, can be obtained by using an average value of the fourth pixel of the Gr-type R4 and the sixth pixel of the Gr-type R6.
R′=CatRB
G′=MissG
B′=CenterC=B5 [Mathematics Formula 8]
Herein, assuming that the representative value of B, i.e., B′, has the fifth pixel of the B type, the formula denoting B′=B5 can be formed.
Miss means missing. The fifth pixel of the B-type B5 is placed in the center and the second pixel of the B-type G2, the fourth pixel of the B-type G4, the sixth pixel of the B-type G6 and the eight pixel of the B-type G8 are in the top, the bottom, the left and the right of the center. However, there is not the G pixel in the center, thereby denoting this structure with MissG. At this time, methods to obtain the representative value of G, i.e., G′, are various such as taking an average value of the four G pixels, i.e., the second pixel of the B-type G2, the fourth pixel of the B-type G4, the sixth pixel of the B-type G6 and the eight pixel of the B-type G8 or a mean value of the aforementioned G pixels.
CatRB denotes a color other than the colors corresponding to the R-type or B-type unit pixel structure. Accordingly, CatRB means the representative value of R, i.e., R′ and can be obtained by using the four surrounding R pixels, i.e., the first pixel of the B-type R1, the third pixel of the B-type R3, the seventh pixel of the B-type R7 and the ninth pixel of the B-type R9.
In accordance with the present invention, the color interpolation method considering the edges corrects the color pixels R and B by considering changed amounts of the R and B pixels, which is. not a simple change, in case of that the G pixel is placed in the center of the unit pixel structure with the size of 3×3 by detecting the edges with use of G in different methods according to each case from a bayer color pattern shown in
Meanwhile, the following table 1 expresses each color C, i.e., R and B colors, in accordance with the above described four unit pixel structures.
Referring to Table 1, a value of the color C on each corresponding horizontal or vertical line can be expressed with using an average value or a representative value of the color C of the existing R or B.
A judgment of the first edge decides whether the edge is horizontal or vertical by using a difference in the G pixels, i.e., the second pixel of the G-type G2, the fourth pixel of the G-type G4, the sixth pixel of the G-type G6 and the eight pixel of the G-type G8. That is, the judgment of the first edge is performed by using a difference between a changed amount of a horizontal direction, i.e., ΔGH, and a changed amount of a vertical direction, i.e., ΔGV.
The following mathematics formula 5 illustrates a method for obtaining the first edge.
That is, if the changed amount of the horizontal direction, i.e., ΔGH, is equal to or greater than the changed amount of the vertical direction, i.e., ΔGV, the edge is placed in the vertical direction as shown in
Since the R-type unit pixel structure is exemplified herein, ΔGH denoting the changed amount of the horizontal direction is |G4−G6| and ΔGV denoting the changed amount of the vertical direction is |G2−G8|.
A method for judging the second edge judges whether the edge is horizontal, i.e., a left-center-right direction or vertical, i.e., a top-center-bottom direction, by using a value of the G pixel placed in the center and the corners.
A position of the second edge depends on the position of the first edge. That is, if the first edge is horizontal, a horizontal top direction and a horizontal bottom direction should be first identified and then, it is necessary to judge whether the first edge is in a horizontal center direction.
In case of that the first edge is vertical, a vertical left direction and a vertical right direction is first identified and then, it is necessary to judge whether the fist edge is in a vertical center direction.
This standard for judgment is attainable by comparing a changed amount of a top-center direction, i.e., ΔGTC, with a changed amount of a bottom-center direction, i.e., ΔGBC, and a changed amount of a left-center direction, i.e., ΔGLC, with a changed amount of a right-center direction, i.e., ΔGRC.
For instance, in the R-type unit pixel structure, the changed amount of the top-center direction, i.e., ΔGTC, the changed amount of the bottom-center direction, i.e., ΔGBC, the changed amount of the left-center direction, i.e., ΔGLC and the changed amount of the right-center direction, i.e., ΔGRC can be obtained as follows.
ΔGTC=|G2−G′|
ΔGBC=|G8−G′|
ΔGLC=|G4−G′|
ΔGRC=|G6−G′| [Mathematics Formula 6]
Herein, the representative value of G, i.e., G′, is the R-type, thereby meaning MissG.
A classification of the horizontal top direction and the horizontal bottom direction is arranged due to the comparison of ΔGTC and ΔGBC and a classification of the vertical left direction and the vertical right direction is arranged due to the comparison of ΔGLC and ΔGRC.
The following mathematics formula 7 illustrates a distribution of the four types of the edge identified by the above comparisons.
Herein, as for Horizontal Top that is the first one, if the first edge is horizontal and the changed amount of the top-center direction is greater than the changed amount of the bottom-center direction, i.e., ΔGTC>ΔGBC, the edge is placed in the horizontal top direction crossing B1, G2 and B3 in
As for Horizontal Bottom that is the second one, if the first edge is horizontal and the changed amount of the bottom-center direction is equal to or less than the changed amount of the top-center direction, i.e., ΔGTC≦ΔGBC, the edge is placed in the horizontal bottom direction crossing B7, G8 and B9 as shown in
As for Vertical Left that is the third one, if the first edge is vertical and the changed amount of the left-center direction is greater than the changed amount of the right-center direction, i.e., ΔGLC>ΔGRC, the edge is placed in the vertical left direction crossing B1, G4 and B7 as shown in
As for Vertical Right that is the fourth one, if the first edge is vertical and the changed amount of the right-center direction is equal to or greater than the changed amount of the left-center direction, i.e., ΔGLC≦ΔGRC, the edge is placed in the vertical right direction crossing B3, G6 and B9 as shown in
Furthermore, whether the edge crosses the center of the unit pixel structure of 3×3 is also judged.
That is, when checking an existence of the first edge, if a difference, i.e., |ΔGH−ΔGV|, between the changed amount of the horizontal direction, i.e., ΔGH and the changed amount of the vertical direction, i.e., ΔGV, is greater than an optional edge threshold, i.e., Eth, the first edge exists in the center of either the horizontal direction or the vertical direction.
When checking an existence of the second edge, if both a difference between the changed amount of the left-center direction and the changed amount of the right-center direction, i.e., |ΔGLC−ΔGRC|, and a difference between the changed amount of the top-center direction and the changed amount of the bottom-center direction, i.e., |ΔGTC−ΔGBC|, are greater than the optional edge threshold, i.e., Eth, the second edge precisely passes from the center to either the horizontal or the vertical direction.
The following mathematics formula 8 illustrates an embodiment checking the existence of the edge in the center when deciding the second edge.
Referring to Mathematics Formula 8, if the difference between the changed amount of the vertical direction, i.e., ΔGV, and the changed amount of the horizontal direction, i.e., ΔGH, is greater than the edge threshold, i.e., Eth, the edge exists in the center. At this time, the difference between the changed amount of the top-center direction, i.e., ΔGTC, and the changed amount of the bottom-center direction, i.e., ΔGTC, is less than the edge threshold, i.e., Eth, the edge precisely exists in the horizontal center direction.
If the difference between the changed amount of the vertical direction, i.e., ΔGV, and the changed amount of the horizontal direction, i.e., ΔGV, is greater than the edge threshold, i.e., Eth, the edge exits in the center. At this time, the difference between the changed amount of the left-center direction, i.e., ΔGLC, and the changed amount of the right-center direction, i.e., ΔGRC, is less than the edge threshold, i.e., Eth, the edge precisely exists in the vertical center.
On the other side, if the difference between the changed amount of the vertical direction, i.e., ΔGV, and the changed amount of the horizontal direction, i.e., ΔGH, is less than the edge threshold, i.e., Eth, the edge does not exist.
Hereinafter, steps of obtaining values of R and B according to locations of each edge except the center will be examined.
Herein, the formula denoting C1atG=CBottom is realized. As for the Gb-type shown in Section (A) of
Meanwhile, in case of the Gr-type shown in Section (B) of
Sections (A) and (B) of
Herein, the formula denoting C2atG=CHorizontal is realized. As for the Gb-type shown in section (A) of
Meanwhile, in case of the Gr-type shown in Section (B) of
Sections (A) and (B) of
Herein, the formula denoting CatRB=CBottom is realized. As for the R-type shown in Section (A) of
Meanwhile, in case of the B-type shown in Section (B) of
Sections (A) and (B) of
Herein, the formula denoting C1atG=CTop is realized. As for the Gb-type as shown in Section (A) of
Meanwhile, in the Gr-type shown in Section (B) of
Sections (A) and (B) of
Herein, the formula denoting C2atG=CHorizontal is realized. The values of R and B are corrected by using the first pixel of the Gb-type G1, the second pixel of the Gb-type R2, the third pixel of the Gb-type G3, the fourth pixel of the Gb-type B4, the fifth pixel of the Gb-type G5 and the sixth pixel of the Gb-type B6 except for the seventh pixel of the Gb-type G7, the eight pixel of the Gb-type R8 and the ninth pixel of the Gb-type G9 where the edge exists in case of the Gb-type as shown in Section (A) of
Meanwhile, as for the Gr-type as shown in Section (B) of
Sections (A) and (B) of
Herein, the formula denoting CatRB=CTop is realized. As for the R-type shown in Section (A) of
Meanwhile, as for the B-type shown in Section (B) of
Sections (A) and (B) of
Herein, the formula denoting C1atG=CVertical is realized. As for the Gb-type shown in Section (A) of
Meanwhile, as for the Gr-type shown in Section (B) of
Sections (A) and (B) of
Herein, the formula denoting C2atG=CHorizontal is realized. As for the Gb-type shown in Section (A) of
Meanwhile, as for the Gr-type shown in Section (B) of
Sections (A) and (B) of
Herein, the formula denoting CatRB=CRight is realized. As for the R-type shown in Section (A) of
Meanwhile, as for the B-type shown in Section (B) of
Sections (A) and (B) of
Herein, the formula denoting C1atG=CVertical is realized. As for the Gb-type shown in Section (A) of
Meanwhile, as for the Gr-type shown in Section (B) of
Sections (A) and (B) of
Herein, the formula C2atG=C1eft is realized. As for the Gb-type shown in Section (A) of
Meanwhile, as for the Gr-type shown in Section (B) of
Sections (A) and (B) of
Herein, the formula denoting CatRB=Cleft is realized. As for the R-type shown in Section (A) of
Meanwhile, as for the B-type shown in Section (B) of
As described above, the color interpolation method in accordance with the present invention corrects the values of R, B and G according to each pattern of the unit pixel structure of 3×3 and lowers color tones according to the values of R, G and B and emphasizes a brightness to emphasize the edge and prevents the incorrect colors in case of that the edge is placed in the horizontal center direction or the vertical center direction.
First, four R-type, B-type, Gb-type and Gr-type unit pixel structures with a size of 3×3 surrounded by eight pixels as shown in
As shown in
As shown in
As shown in
As shown in
Subsequently, in the defined four unit pixel structures with the size of 3×3, each representative value of R, G and B, i.e., R′, G′ and B′, is defined. At this time, each of the representative values of R, G and B, i.e., R′, G′ and B′, of the four unit pixel structures is same as the mathematics formulae 1 to 4 listed above.
If the four unit pixel structures and the representative values of R, G and B, i.e., R′, G′ and B′, of the four unit pixel structures are determined, a color interpolation is employed at step S802.
Meanwhile, in accordance with the present invention, the edges are precisely found and the color interpolation is performed in various methods according to the edges. That is, a unit pixel structure corresponding to a pixel subjected to the color interpolation is selected by understanding that a structure provided with the surrounding eight pixels and the corresponding pixel is belonged to which unit pixel structure among the four unit pixel structures at step S803. Herein, the four unit pixel structures with the size of 3×3 include a mode type of a pixel pattern comprised of R, G and B.
Subsequently, an existence of the edge of either a horizontal direction or a vertical direction of the selected unit pixel structures is checked, thereby judging whether the edge of the corresponding unit pixel structure is placed in the horizontal direction or the vertical direction at step S804. If there is not the edge of the horizontal direction or the vertical direction, a general color interpolation is performed at step S805.
A judgment of the existence of the edge in either the horizontal direction or the vertical direction can be performed by comparing a changed amount of the G value in the horizontal direction, i.e., ΔGH, with a changed amount of the G value in the vertical direction, i.e., ΔGV, thereby deciding whether the edge is placed in the horizontal direction or the vertical direction.
That is, as shown in
ΔGH and ΔGV are compared with each other at step S902. As a result of the comparison, as ΔGV is greater than ΔGH, the edge is judged to be placed in the horizontal direction at step S904 and as ΔGH is greater than ΔGV, the edge is judged to be placed in the vertical direction at step S903.
Next, as shown in
Subsequently, as the edge of the horizontal top direction or the horizontal bottom direction and the edge of the vertical left direction or the vertical right direction are decided, the existence of the edge in the center is judged at step S808.
Next, according to all of the judged edges, the color interpolation is performed by using the surrounding pixels except for the pixels that the edge is placed at step S809. At this time, the color interpolation is performed in a method for making each of the values of R, G and B identically, thereby lowering the color tones and emphasizing the brightness according to the existence of the edge in the center.
Steps S806 to S808 will be examined in more details through
As the edge of the horizontal direction exists at step S904, the existence of the edge of the horizontal top or the horizontal bottom is decided at step S905. At this time, the changed amount of the top-center direction, i.e., ΔGTC, calculated by subtracting the representative value of G, i.e., G′, from the G value of an upper portion of the corresponding unit pixel structure and the changed amount of the bottom-center direction, i.e., ΔGBC, calculated by subtracting the representative value of G, i.e., G′, from the G value of a lower portion of the corresponding unit pixel structure are compared with each other. As a result, as ΔGTC is greater than ΔGBC, the unit pixel structure has the edge of the horizontal direction and as ΔGBC is greater than ΔGTC, the unit pixel structure has the edge of the vertical direction at step S907.
As the edge of the vertical direction exists at step S903, the existence of the edge of the vertical left direction or the vertical right direction is decided at step S906. At this time, the changed amount of the left-center direction, i.e., ΔGLC, calculated by subtracting the representative value of G, i.e., G′, from the G value of a left portion of the corresponding unit pixel structure and the changed amount of the right-center direction, i.e., ΔGRC, calculated by subtracting the representative value of G, i.e., G′, from the G value of a right portion of the corresponding unit pixel structure are compared with each other. As a result, as ΔGLC is greater than ΔGRC, the unit pixel structure has the edge of the vertical left direction and as ΔGRC is greater than ΔGLC, the unit pixel structure has the edge of the vertical right direction at step S908.
The step of judging the existence of the edge in the center is performed by comparing a difference between ΔGH and ΔGV and the edge threshold value, i.e., Eth. This step is performed in the horizontal direction and the vertical direction, respectively at steps S909 and S910.
At this time, the difference between ΔGH and ΔGV, i.e., |ΔGH−ΔGV|, is greater than Eth, the edge is placed in the center at steps S913 and S914. If the difference between ΔGH and ΔGV, i.e., |ΔGH−ΔGV|, is less than Eth, the edge is not placed in the center at steps S911 and S912.
If the difference between the changed amount of the vertical direction, i.e., ΔGV, and the changed amount of the horizontal direction, i.e., ΔGH, is greater than Eth, the edge is placed in the center at step S913. At this time, the difference between the changed amount of the top-center direction, i.e., ΔGTC, and the changed amount of the bottom-center direction, i.e., ΔGBC is compared with the edge threshold value, i.e., Eth at step S915. At this time, if |ΔGTC−ΔGBC| is less than Eth, the edge is placed in the center direction at step S917. If |ΔGTC−ΔGBC| is greater than Eth, the edge is not placed in the horizontal center direction at step S918
If the difference between the changed amount of the vertical direction, i.e., ΔGV, and the changed amount of the horizontal direction, i.e., ΔGH, is greater than Eth, the edge is placed in the center at step S914. At this time, the difference between the changed amount of the left-center direction, i.e., ΔGLC, and the changed amount of the right-center direction, i.e., ΔGRC, is compared with the edge threshold value, i.e., Eth, at step S916. At this time, if |ΔGLC−ΔGRC| is less than Eth, the edge is placed in the center at step S919. If |ΔGLC−ΔGRC| is greater than Eth, the edge is not placed in the vertical center direction at step S920.
Meanwhile, the color interpolation subjected to the four unit pixel structures by considering the edges can be expressed with a mathematics formula 9 as follows.
That is, regardless of the four types of the unit pixel structures, if the edge is placed in the center as denoted with Center_Edge=True, the color interpolation is performed by using the G pixel in the center, i.e., CenterG, having a strong brightness in the representative values of R, G and B, i.e., R′, G′ and B′. Through the steps described above in accordance with each different type, the representative values of R, G and B, i.e., R′, G′, and B′, are determined according to the existence of the edge in the center as shown in Mathematics Formula 9, thereby performing the color interpolation.
The present invention decides a precise position of an edge with use of a G value in a unit pixel structure with a size of 3×3, thereby using different color interpolations according to the position of the edge. Also, the present invention provides an effect of emphasizing an edge by emphasizing a brightness and lowering colors when the edge is placed in the vertical center direction or the horizontal center direction of the unit pixel structure of 3×3 with use of a property that the edge has stronger brightness than the colors and prevents an incorrect color.
The present invention provides effects of emphasizing an edge and preventing incorrect colors by performing different color interpolation methods according to positions of the edge.
The present application contains subject matter related to the Korean patent application No. KR 2004-0027520, filed in the Korean Patent Office on Apr. 21, 2004 the entire contents of which being incorporated herein by reference.
While the present invention has been described with respect to certain preferred embodiments, it will be apparent to those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the following claims.
Claims
1. A color interpolation method, comprising the steps of:
- obtaining a color image using a semiconductor device, the color image comprising unit pixel structures;
- defining four R-type, B-type, Gb-type and Gr-type unit pixel structures with a size of 3×3 comprised of one pixel subjected to the color interpolation and eight pixels surrounding the pixel subjected to the color interpolation, wherein the R-type unit pixel structure includes a first pixel of the R-type B1, a second pixel of the R-type G2, a third pixel of the R-type B3, a fourth pixel of the R-type G4, a fifth pixel of the R-type R5, a sixth pixel of the R-type G6, a seventh pixel of the R-type B7, an eight pixel of the R-type G8 and a ninth pixel of the R-type B9, the Gb-type unit pixel is comprised of a first pixel of the Gb-type G1, a second pixel of the Gb-type R2, a third pixel of the Gb-type G3, a fourth pixel of the Gb-type B4, a fifth pixel of the Gb-type G5, a sixth pixel of the Gb-type B6, a seventh pixel of the Gb-type G7, an eight pixel of the Gb-type R8 and a ninth pixel of the Gb-type G9, the Gr-type is comprised of a first pixel of the Gr-type G1, a second pixel of the Gr-type B2, a third pixel of the Gr-type G3, a fourth pixel of the Gr-type R4, a fifth pixel of the Gr-type G5, a sixth pixel of the Gr-type R6, a seventh pixel of the Gr-type G7, an eight pixel of the Gr-type B8 and a ninth pixel of the G9 and the B-type is comprised of a first pixel of the B-type R1, a second pixel of the B-type G2, a third pixel of the B-type R3, a fourth pixel of the B-type G4, a fifth pixel of the B-type B5, a sixth pixel of the B-type G6, a seventh pixel of the B-type R7, an eight pixel of the B-type G8 and a ninth pixel of the B-type R9;
- defining R′, G′ and B′ that are representative values of R, G and B at the four unit pixel structures with the size of 3×3;
- deciding one of the four unit pixel structures with the size of 3×3 that the corresponding pixel is falling under for performing the color interpolation with respect to the corresponding pixel;
- judging whether an edge exists in one of a horizontal direction and a vertical direction at the unit pixel structure as the unit pixel structure is decided;
- judging whether the edge exists in one of a horizontal top direction and a horizontal bottom direction as the corresponding unit pixel structure has the horizontal edge and judging whether the edge exists in one of a vertical left direction and a vertical right direction as the corresponding unit pixel structure has the vertical direction;
- judging whether the edge exists in the center as the edge existing one of the horizontal top direction and the horizontal bottom direction and the edge existing one of the vertical left direction and the vertical right direction are checked; and
- performing the color interpolation according to all the discriminated edges by using the surrounding pixels except for the pixels where the edge exists, wherein the color interpolation is performed by making the values of R, G and B identically as the edge exists in the center, thereby lowering a color and emphasizing a brightness, wherein
- the step of judging the existence of the edge of one of the horizontal top direction and the horizontal bottom direction is performed by comparing a changed amount of a top-center direction denoted with ΔGTC that is obtained by subtracting the representative green value from the green value in an upper portion of the corresponding pixel structure with the changed amount of a bottom-center direction that is denoted with ΔGBC obtained by subtracting the representative green value from the green value in a lower portion of the corresponding pixel structure, thereby concluding that the unit pixel structure has the edge of the horizontal-top direction as ΔGTC is greater than ΔGBC and concluding that the unit pixel structure has the edge of the horizontal-bottom direction as ΔGBC is greater than ΔGTC.
2. The method of claim 1, wherein the step of judging the existence of the edge of one of the horizontal direction and the vertical direction is performed by comparing a changed amount of the green value of the horizontal direction denoted with ΔGH, with the changed amount of the green value of the vertical direction denoted with ΔGV.
3. The method of claim 2, wherein as ΔGH is greater than ΔGV, the edge is judged as the edge of the vertical direction and as ΔGV is greater than ΔGH, the edge is judged as the edge of the horizontal direction.
4. The method of claim 1, wherein the step of judging the existence of the edge of one of the vertical left direction and the vertical right direction is performed by comparing a changed amount of a left-center direction denoted with ΔGLC that is obtained by subtracting the representative green value from the green value in a right portion of the corresponding pixel structure with the changed amount of a right-center direction denoted with ΔGRC that is obtained by subtracting the representative green value from the green value in a left portion of the corresponding pixel structure, thereby concluding that the unit pixel structure has the edge of the vertical left direction as ΔGLC is greater than ΔGRC and concluding that the unit pixel structure has the edge of the vertical right direction as ΔGRC is greater than ΔGLC.
5. The method of claim 4, wherein the step of judging the existence of the edge in the center concludes that the edge exits in the center as a difference between ΔGH and ΔGV is greater than a threshold value of a first edge.
6. The method of claim 5, wherein the edge exists in the center as a difference between ΔGTC and ΔGBC is less than a second edge.
7. The method of claim 5, wherein the edge exists in the center as a difference between ΔGLC and ΔGRC is less than a second edge.
8. The method of claim 1, wherein the step of judging the existence of the edge in the center concludes that the edge exits in the center as a difference between ΔGH and ΔGV is greater than a threshold value of a first edge.
9. The method of claim 8, wherein the edge exists in the center as a difference between ΔGTC and ΔGBC is less than a second edge.
10. The method of claim 8, wherein the edge exists in the center as a difference between ΔGLC and ΔGRC is less than a second edge.
11. A color interpolation method, comprising:
- obtaining a color image using a semiconductor device, the color image comprising a plurality of unit pixel structures; and
- determining whether an edge exists in one of a horizontal top direction and a horizontal bottom direction and determining whether the edge exists in one of a vertical left direction and a vertical right direction;
- wherein the step of determining the existence of the edge of one of the horizontal top direction and the horizontal bottom direction is performed by comparing a changed amount of a top-center direction denoted with ΔGTC that is obtained by subtracting a representative green value from a green value in an upper portion of a corresponding pixel structure with the changed amount of a bottom center direction that is denoted with ΔGBC obtained by subtracting the representative green value from a green value in a lower portion of the corresponding pixel structure, thereby concluding that the unit pixel structure has the edge of the horizontal top direction if ΔGTC is greater than ΔGBC and concluding that the unit pixel structure has the edge of the horizontal bottom direction as ΔGBC as if greater than ΔGTC.
12. The method of claim 11, further comprising determining whether an edge exists in one of a horizontal direction and a vertical direction at the corresponding unit pixel structure as the corresponding unit pixel structure is decided.
13. The method of claim 12, wherein the determining the existence of the edge of one of the horizontal direction and the vertical direction is performed by comparing a changed amount of the green value of the horizontal direction denoted with ΔGH, with the changed amount of the green value of the vertical direction denoted with ΔGV.
14. The method of claim 13, wherein if ΔG H is greater than ΔGV, the edge is determined as the edge of the vertical direction and if ΔGV is greater than ΔGH, the edge is determined as the edge of the horizontal direction.
15. The method of claim 11, wherein the determining the existence of the edge of one of the vertical left direction and the vertical right direction is performed by comparing a changed amount of a left center direction denoted with ΔGLC that is obtained by subtracting the representative green value from the green value in a right portion of the corresponding pixel structure with the changed amount of a right center direction denoted with ΔGRC that is obtained by subtracting the representative green value from the green value in a left portion of the corresponding pixel structure, thereby concluding that the corresponding unit pixel structure has the edge of the vertical left direction as ΔGLC is greater than ΔGRC and concluding that the corresponding unit pixel structure has the edge of the vertical right direction as ΔGRC is greater than ΔGLC.
16. The method of claim 15, further comprising determining whether the edge exists in the center as the edge existing in one of the horizontal top direction and the horizontal bottom direction and the edge existing in one of the vertical left direction and the vertical right direction are checked.
17. The method of claim 16, wherein the determining the existence of the edge in the center concludes that the edge exists in the center if a difference between ΔGH and ΔGV is greater than a threshold value of a first edge.
18. The method of claim 16, wherein the edge exists in the center if a difference between ΔGTC and ΔGBC is less than a second edge.
19. The method of claim 16, wherein the edge exists in the center if a difference between ΔGLC and ΔGRC is less than a second edge.
20. The method of claim 11, further comprising defining four R-type, B-type, Gb-type and Gr-type unit pixel structures with a size of 3×3 comprised of one pixel subjected to the color interpolation and eight pixels surrounding the pixel subjected to the color interpolation, wherein the R-type unit pixel structure includes a first pixel of the R-type B1, a second pixel of the R-type G2, a third pixel of the R-type B3, a fourth pixel of the R-type G4, a fifth pixel of the R-type R5, a sixth pixel of the R-type G6, a seventh pixel of the R-type B7, an eight pixel of the R-type G8 and a ninth pixel of the R-type B9, the Gb-type unit pixel is comprised of a first pixel of the Gb-type G1, a second pixel of the Gb-type R2, a third pixel of the Gb-type G3, a fourth pixel of the Gb-type B4, a fifth pixel of the Gb-type G5, a sixth pixel of the Gb-type B6, a seventh pixel of the Gb-type G7, an eight pixel of the Gb-type R8 and a ninth pixel of the Gb-type G9, the Gr-type is comprised of a first pixel of the Gr-type G1, a second pixel of the Gr-type B2, a third pixel of the Gr-type G3, a fourth pixel of the Gr-type R4, a fifth pixel of the Gr-type G5, a sixth pixel of the Gr-type R6, a seventh pixel of the Gr-type G7, an eight pixel of the Gr-type B8 and a ninth pixel of the G9 and the B-type is comprised of a first pixel of the B-type R1, a second pixel of the B-type G2, a third pixel of the B-type R3, a fourth pixel of the B-type G4, a fifth pixel of the B-type B5, a sixth pixel of the B-type G6, a seventh pixel of the B-type R7, an eight pixel of the B-type G8 and a ninth pixel of the B-type R9.
21. The method of claim 11, further comprising defining R′, G′ and B′ that are representative values of R, G and B at four unit pixel structures with a size of 3×3.
22. The method of claim 11, further comprising deciding one of four unit pixel structures with a size of 3×3 that the corresponding unit pixel structure is falling under for performing a color interpolation with respect to the corresponding unit pixel structure.
23. The method of claim 11, further comprising performing a color interpolation according to all the discriminated edges by using the surrounding pixels except for the pixels where the edge exists, wherein the color interpolation is performed by making the values of R, G and B identical if the edge exists in the center, thereby lowering a color and emphasizing a brightness.
24. A color interpolation method, comprising:
- obtaining a color image using a semiconductor device, the color image comprising a plurality of unit pixel structures; and
- determining whether an edge exists in one of a horizontal top direction and a horizontal bottom direction and determining whether the edge exists in one of a vertical left direction and a vertical right direction;
- wherein the determining the existence of the edge of one of the vertical left direction and the vertical right direction is performed by comparing a changed amount of a left center direction denoted with ΔGLC that is obtained by subtracting the representative green value from the green value in a right portion of a corresponding pixel structure with the changed amount of a right center direction denoted with ΔGRC that is obtained by subtracting the representative green value from the green value in a left portion of the corresponding pixel structure, thereby concluding that the corresponding unit pixel structure has the edge of the vertical left direction as ΔGLC is greater than ΔGRC and concluding that the corresponding unit pixel structure has the edge of the vertical right direction as ΔGRC is greater than ΔGLC.
25. The method of claim 24, further comprising determining whether the edge exists in the center as the edge existing in one of the horizontal top direction and the horizontal bottom direction and the edge existing in one of the vertical left direction and the vertical right direction are checked.
26. The method of claim 25, wherein the determining the existence of the edge in the center concludes that the edge exists in the center if a difference between ΔGH and ΔGV is greater than a threshold value of a first edge.
27. The method of claim 25, wherein the edge exists in the center if a difference between ΔGTC and ΔGBC is less than a second edge.
28. The method of claim 25, wherein the edge exists in the center if a difference between ΔGLC and ΔGRC is less than a second edge.
4780711 | October 25, 1988 | Doumas |
5404411 | April 4, 1995 | Banton et al. |
6563537 | May 13, 2003 | Kawamura et al. |
6642960 | November 4, 2003 | Kohashi et al. |
6920252 | July 19, 2005 | Rouvellou |
7142239 | November 28, 2006 | Cho |
10108208 | April 1998 | JP |
20010005128 | January 2001 | KR |
- Wu et al., Color Interpolation for Single CCD Color Camera, Mar. 28-30, 2004, 6th IEEE Southwest Symposium on Image Analysis and Interpretation, pp. 90-94.
- Hsia, Fast High-Quality Color-Filter-Array Interpolation Method for Digital Camera Systems, Jan. 2004, SPIE: Journal of Electronic Imaging, vol. 13 (1), pp. 244-250.
- Pei et al., Effective Color Interpolation in CCD Color Filter Arrays Using Signal Correlation, Jun. 2003, IEEE Transactions on Circuits and Systems for Video Technology, vol. 13 Issue:6,pp. 503-513.
- Korean Office Action, dated Jul. 27, 2010, along with English translation.
Type: Grant
Filed: Feb 21, 2011
Date of Patent: Jun 19, 2012
Assignee: Intellectual Ventures II LLC (Wilmington, DE)
Inventor: Byung-Geun Jung (Gyounggi-do)
Primary Examiner: Matthew Bella
Assistant Examiner: Dennis Rosario
Attorney: McAndrews, Held & Malloy, Ltd.
Application Number: 13/031,377
International Classification: G06K 9/48 (20060101);