SPATIO-TEMPORAL COLOR LUMINANCE DITHERING TECHNIQUES
Systems and methods are disclosed to enable the creation and the display of spatio-temporal dithered images. Embodiments include techniques that use color-shifting and luminance. In one embodiment, adjacent pixels are color-shifted with respect to each other and the color values of the adjacent pixels are temporally alternated with color values of pixels in the group. In another embodiment, the luminance of group of adjacent pixels is determined and the luminance of the group is made more homogenous spatially and temporally by distributing color variations over a larger number of pixels so as to reduce the luminance difference between the pixel with the least luminance and the pixel with the greatest luminance. Individual color components (e.g., red, green, blue) may also be separated and used so that the color-shifts associated with each color component may be simultaneously present in different pixels.
Latest Apple Patents:
The present disclosure relates generally to techniques for dithering images using a luminance approach.
This section is intended to introduce the reader to various aspects of art that may be related to various aspects of the present disclosure, which are described and/or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art.
In recent years, electronic display devices have become increasingly popular due, at least in part, to such devices becoming more and more affordable for the average consumer. Further, in addition to a number of electronic display devices currently available for desktop monitors and notebook computers, it is not uncommon for digital display devices to be integrated as part of another electronic device, such as a cellular phone, a tablet computing device, or a portable media player.
Electronic displays are typically configured to output a set number of colors within a color range. In certain cases, a graphical image to be displayed may have a number of colors greater than the number of colors that are capable of being shown by the electronic display. For example, a graphical image may be encoded with a 24-bit color depth (e.g., 8 bits for each of red, green, and blue components of the image), while an electronic display may be configured to provide output images at an 18-bit color depth (e.g., 6 bits for each of red, green, and blue components of the image). Rather than simply discarding least-significant bits, dithering techniques may be used to output a graphical image that appears to be a closer approximation of the original color image. However, the dithering techniques may not approximate the original image as closely as desired.
SUMMARYA summary of certain embodiments disclosed herein is set forth below. It should be understood that these aspects are presented merely to provide the reader with a brief summary of these certain embodiments and that these aspects are not intended to limit the scope of this disclosure. Indeed, this disclosure may encompass a variety of aspects that may not be set forth below.
The present disclosure generally relates to dithering techniques that may be used to display color images on an electronic display. The electronic display may include one or more electronic components, including a power source, pixelation hardware (e.g., light emitting diodes, liquid crystal display), and circuitry for receiving signals representative of image data to be displayed. In certain embodiments, a processor may be internal to the display while in other embodiments the processor may be external to the display and included as part of an electronic device, such as a computer workstation or a cell phone.
The processor may use dithering techniques, including spatial and temporal dithering techniques disclosed herein, to output color images on the electronic display. In one embodiment, adjacent pixels are color-shifted with respect to each other and the color values of certain pixels are temporally alternated with color values of other pixels in the group. In another embodiment, the luminance of a group of adjacent pixels is determined and the luminance of the group is made more homogenous spatially and temporally by distributing color variations over a larger number of pixels so as to reduce the luminance difference between the pixel with the least luminance and the pixel with the greatest luminance. Individual color components (e.g., red, green, blue) may also be separated and used so that the color-shifts associated with each color component may be simultaneously present in different pixels.
Various refinements of the features noted above may exist in relation to various aspects of the present disclosure. Further features may also be incorporated in these various aspects as well. These refinements and additional features may exist individually or in any combination. For instance, various features discussed below in relation to one or more of the illustrated embodiments may be incorporated into any of the above-described aspects of the present disclosure alone or in any combination. Again, the brief summary presented above is intended only to familiarize the reader with certain aspects and contexts of embodiments of the present disclosure without limitation to the claimed subject matter.
Various aspects of this disclosure may be better understood upon reading the following detailed description and upon reference to the drawings in which:
One or more specific embodiments will be described below. In an effort to provide a concise description of these embodiments, not all features of an actual implementation are described in the specification. It should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another. Moreover, it should be appreciated that such a development effort might be complex and time consuming, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure.
As will be discussed below, the present disclosure relates generally to techniques for processing and displaying image data on an electronic display device. In particular, certain aspects of the present disclosure may relate to techniques for processing images using temporal and spatial dithering techniques. Further, it should be understood that the presently disclosed techniques may be applied to both still images and moving images (e.g., video), and may be utilized in any suitable type of electronic display, such as a cell phone, a desktop computer monitor, a tablet computing device, an e-book reader, a television, and so forth.
With the foregoing in mind, it may be beneficial to first discuss embodiments of certain display systems that may incorporate the dithering techniques as described herein. With this in mind, and turning now to the figures,
Regardless of its form (e.g., portable or non-portable), it should be understood that the electronic device 10 may provide for the processing of image data using one or more of the image processing techniques briefly discussed above, which may include spatial and/or temporal dithering techniques, among others. In some embodiments, the electronic device 10 may apply such image processing techniques to image data stored in a memory of the electronic device 10. Embodiments showing both portable and non-portable embodiments of the electronic device 10 will be further discussed below with respect to
As shown in
It should be understood that the system block diagram of the device 10 shown in
The input structures 14 may provide user input or feedback to the processor(s) 16. For instance, input structures 14 may be configured to control one or more functions of electronic device 10, such as applications running on electronic device 10. In addition to processing various input signals received via the input structure(s) 14, the processor(s) 16 may control the general operation of the device 10. For instance, the processor(s) 16 may provide the processing capability to execute an operating system, programs, user and application interfaces, and any other functions of the electronic device 10.
The processor(s) 16 may include one or more microprocessors, such as one or more “general-purpose” microprocessors, one or more special-purpose microprocessors and/or application-specific microprocessors (ASICs), or a combination of such processing components. For example, the processor(s) 16 may include one or more reduced instruction set (e.g., RISC) processors, as well as graphics processors (GPU), video (GPU), video processors, audio processors and/or related chip sets. As will be appreciated, the processor(s) 16 may be coupled to one or more data buses for transferring data and instructions between various components of the device 10. In certain embodiments, the processor(s) 16 may provide the processing capability to execute source code embodiments capable of employing the dithering techniques described herein.
The instructions or data to be processed by the processor(s) 16 may be stored in a computer-readable medium, such as a memory device 18. The memory device 18 may be provided as a volatile memory, such as random access memory (RAM) or as a non-volatile memory, such as read-only memory (ROM), or as a combination of one or more RAM and ROM devices. In addition, the memory 18 may be used for buffering or caching during operation of the electronic device 10. For instance, in one embodiment, the memory 18 includes one or more frame buffers for buffering video data as it is being output to the display 28.
In addition to the memory device 18, the electronic device 10 may further include a non-volatile storage 20 for persistent storage of data and/or instructions. The non-volatile storage 20 may include flash memory, a hard drive, or any other optical, magnetic, and/or solid-state storage media, or some combination thereof. In accordance with aspects of the present disclosure, image processing data stored in the non-volatile storage 20 and/or the memory device 18 may be processed by the image processing circuitry 32 prior to being output on a display.
The embodiment illustrated in
The power source 26 of the device 10 may include the capability to power the device 10 in both non-portable and portable settings. The display 28 may be used to display various images generated by device 10, such as a GUI for an operating system, or image data (including still images and video data) processed by the image processing circuitry 32, as will be discussed further below. As mentioned above, the image data may include image data acquired using the imaging device 30 or image data retrieved from the memory 18 and/or non-volatile storage 20. The display 28 may be any suitable type of display, such as a liquid crystal display (LCD), plasma display, or an organic light emitting diode (OLED) display, for example. Additionally, as discussed above, the display 28 may be provided in conjunction with the above-discussed touch-sensitive mechanism (e.g., a touch screen) that may function as part of a control interface for the electronic device 10. The illustrated imaging device(s) 30 may be provided as a digital camera configured to acquire both still images and moving images (e.g., video).
The image processing circuitry 32 may provide for various image processing steps, such as spatial dithering, temporal dithering, pixel color-space conversion, luminance determination, luminance optimization, image scaling operations, and so forth. In some embodiments, the image processing circuitry 32 may include various subcomponents and/or discrete units of logic that collectively form an image processing “pipeline” for performing each of the various image processing steps. These subcomponents may be implemented using hardware (e.g., digital signal processors or ASICs) or software, or via a combination of hardware and software components. The various image processing operations that may be provided by the image processing circuitry 32 and, particularly those processing operations relating to spatial dithering, temporal dithering, pixel color-space conversion, luminance determination, and luminance optimization, will be discussed in greater detail below.
Referring again to the electronic device 10,
As further shown, the display 28 may be configured to generate various images that may be viewed by a user, such as a dithered image 42. The dithered image 42 may have been generated by using, for example, the spatial and temporal dithering techniques described in more detail below. During operation of the computer 34, the display 28 may display a graphical user interface (“GUI”) 44 that allows the user to interact with an operating system and/or application running on the computer 34.
Turning to
Having provided some context with regard to various forms that the electronic device 10 may take and now turning to
A pixel matrix 50 is depicted in greater detail and includes four adjacent pixels 52, 54, 56, and 58. In the depicted embodiment, each pixel of the display device 28 may include three sub-pixels capable of displaying a red (R), a green (G), and a blue (B) color. The human eye is capable of perceiving the particular RGB color combination displayed by the pixel and translating the combination into a specific color. A number of colors may be displayed by each individual pixel by varying the individual RGB intensity levels of the pixel. For example, a pixel having a level of 50% R, 50% G, and 50% B may be perceived as the color gray, while a pixel having a level of 100% R, 100% G, and 0% B may be perceived as the color yellow.
The number of colors that a pixel is capable of displaying is dependent on the hardware capabilities of the display 28. For example, a display 28 with a 6-bit color depth for each sub-pixel is capable of producing 64 (26) intensity levels for each of the R, G, and B color components. The number of bits per sub-pixel, e.g. 6 bits, is referred to as the pixel depth. At a pixel depth of 6 bits, 262,144 (26×26×26) color combinations are possible, while at pixel depth of 8 bits, 16,777,216 (28×28×28) color combinations are possible. Although the visual quality of images produced by an 8-bit pixel depth display 28 may be superior to the visual quality of images produced by a display 28 using 6-bit pixel depth, the cost of the 8-bit display 28 is also higher. Accordingly, it would be beneficial to apply imaging processing techniques, such as the techniques described herein, that are capable displaying a source image with improved visual reproduction even when utilizing lower pixel depth displays 28. Further, a source image may contain more colors than those supported by the display 28, even displays 28 having higher pixel depths. Accordingly, it would also be beneficial to apply imaging processing techniques that are capable of improved visual representation of any number of colors. Indeed, the image processing techniques described herein, such as those described in more detail with respect to
Turning to
The source image 62 may first undergo color decomposition (block 64). The color decomposition (block 64) is capable of decomposing the color of each pixel of the source image 62 into the three RGB color levels. That is, the RGB intensity levels for each pixel may be determined by the color decomposition (block 64). Such a decomposition may be referred to as a three-channel decomposition, because the colors may be decomposed into a red channel, a green channel, and a blue channel, for example.
In the depicted embodiment, the source image 62 may also undergo a luminance analysis (block 66). A luminance is related to the perceived brightness of an image or an image component (such as a pixel) to the human eye. Further, humans typically perceive colors as having different luminance even if each color has equal radiance. For example, at equal radiances, humans typically perceive the color green as having higher luminance than the color red. Additionally, the color red is perceived as having a higher luminance than the color blue. In one example, a luminance formula Y may be arrived at by incorporating observations based on the perception of luminance by humans as defined below.
Y=0.30 R+0.60 G+0.10 B
Indeed, the luminance equation Y above is an additive formula based on 30% red, 60% green, and 10% blue chromaticities (e.g., color values). The luminance formula Y can thus use the RGB color levels of a pixel to determine an approximate human perception of the luminance of the pixel. It is to be understood that because of the variability of human perception, among other factors, the values used in the luminance equation are approximate. Indeed, in other embodiments, the percentage values for R, G, and B may be different. For example, in another embodiment, the values may be approximately 29.9% red, 58.7% green, and 11.4% blue.
The luminance value of each pixel may then be utilized for spatial dithering (block 68). In spatial dithering, the image may be manipulated so as to increase the “noise” of the image, decrease color banding, and make sharp edges of the image less detectable. Spatial dithering may therefore improve the image perception and quality. In certain spatial dithering embodiments, the pixels from the source image 62 may first be converted to a lower pixel depth, for example, through a most significant bit (MSB) and a least significant bit (LSB) process, as described in more detail below with respect to
Multiple dither patterns 70 may also be used during spatial dithering so as to enable a displayed image 74 to more closely approximate the source image 62. In one embodiment, two sets of dither patterns 70 and 70′ may be stored in memory. In this embodiment, the set of dither patterns 70 may be used with a color channel such as green, and the set 70′ may be used with color channels red and blue. In another embodiment, the dither patterns 70 (and 70′) may be dynamically calculated based on the luminance analysis (block 66) and not stored in memory. In yet another embodiment, the dither patterns 70 corresponding to a single color channel, such as green, may be stored in memory. In this embodiment the dither patterns 70′ may be derived based on the stored dither patterns 70. The dither patterns 70 and 70′ are described in more detail below with respect to
Additionally, the ISP logic 60 may be capable of utilizing temporal dithering (block 72). In temporal dithering (block 72), the colors and/or luminosity of pixels may be alternated frame-by-frame so as to improve the perceived image quality of the displayed image 74. That is, a first frame of the processed image may be presented at time T0, followed by a second frame of the processed image which may be presented at time T1. The second frame may have color and/or luminance variations from the first frame. Likewise, a third frame of the processed image may be presented at time T2 having color/and or luminance that differ from the second frame. In certain embodiments, additional frames may then be presented also having color/and or luminance values that differ form the third frame. Additionally, the temporal dithering (block 72) may iteratively loop through the frame presentations. That is, after presenting a certain n-th frame at time Tn, the first frame may then be presented again, followed by the second frame, and so on, up to the n-th frame and then returning to the first frame.
Humans may perceive multiple frames presented sequentially one after the other as a single image. Indeed, in some embodiments, 60, 120, 240, or more frames per second (FPS) may be presented sequentially. By alternating the color and/or the luminance of each frame and by presenting the frames sequentially, it is possible to enable a single perceived image that is more natural and pleasing to the human eye. Accordingly, the dithering techniques described herein, such as the MSB-LSB based technique described in more detail with respect to
Each R, G, B color channel matrix 80 may then be subdivided (block 82) into multiple source image groups (e.g., matrices) 84 corresponding to different areas of the image. In one example, a group 84 is sized as 4×4 pixel group having a total of 16 pixels. Accordingly, the subdivision (block 82) of the source image 62 may be accomplished by selecting multiple 4×4 adjacent pixel groups 84 so as to partition the entire image into the 4×4 pixel groups 84. Each 4×4 pixel group 84 may then be used to create (block 86) a corresponding LSB group 88 and MSB group 90, as shown in more detail with respect to
To arrive at the MSB and LSB values, the pixel's color value may be provided in, or converted to, a binary value. The binary value may then be divided into two binary values, the LSB value and the MSB value. The most significant bits equal to the pixel depth (e.g., 6 bits) of the display device 28 are selected as the MSB value and the remainder bits are selected as the LSB value. As an example, suppose that the original image is stored at a 9-bit pixel depth and the display 28 is a 6-bit pixel depth display. If the original pixel color channel has a decimal color value of forty-four, the resulting binary number is “000101100”. The six most significant bits are “000101”, which corresponds to the decimal number five. Accordingly the MSB value becomes equal to the number five. The remainder three binary bits of “100” correspond to the decimal number four. Accordingly, the LSB value becomes equal to four. A dither pattern 70 may then be selected and used to create a modification matrix 94 (block 92).
In one embodiment, one of the dither patterns 70 may be selected based on the LSB value or magnitude and used to create (block 92) the modification matrix 94. Indeed, the values of the LSB group 88 may be used to define the modification matrix's 94 values, resulting in the modification matrix 94 having ones and zeros. Examples of the use of the LSB group 88 to create the modification matrix 94 based on the dither patterns 70 are described in more detail with respect to
The modification matrix 94 may then be mathematically added (i.e., through matrix addition) to the MSB group 90 (block 96) to create a new lower pixel depth (e.g., 6 bit) MSB matrix 98. The resulting lower pixel depth MSB matrix 98 is thus capable of being displayed by the display 28. Indeed, multiple new MSB matrices 98 98 may be derived corresponding to all the pixels groups of the source image 94. The multiple new MSB matrices 98 may then be displayed as the displayed image 74.
The most significant bits (e.g., six bits) of the source image values A, B, C, and D, may then be used to derive the values M1=“010001”, M2=“110011”, M3=“000101”, M4=“111100”, of a first row of the MSB group 90. The number of most significant bits may be based on the pixel depth of the display 28. For instance, six bits may be selected as the most significant bits if the display 28 is capable of a 6-bit pixel depth. Should the display be capable of, for example, only a 4-bit pixel depth, then the first four bits of the source image values could be used. For the 6-bit pixel depth example, the decimal values for the 6-bit binary values are M1=“17”, M2=“51”, M3=“5”, M4=“60”.
The remainder three bits of the of the source image values A, B, C, D, may then be used to derive the binary values L1=“101”, L2=“011”, L3=“100”, L=“000”, of a first row of the LSB group 88. The decimal values equivalent to the 3-bit binary values are L1=“5”, L2=“3”, L3=“4”, L4=“0”. One of the dither patterns 70 (e.g., individual dither patterns 102, 104, 106, 108, 110, 112, 114, and 116) may then be selected and used to create the modification group 94 based on the LSB group 88. In the depicted example, the dither pattern 110 is selected. In certain embodiments, a dither pattern, such as dither pattern 110, is selected based on the LSB group 88 as described in more detail below. Once selected, the dither pattern 110 and the LSB 88 may be used to create the modification matrix 94.
In one embodiment, the value (i.e., magnitude) of each cell of the LSB 88, such as cells L1, L2, L3, and L4, is used to select one of the dither patterns 70. Because the values of the 3-bit LSB cells may vary from the decimal value “0” to the decimal “7”, there are eight possible values. Accordingly, eight dither patterns 70 are provided when using the 3-bit LSB 88. It is to be understood that when the LSB 88 stores more (or less) binary bits, then more (or less) dither patterns 70 may be provided. For example, when using a 2-bit LSB 88, there may be four (i.e., 22) dither patterns 70 provided. Likewise, when using a 4-bit LSB 88, sixteen (i.e., 24) dither patterns 70 may be provided.
The magnitude or value of the 3-bit binary number stored in each cell of the LSB 88 may then be used to select one of the eight illustrated dither patterns 70. For example, cell L4 of the LSB 88 may have the value “0”, which corresponds to the first of eight possible values “0” to “7”. Accordingly, the first dither pattern 102 of the eight dither patterns 70 may be selected. Similarly, the cell L3 contains the value “4”, which corresponds to the fifth of eight possible values “0” to “7”. Accordingly, the fifth dither pattern 110 may be selected. Likewise, the cell L2 contains the value “3” which corresponds to the fourth dither pattern 108. L1 contains the value “5”, which in turn corresponds to the sixth dither pattern 112. In this way, the first row of the LSB group 88 containing the cells L1, L2, L3, and L4, may map to one of the dither patterns 70. All other cells of the LSB group 88 may be mapped to one of the dither patterns 70 in a similar manner.
As mentioned above with respect to
Indeed, all dither patterns 104, 106, 108, 110, 112, 114, and 116 may be phase-shifted into dither patterns 104′, 106′, 108′, 110′, 112′, 114′, and 116′ so as more homogeneously distribute the luminance. As mentioned above, the phase-shifting may be accomplished by shifting the “1” values to counterbalance the effect on luminance of the previous position of the “1” values. For example, dither pattern 108′ may be arrived at having a first row “0 1 0 0”, a second row “1 0 0 1”, a third row “0 0 0 1” and a fourth row “0 1 1 0” by counterbalancing the effect of the “1” values of dither pattern 108. In yet another example, a dither pattern 116′ may be arrived at having a first row “1 0 1 1”, a second row “1 1 1 1”, a third row “1 1 1 0”, and a fourth row “1 1 1 1” by counterbalancing the effect of the “1” values of the dither pattern 116.
Once one of the dither patterns 70 (or 70′) is selected, then the LSB group 88 may again be used to select one of the cells in each of the selected dither patterns 70 (or 70′). To make such a cell selection, the position of each cell in the LSB group 88 is used to “point” to the same position in the selected dither pattern 70 (or 70′). In the depicted example, L3 may first be used to select the dither pattern 110 and then L3's cell position may be used to select one of the cells of the dither pattern 110. L3 is positioned in the first row, third column cell. Accordingly, the cell in the first row, third column of the dither pattern 110 may then be selected. The value in this first row, third column cell (i.e., “1”) of the dither pattern 110, may then be used to fill the cell at the same position (i.e., first row, third column) in the modification matrix 94. Likewise, the cells L1, L2, and L4 may be used. For example, L1 is in the first row, first column of the LSB group 88, so the first row and first column value of the dither pattern 112 (i.e., “1”) is copied to the first row and first column cell of the modification matrix 94. Similarly, L2 is in the first first row, second column of the LSB group 88, so the first row and second column value of the dither pattern 108 (i.e., “0”) is copied to the first, row second column cell of the modification matrix 94. In a similar way, the value (i.e., “0”) of the cell in the first row, fourth column of the dither pattern 102 is copied into the first row, fourth column cell of the modification matrix 94. By using this methodology, all of the cells of the modification matrix 94 may be derived as having a zero or a one.
The MSB block 90 may then be added to the modification matrix 94 by using, for example, matrix addition. That is, every cell in the MSB block 90 may be added to the corresponding cell in the modification matrix 94. The result of the addition operation is a new MSB block 98. Using the numbers used in the depicted example, the decimal values for the first row of the new MSB block 98 are A1=“17”+“1”=“18”, B1=“51”+“0”=“51”, C1=“5”+“1”=“6”, and D1=“60”+“0”=“60”. The remaining rows of the new MSB block 98 may then be similarly computed based on the values for the corresponding rows of the source image block 84. As mentioned above, the new MSB block 98 may include color values at a lower pixel depth than the source image block 84 suitable for display by the display 28. Indeed, the dithering techniques disclosed herein allow for the creation of multiple new MSB blocks 98 suitable for displaying the higher pixel depth (e.g., 9-bit) source image 62 at a lower pixel depth (e.g., 6-bit).
Turning to
The second row of the depicted example corresponds to time T1. As illustrated, the bits of the dither patterns at time T1 have been temporally shifted from their positions at time T0. In certain embodiments, the shift of the bits is accomplished by a clockwise rotation of the bits. In one example, each of the dither patterns may be divided into a top left quadrant 118, a top right quadrant 120, a bottom right quadrant 122, and a bottom left quadrant 124, each quadrant having four bits. In this example, each of the quadrants may have the bits rotated in a clockwise direction as depicted in
A third row in
Each R, G, B color channel matrix 80 may then be used (block 82) to create multiple source image groups (e.g., matrices) 84 corresponding to different areas of the image, or to different pixels of the image, with each cell in the group having an red, green, and blue color component. In certain embodiments of the source group 84, the group is sized as 2×2 pixel group having a total of 4 pixels. In one embodiment, the values for each of the pixels in the source group 84 may be derived from a single pixel of the source image. That is, the RGB values of a source image pixel may be copied into the 2×2 pixel group 84. In other embodiments, multiple 2×2 adjacent pixels of the source image may be copied into the 2×2 pixel group 84. Accordingly, the entire image may be divided either pixel by pixel or by selecting adjacent pixels. It is to be understood that, in other embodiments, other sizes of source image groups 84 may be used, for example, 4×4, 6×6, 8×8, and so forth.
A luminance value of each cell in the source image group 84 may then be determined (block 128), for example, through the use of the luminance formula Y described above. A matrix of source image RGB values may then be derived based on the color values of each cell in the source image group 84. The source image RGB matrix may include four cells where each cell includes three sub-cells, each sub-cell storing a luminance for each RGB channel. An example 4×4 source image RGB matrix is shown in
In one example, the color shifting (block 134) reduces the overall luminance amplitude by dividing the luminance of each source image RGB channel into four lower pixel depth values. That is, a higher pixel depth value, such as an 8-bit value may be divided into four lower pixel depth values, such as four 6-bit values. The overall luminance difference of the lower pixel are reduced by reapportioning the red, green, and color values of the four lower pixel depth RGB values so as to result in a reduced-amplitude luminance matrix 136 that has more homogenous luminance values. That is, the RGB color components of the cells in the reduced-amplitude luminance matrix are distributed spatially (e.g., moved from one cell to another cell) so as to reduce the luminance amplitude (e.g., luminance difference of the highest luminance versus the lowest luminance) of the reduced-amplitude luminance matrix 136. An example of such a spatial distribution of values is described in more detail with respect to
Turning to
A table 142 depicts example decimal values for Rs, Gs, and Bs (e.g., “229”, “131”, and “190”). Because the values in the source image RGB matrix 130 are stored at a higher pixel depth (e.g., 8-bits), the values may need to be transformed to lower bit values (e.g., 6-bit pixel depth values) in order to allow display by the display 28. In one embodiment, each of the Rs, Gs, and Bs values (e.g., 8-bit values) may first be converted into lower pixel depth integer values (e.g., 6-bit values). One such conversion from an 8-bit value into a 6-bit value may include dividing the original source value by four (i.e., divide by 22). In another conversion, the first six bits of the 8-bit values may be used to arrive at the 6-bit value. In the depicted embodiment, the resulting decimal values for the conversion are depicted as R1, R2, R3, and R4.
It is to be noted that the conversion from a higher pixel depth value into a lower pixel depth may result in the numbers having fractional components. For example, for the Rs value of “229”, a division by four results in the number “57.25” having the fractional component “0.25”. Because the hardware may not be suitable for displaying fractional color levels, the fractional component is usually not used. In one embodiment, the original source value “229” is approximated by using four lower-pixel depth values R1, R2, R3, and R4 set to “57”, “57”, “57”, and “58”, respectively. Likewise, the Gs value of “131” may result in G1, G2, G3, and G4 set to “32”, “33”, “33”, and “33”, respectively. Similarly, the Bs value of “190” may result in B1, B2, B3, and B4 set to values “47”, “47”, “48”, and “48”, respectively. These four sets of values representing the lower pixel-depth bit (e.g., 6-bit) values may then be color-shifted, that is, distributed spatially so as to reduce the luminance amplitude of the matrix 136.
In order to reduce the luminance amplitude of the matrix 136, a luminance difference may be first calculated by finding a highest luminance value and a lowest luminance value based on all the RGB values of the luminance matrix 136, by using, for example, the luminance equation Y. In our example, the highest luminance value could be obtained with a cell having the values R=“58”, G=“33”, and B=“48”. The lowest luminance value could be obtained with a cell having the values R=“57”, G=“32”, and B=“47”. But in some embodiments, the luminance difference may be adjusted by increasing or decreasing the values for red, green, and blue to reduce luminance variation within the matrix 136. Increasing or decreasing the green value (while keeping the other colors the same) has the greatest perceived effect on luminance, based on the perceived luminance equation Y described above. Increasing or decreasing the color red (while keeping the other colors the same) has second greatest effect on luminance, and increasing or decreasing the color blue (while keeping the other colors the same) has the least perceived effect on luminance.
In certain embodiments, an algorithm, such a value optimization algorithm (e.g., greedy algorithm), may be used to assign the sets of values into specific cells (e.g., spatially distribute the values) so as to minimize the luminance difference of the reduced-amplitude luminance matrix 136 by using the luminance Y equation to more evenly distribute the integer values. For example, the algorithm may first assign the four R1, R2, R3, and R4 values by increasing order, random order, or any other ordering. A table 144 of display luminance values depicts the four R1, R2, R3, and R4 values assigned by increasing order (e.g., R1=“57”, R2=“57”, R3=“57”, and R4=“58”). The four green values may then be assigned to minimize the red-green luminance difference between the four cells. For example, if a cell has a high red value compared to one or more other cells, then the cell may be used to store a low green value (compared to one or more other cells). In the depicted example, the highest red value is stored in R4, therefore, G4 may get the lowest green value.
The blue color values may then be similarly assigned so that the resulting luminance difference of the reduced-amplitude luminance matrix 136 is lowered or minimized. For example, the lowest blue value of “47” may be assigned to the cells having red=“57” and green=“33” (e.g., second and third cells) of the matrix 136 to counterbalance an assignment of a blue value of “48” to the fourth cell of the matrix 136. A high perceived luminance value YH=“41.7” for the reassigned matrix 136 may then found in the first cell having the values R1=“57”, G1=“33”, and B1=“48”. A low luminance value YL=“41.4” for the matrix 136 may be found in the fourth cell having the values R4=“58”, G4=“32”, and B4=“48”, with the luminance values of the second and third cells falling between YL and YH. It is to be understood that any algorithm, including brute force search algorithms, suitable for spatially redistributing the sets of values (e.g., R1, R2, R3, R4, G1, G2, G3, G4, B1, B2, B3, and B4) may be used to derive the reduced-amplitude luminance matrix 136.
In one embodiment, the values of the reduced-amplitude luminance matrix 136 may then be used to display a more improved and visually pleasing displayed image 74. In another embodiment, such as the embodiment described in more detail below with respect to
Turning to
The reduced-amplitude luminance matrix 136 may then be arrived at by color-shifting or spatially distributing the 8-bit values so as to reduce the overall perceived luminance difference of the reduced-amplitude luminance matrix 136. In our example, the highest luminance value could be obtained with a cell having the values R=“234”, G=“152”, and B=“92”. The lowest luminance value could be obtained with a cell having the values R=“233”, G=“151”, and B=“91”. The lower pixel depth values may then be re-assigned as depicted in table 144 so as to reduce the luminance difference between the cell having the highest luminance and the cell having the lowest luminance. In this example, the four R1, R2, R3, and R4 values are first assigned by decreasing order (e.g., R1=“234”, R2=“234”, R3=“234”, and R4=“233”). The four green values may then be assigned to minimize the red-green luminance difference between the four cells. For example, the luminance difference of the cells may be minimized by balancing the assignment of the high red value in one cell with the assignment of the high green value in another cell so as to more evenly spread the high value assignments. In the depicted example, the highest red values are stored in R1, R2, and R3, therefore, G1 and G2 may get the two lowest green values (e.g., “151”, “151”). The blue color values may then be similarly assigned so that the resulting luminance difference of the reduced-amplitude luminance matrix 136 is lowered or minimized. In this example, the blue value “91” may be assigned to the two cells of the matrix 136 containing the highest green values (e.g., third and fourth cells) to counterbalance the assignment of the blue value “92” to the first two cells of the matrix 136. By using the techniques described herein, the resulting displayed image 74 may be perceived as having having an improved visual quality.
The specific embodiments described above have been shown by way of example, and it should be understood that these embodiments may be susceptible to various modifications and alternative forms. It should be further understood that the claims are not intended to be limited to the particular forms disclosed, but rather to cover all modifications, equivalents, and alternatives falling within the spirit and scope of this disclosure.
Claims
1. A method comprising:
- decomposing a source image comprising a plurality of pixels into Red (R), Green (G), and Blue (B) color components corresponding to each pixel;
- creating a red source image group, a green source image group, and a blue source image group by assigning the respective RGB color components of each pixel to the red, green and blue source image groups;
- deriving a matrix based on the red, green, and blue source image groups;
- determining a luminance difference between cells in the matrix, each cell including a red component, a green component, and a blue component, wherein determining the luminance difference includes determining a luminance amplitude of the matrix, the luminance amplitude of the matrix equal to the difference in luminance between a cell of the matrix having the highest luminance and a cell of the matrix having the lowest luminance based on their respective red, green, and blue components; and
- reducing the luminance amplitude of the matrix.
2. The method of claim 1, comprising temporally dithering at least one of the red, green, or blue components of two or more cells in the matrix.
3. The method of claim 2, wherein the temporally dithering the at least one of the red, green, or blue components of the two or more cells in the matrix comprises a clockwise or counterclockwise temporal dithering of the cells in the matrix.
4. The method of claim 1, wherein the deriving the matrix comprises deriving a lower pixel depth value from a higher pixel depth value of the red, green, or blue source image groups.
5. The method of claim 4, wherein the deriving the matrix includes spatially dithering at least one of the RGB color components.
6. The method of claim 1, wherein the reducing the luminance amplitude of the matrix comprises a color-shifting of at least one of the red, green, or blue components by distributing the color components spatially across one or more cells in the matrix such that the luminance amplitude between the most and least luminescent cells of the matrix following the color-shifting is less than the luminance amplitude between the most and least luminescent cells of the matrix before the color-shifting.
7. A non-transitory computer-readable medium comprising code adapted to:
- decompose a source image comprising pixels into Red (R), Green (G), and Blue (B) color components corresponding to each pixel;
- create an individual color component source image group by assigning the R, G, or B color component of each pixel to one or more adjacent cells of the source image group;
- create a first most significant bit (MSB) matrix by using the most significant bits of each cell of the source image group;
- create a least significant bit (LSB) matrix by using the least significant bits of each cell of the source image group;
- select a dither pattern from a plurality of dither patterns by using the LSB matrix;
- create a modification matrix by using the selected dither pattern and the LSB matrix;
- add the modification matrix to the first MSB matrix to create a second MSB matrix; and
- provide visual output to a display based on the second MSB matrix.
8. The non-transitory computer-readable medium of claim 7, wherein the code adapted to select the dither pattern by using the LSB matrix comprises code adapted to use a magnitude of a value of a cell in the LSB matrix to select the dither pattern.
9. The non-transitory computer-readable medium of claim 7, wherein the code adapted to select the dither pattern by using the LSB matrix comprises code adapted to temporally dither the selected dither pattern.
10. The non-transitory computer-readable medium of claim 7, wherein the code adapted to create the modification matrix by using the selected dither pattern and the LSB matrix comprises code adapted to use a row and column position of a first cell of the LSB matrix to select a value in a second cell of the selected dither pattern, the second cell of the selected dither pattern having the row and column position of the first cell.
11. The non-transitory computer-readable medium of claim 7, comprising code adapted to temporally dither the second MSB matrix.
12. The non-transitory computer-readable medium of claim 11, wherein the code adapted to temporally dither the second MSB matrix comprises code adapted to perform a clockwise or counterclockwise temporal dithering of the second MSB matrix.
13. An electronic device comprising:
- a display comprising a plurality of pixels; and
- a processor configured to transmit signals representative of image data to the plurality of pixels of the display, wherein the processor is adapted to decompose an area of a source image into Red (R), Green (G), and Blue (B) color components; create a source image group by assigning the RGB color components of the area to one or more adjacent cells of the source image group; create a most significant bit (MSB) matrix by using the source image group; derive a matrix based on the RGB colors of each cell of the MSB matrix; determine a luminance difference of the cells in the matrix; and reduce the luminance amplitude of the matrix.
14. The electronic device of claim 13, wherein the area of the source image comprises a single pixel.
15. The electronic device of claim 13, wherein the processor is adapted to create the MSB matrix using only the most significant bits of the source image group.
16. A method comprising:
- decomposing a source image comprising pixels into Red (R), Green (G), and Blue (B) color components corresponding to each pixel;
- creating an individual color source image group by assigning the R, G, or B color component of each pixel to one or more adjacent cells of the source image group;
- creating a first most significant bit (MSB) group by using the source image group;
- creating a least significant bit (LSB) group by using the source image group;
- selecting a spatial dither pattern from a plurality of spatial dither patterns by using the LSB group;
- creating a modification matrix by using the dither pattern and the LSB group;
- creating a second MSB group by using the modification matrix and the first MSB group;
- creating a reduced-amplitude luminance matrix based on the second MSB group; and
- temporally dithering the reduced-amplitude luminance matrix.
17. The method of claim 16, comprising temporally dithering the selected spatial dither pattern.
18. The method of claim 17, wherein the temporally dithering of the selected spatial dither pattern comprises a clockwise temporal dithering or a counterclockwise temporal dithering.
19. The method of claim 17, wherein the temporally dithering of the selected spatial dither pattern comprises dividing the spatial dither pattern into a plurality of quadrants and then performing either a clockwise shifting of bit values in each quadrant or a counterclockwise shifting of the bit values in each quadrant.
20. The method of claim 16, comprising temporally dithering the second MSB group.
21. A non-transitory computer-readable medium comprising code adapted to:
- create a first most significant bit (MSB) group by selecting most significant bits from an area of a source image;
- create a least significant bit (LSB) group by selecting least significant bits from the area of the source image;
- select a dither pattern;
- create a modification matrix by using the LSB group; and
- create a second MSB group by adding the modification matrix to the first MSB group.
22. The non-transitory computer-readable medium of claim 21, wherein the code adapted to select the dither pattern comprises code adapted to use the LSB group to select the dither pattern.
23. The non-transitory computer-readable medium of claim 22, wherein the code adapted to use the LSB group to select the dither pattern comprises code adapted to select the dither pattern based on a magnitude of a value of a cell in the LSB group.
24. The non-transitory computer-readable medium of claim 21, wherein the code adapted to create the second MSB group by adding the modification matrix to the first MSB group comprises code adapted to add a first value corresponding to a first cell in the first MSB group to a second value corresponding to a second cell in the modification matrix.
Type: Application
Filed: Dec 16, 2010
Publication Date: Jun 21, 2012
Applicant: APPLE INC. (Cupertino, CA)
Inventor: Ulrich T. Barnhoefer (Cupertino, CA)
Application Number: 12/970,543
International Classification: G09G 5/02 (20060101); G06K 9/00 (20060101);