Abstract: A gamut size metric for use in all phases of color image processing (e.g., capture, transmission, and display) is described. In general, techniques are disclosed for determining a single-valued metric that changes as image content changes. More particularly, techniques disclosed herein may be used to efficiently determine a gamut boundary histogram which may be used to estimate a gamut size metric. A gamut size metric as disclosed herein identifies a minimum size gamut needed to encompass each pixel in an image, where the gamut size is limited at one end by a first device independent gamut (S1), and at another end by a second device independent color space (S2), where S1 is wholly enclosed within S2. The described gamut size metric may be based on strict pixel color value differences. In other embodiments the gamut size metric may take into effect perceptual color differences and significance.

Abstract: A system and method convert a pixel of binary image data to a pixel of contone image data by determining if a predetermined pixel of binary image data is part of a solid edge or part of a fuzzy edge. A binary to contone conversion circuit converts the predetermined pixel of binary image data to a pixel of a first contone image data value, and a filter circuit converts the predetermined pixel of binary image data to a pixel of a second contone image data value. The filter circuit uses an adaptive filtering operation wherein the adaptive filtering operation utilizes one of a plurality of sets of weighting coefficients to change a characteristic of the filtering operation. The set of weighting coefficients used in the filtering operation are selected in response to a fuzzy edge detection. A selection between the first contone image data value and the second contone image data value is made based upon the determination as whether the predetermined pixel of binary image data is part of a solid edge.

Abstract: This disclosure pertains to systems, methods, and computer readable media for extending the dynamic range of images using an operation referred to herein as “Adaptive Auto Exposure” (AAE). According to the embodiments disclosed herein, the AAE-enabled higher dynamic range capture operations are accomplished without blending multiple or bracketed exposure captures (as is the case with traditional high dynamic range (HDR) photography). AAE also enables high signal-to-noise ratio (SNR) rendering when scene content allows for it and/or certain highlight clipping is tolerable. Decisions with regard to preferred AE strategies may be based, at least in part, on one or more of the following: sensor characteristics; scene content; and pre-defined preferences under different scenarios.

Abstract: Methods, devices and computer readable instructions to generate multi-scale tone curves are disclosed. One method includes finding, for a given input image, a global tone curve that exhibits monotonic behavior. The input image may then be partitioned into a first number of sub-regions. For each sub-region, a local tone curve may be determined that has an output level that is constrained to the global tone curve at one or more first luminance levels so that each sub-region's local tone curve's output follows the global tone curve's monotonic behavior. If the resulting local tone curves provide sufficient control of shadow-boost, highlight-suppression, and contrast optimization the first number of local tone curves may be applied directly to the input image. If additional control is needed, each sub-region may again be partitioned and local tone curves determined for each of the new sub-regions.

Abstract: Methods, devices and computer readable instructions to generate region-of-interest (ROI) tone curves are disclosed. One method includes obtaining a statistic for an entire image such as, for example, a luminance statistic. The same statistic may then be found for a specified ROI of the image. A weighted combination of the statistic of the entire image and the statistic of the ROI yields a combined statistic which may then be converted to a ROI-biased tone curve. The weight used to combine the two statistics may be selected to emphasize or de-emphasize the role of the ROI's statistic in the final tone curve.

Abstract: Systems, methods, and computer readable media for the use of a metric whose value is especially sensitive to the information lost when an image's pixels are clipped are disclosed. The metric may be used as an image's score, where higher values are indicative of lost highlight information (more clipped pixels). One use of the disclosed metric would be to determine when the use of high dynamic range (HDR) techniques are appropriate. The disclosed metric may also be used to bias a scene's exposure value (EV) such as to a lower or underexposed value (EV?) so that the scene may be captured with no more than an acceptable number of clipped pixels.

Abstract: Systems, methods, and computer readable media to capture and process high dynamic range (HDR) images when appropriate for a scene are disclosed. When appropriate, multiple images at a single—slightly underexposed—exposure value are captured (making a constant bracket HDR capture sequence) and local tone mapping (LTM) applied to each image. Local tone map and histogram information can be used to generate a noise-amplification mask which can be used during fusion operations. Images obtained and fused in the disclosed manner provide high dynamic range with improved noise and de-ghosting characteristics.

Abstract: Methods, devices and computer readable instructions to generate multi-scale tone curves are disclosed. One method includes finding, for a given input image, a global tone curve that exhibits monotonic behavior. The input image may then be partitioned into a first number of sub-regions. For each sub-region, a local tone curve may be determined that has an output level that is constrained to the global tone curve at one or more first luminance levels so that each sub-region's local tone curve's output follows the global tone curve's monotonic behavior. If the resulting local tone curves provide sufficient control of shadow-boost, highlight-suppression, and contrast optimization the first number of local tone curves may be applied directly to the input image. If additional control is needed, each sub-region may again be partitioned and local tone curves determined for each of the new sub-regions.

Abstract: Systems, methods, and computer readable media for the use of a metric whose value is especially sensitive to the information lost when an image's pixels are clipped are disclosed. The metric may be used as an image's score, where higher values are indicative of lost highlight information (more clipped pixels). One use of the disclosed metric would be to determine when the use of high dynamic range (HDR) techniques are appropriate. The disclosed metric may also be used to bias a scene's exposure value (EV) such as to a lower or underexposed value (EV?) so that the scene may be captured with no more than an acceptable number of clipped pixels.

Abstract: Systems, methods, and computer readable media to capture and process high dynamic range (HDR) images when appropriate for a scene are disclosed. When appropriate, multiple images at a single—slightly underexposed—exposure value are captured (making a constant bracket HDR capture sequence) and local tone mapping (LTM) applied to each image. Local tone map and histogram information can be used to generate a noise-amplification mask which can be used during fusion operations. Images obtained and fused in the disclosed manner provide high dynamic range with improved noise and de-ghosting characteristics.

Abstract: An image system with a dual layer photodiode structure is provided for processing color images. In particular, the image system can include an image sensor that can include photodiodes with a dual layer photodiode structure. In some embodiments, the dual layer photodiode can include a first layer of photodiodes (e.g., a bottom layer), an insulation layer disposed on the first layer of photodiodes, and a second layer of photodiodes (e.g., a top layer) disposed on the insulation layer. The first layer of photodiodes can include one or more suitable pixels (e.g., green, blue, clear, luminance, and/or infrared pixels). Likewise, the second layer of photodiodes can include one or more suitable pixels (e.g., green, red, clear, luminance, and/or infrared pixels). An image sensor incorporating dual layer photodiodes can gain light sensitivity with additional clear pixels and maintain luminance information with green pixels.

Abstract: Systems and methods are provided for calibrating image sensors. In some embodiments, a processing module of an image system can automatically perform a self-calibration process after a production unit of an image sensor has been integrated into an end product system. For example, the processing module can calibrate a production unit based on one or more reference pixels of the production unit, where the one or more reference pixels have minimal color filtration. In some embodiments, the processing module may perform local calibrations by correcting specifically for spatial variations in a color filter array (“CFA”). In some embodiments, the processing module can perform global calibrations by correcting for optical density variations in the CFA. In some embodiments, a processing module can determine whether the cause of production variations is related to production variations of a CFA or production variations of an infrared (“IR”) cutoff filter.

Abstract: A novel chromaticity space is disclosed that may be used as a framework to implement an auto-white balance solution or other color image processing solutions that take advantage of the particular properties of the novel chromaticity space. The chromaticity space may be defined by using a series of mathematical transformations having parameters that are optimized to adapt to specific sensors' spectral sensitivities. The unique properties of the novel chromaticity space provide a conscious white point constraining strategy with clear physical meaning. In this chromaticity space, the ranges of possible white points under different kinds of lighting conditions can be defined by polygons. Because of the physical meaning the chromaticity space, the projection that is needed to bring an initially “out-of-bounds” white point back into the polygon also carries physical meaning, making the definition of projection behavior and its consequences conceptually clean and predictable.

Abstract: Methods, devices and computer readable media for implementing novel dominant color alleviation techniques for color balancing are described. The techniques take advantage of unique properties of 2D image data histograms accumulated in a chromaticity space, along with other factors such as estimated scene lux and knowledge of plausible scene illuminant white point values within the chromaticity space. The accumulated 2D image data histograms may be refined and “trimmed,” such that the resultant image data passed to an auto white balance solution has much less influence from the dominant colors in the image, even those that overlap the plausible scene illuminant color region. The described techniques provide for white point estimates that are much less prone to dominant color failures.

Abstract: This disclosure pertains to systems, methods, and computer readable media for extending the dynamic range of images using an operation referred to herein as “Adaptive Auto Exposure” (AAE). According to the embodiments disclosed herein, the AAE-enabled higher dynamic range capture operations are accomplished without blending multiple or bracketed exposure captures (as is the case with traditional high dynamic range (HDR) photography). AAE also enables high signal-to-noise ratio (SNR) rendering when scene content allows for it and/or certain highlight clipping is tolerable. Decisions with regard to preferred AE strategies may be based, at least in part, on one or more of the following: sensor characteristics; scene content; and pre-defined preferences under different scenarios.

Abstract: This disclosure pertains to systems, methods, and computer readable media for performing lens shading correction (LSC) operations that modulate gains based on scene lux level and lens focus distance. These gains compensate for both color lens shading (i.e., the deviation between R, G, and B channels) and vignetting (i.e., the drop off in pixel intensity around the edges of an image). As scene illuminance increases, the sensor captures more signal from the actual scene, and the lens shading effects begin to appear. To deal with the situation, the lens shading gains are configured to adaptively ‘scale down’ when scene lux approaches zero and ‘scale up’ when scene lux changes from near zero to become larger. The lens shading gain may also be modulated based on the focus distance. For optical systems without zoom, the inventors have discovered that the amount of lens shading fall off changes as focus distance changes.

Abstract: Methods, devices and computer readable media for implementing a “selective gray world” approach for color balancing are described. The disclosed techniques involve the use of noise-optimized selection criteria and, more specifically, in some embodiments, the interpolation between corresponding values in noise-optimized weighting tables when calculating white balance gains. Estimated scene lux levels may provide a valuable indicator of expected scene noise levels. The image processing techniques described herein may be executed by an image capture device or a general purpose processor (e.g., personal computer) executing a user-level software application. The described color balancing techniques may be implemented by dedicated or general purpose hardware, general application software, or a combination of software and hardware in a computer system.

Abstract: Stereoscopic imaging devices may include stereoscopic imagers, stereoscopic displays, and processing circuitry. The processing circuitry may be used to collect auto white balance (AWB) statistics for each image captured by the stereoscopic imager. A stereoscopic imager may include two image modules that may be color calibrated relative to each other or relative to a standard calibrator. AWB statistics may be used by the processing circuitry to determine global, local and spatial offset gain adjustments to provide intensity matched stereoscopic images for display. AWB statistics may be combined by the processing circuitry with color correction offsets determined during color calibration to determine color-transformation matrices for displaying color matched stereoscopic images using the stereoscopic display.

Abstract: Systems and methods are provided for obtaining adaptive exposure control and dynamic range extension of image sensors. In some embodiments, an image sensor of an image system can include a pixel array with one or more clear pixels. The image system can separately control the amount of time that pixels in different lines of the pixel array are exposed to light. As a result, the image system can adjust the exposure times to prevent over-saturation of the clear pixels, while also allowing color pixels of the pixel array to be exposed to light for a longer period of time. In some embodiments, the dynamic range of the image system can be extended through a reconstruction and interpolation process. For example, a signal reconstruction module can extend the dynamic range of one or more green pixels by combining signals associated with green pixels in different lines of the pixel array.

Abstract: Systems and methods are provided for obtaining adaptive exposure control and dynamic range extension of image sensors. In some embodiments, an image sensor of an image system can include a pixel array with one or more clear pixels. The image system can separately control the amount of time that pixels in different lines of the pixel array are exposed to light. As a result, the image system can adjust the exposure times to prevent over-saturation of the clear pixels, while also allowing color pixels of the pixel array to be exposed to light for a longer period of time. In some embodiments, the dynamic range of the image system can be extended through a reconstruction and interpolation process. For example, a signal reconstruction module can extend the dynamic range of one or more green pixels by combining signals associated with green pixels in different lines of the pixel array.