Abstract: Various techniques are provided for processing image data acquired using a digital image sensor. In accordance with aspects of the present disclosure, one such technique may relate to the processing of image data in a system that supports multiple image sensors. In one embodiment, the image processing system may include control circuitry configured to determine whether a device is operating in a single sensor mode (one active sensor) or a dual sensor mode (two active sensors). When operating in the single sensor mode, data may be provided directly to a front-end pixel processing unit from the sensor interface of the active sensor. When operating in a dual sensor mode, the image frames from the first and second sensors are provided to the front-end pixel processing unit in an interleaved manner. For instance, in one embodiment, the image frames from the first and second sensors are written to a memory, and then read out to the front-end pixel processing unit in an interleaved manner.

Abstract: Various techniques for applying binning compensation filtering to binned raw image data acquired by an image sensor are provided. In one embodiment, a binning compensation filter (BCF) includes separate digital differential analyzers (DDA) for vertical and horizontal scaling. A current position of an output pixel is determined by incrementing the DDA based upon a step size. Using the known output pixel position, a center source input pixel and an index corresponding to the between-pixel fractional position of the output pixel position relative to the input pixels may be selected for filtering. Using the selected center input pixel, one or more same-colored neighboring source pixels may be selected. The number of selected source pixels may depend on the number of taps used by the scaling logic, and may depend on whether horizontal or vertical scaling is being applied.

Abstract: In an embodiment, a compression unit is provided which may perform compression of images with low latency and relatively little hardware. Similarly, a decompression unit may be provided which may decompress the images with low latency and hardware. In an embodiment, the transmission of compressed coefficients may be performed using less than two passes through the list of coefficients. During the first pass, the most significant coefficients may be transmitted and other significance groups may be identified as linked lists. The linked lists may then be traverse to send the other significance groups. In an embodiment, a color space conversion may be made to permit filtering of fewer color components than might be possible in the source color space.

Abstract: In an embodiment, a compression unit is provided which may perform compression of images with low latency and relatively little hardware. Similarly, a decompression unit may be provided which may decompress the images with low latency and hardware. In an embodiment, the transmission of compressed coefficients may be performed using less than two passes through the list of coefficients. During the first pass, the most significant coefficients may be transmitted and other significance groups may be identified as linked lists. The linked lists may then be traverse to send the other significance groups. In an embodiment, a color space conversion may be made to permit filtering of fewer color components than might be possible in the source color space.

Abstract: Various techniques are provided herein for processing raw image data in front-end processing logic of an image signal processing system. In one embodiment, the front-end processing logic includes a statistics processing unit configured to process raw image data acquired by an image sensor to obtain one or more sets of statistics. The statistics processing unit may first correct defective pixels in the raw image data and then correct lens shading errors in the raw image data prior to extracting the statistics information. In certain embodiments, black level compensation may be applied between the defective pixel correction and lens shading correction steps, and inverse black level compensation may be applied between the lens shading correction step and the extraction of the statistics information. The acquired statistics information may be utilized by an image signal processing pipeline for converting the raw image data into a color (e.g., RGB) and/or luma (e.g., YCbCr) image.

Abstract: Various techniques relating to image sharpening are provided. In one embodiment, a luminance image is obtained based upon image data acquired by an image sensor. A multi-scale unsharp mask, which may include at least two Gaussian filters of difference radii, is applied to the luminance image to determine a plurality of unsharp values. Each of the unsharp values may be compared to a corresponding threshold and, for the unsharp values that exceed their respective thresholds, the unsharp value is multiplied by a corresponding gain and added to a base image, which may be selected as the luminance image or the output of one of the Gaussian filters. Each gained unsharp value may be summed with the base image to produce a final sharpened output. In some embodiments, an attenuated gain may be applied to unsharp values that do not exceed their respective thresholds.

Abstract: Various techniques for lens shading correction are provided. In one embodiment, the location of a current pixel is determined relative to a gain grid having a plurality of grid points distributed in horizontal and vertical directions. If the location of the current pixel corresponds to a grid point, a lens shading gain associated with that grid point is applied to the current pixel. If the location of the current pixel is between four grid points, bi-linear interpolation is applied to the four grid points to determine an interpolated lens shading gain. In another embodiment, a radial gain grid may be provided, and lens shading gains may be interpolated based upon grid points neighboring a current pixel in the radial and angular directions. In a further embodiment, a radial lens shading gain is determined by determining a radial distance from the center of the image to the current pixel and multiplying the radial distance by a global gain parameter based upon the color of the current pixel.

Abstract: Various techniques are provided for the detection and correction of defective pixels in an image sensor. In accordance with one embodiment, a static defect table storing the locations of known static defects is provided, and the location of a current pixel is compared to the static defect table. If the location of the current pixel is found in the static defect table, the current pixel is identified as a static defect and is corrected using the value of the previous pixel of the same color. If the current pixel is not identified as a static defect, a dynamic defect detection process includes comparing pixel-to-pixel gradients between the current pixel a set of neighboring pixels against a dynamic defect threshold. If a dynamic defect is detected, a replacement value for correcting the dynamic defect may be determined by interpolating the value of two neighboring pixels on opposite sides of the current pixel in a direction exhibiting the smallest gradient.

Abstract: Various techniques are provided herein for the demosaicing of images acquired and processed by an imaging system. The imaging system includes an image signal processor and image sensors utilizing color filter arrays (CFA) for acquiring red, green, and blue color data using one pixel array. In one embodiment, the CFA may include a Bayer pattern. During image signal processing, demosaicing may be applied to interpolate missing color samples from the raw image pattern. In one embodiment, interpolation for the green color channel may include employing edge-adaptive filters with weighted gradients of horizontal and vertical filtered values. The red and blue color channels may be interpolated using color difference samples with co-located interpolated values of the green color channel. In another embodiment, interpolation of the red and blue color channels may be performed using color ratios (e.g., versus color difference data).

Abstract: Various techniques are provided herein for processing raw image data acquired using a digital image sensor in an image processing pipeline of an image signal processing system. In one embodiment, the image processing pipeline may first process the raw image data (e.g., Bayer image data) for the detection and correction of defective pixels. Next, the image processing pipeline may process the raw image data to reduce noise. Thereafter, the image processing pipeline may correct lens shading distortion in the raw image data and, subsequently, apply a demosaicing algorithm to convert the raw image data into full color image data (e.g., RGB image data). The color image data may be further processed by the image processing pipeline to correct color and gamma properties prior to being converted into a luma and chroma color space (e.g., YCbCr color space).

Abstract: Various techniques for temporally filtering raw image data acquired by an image sensor are provided. In one embodiment, a temporal filter determines a spatial location of a current pixel and identifies at least one collocated reference pixel from a previous frame. A motion delta value is determined based at least partially upon the current pixel and its collocated reference pixel. Next, an index is determined based upon the motion delta value and a motion history value corresponding to the spatial location of the current pixel, but from the previous frame. Using the index, a first filtering coefficient may be selected from a motion table. After selecting the first filtering coefficient, an attenuation factor may be selected from a luma table based upon the value of the current pixel, and a second filtering coefficient may subsequently be determined based upon the selected attenuation factor and the first filtering coefficient.

Abstract: A method system for data reduction of digital video signals based on vector quantization of vectors formed from coefficients of a discrete cosine transform of pixel blocks. The coefficients are grouped into subbands and both scalar and vector quantization are used. Vector quantization is implemented either directly on the vectors or on vectors formed from inter-frame differences between the transformed vectors. The vector quantization searching routine is in accordance with the Voronoi regions resulting from an off-line codeword clustering method using a minimum distance criterion.

Abstract: A method and system for video motion compensation in which an overall interframe motion vector is divided into a relative block motion vector and a subblock motion vector. Image reconstruction is achieved by adjusting the delay between the synchronization pulse and the start of the pixel data according to the subblock motion vector, and accessing stored data from memory according to the relative block motion vector. Accessing memory in a block-quantized format ensures that page mode accesses are confined to the same memory row.

Abstract: Programmable apparatus for digital processing of video signals from multiple sources converted to digital format to provide real-time multiple simultaneous special video effects and suitable for direct interface to a conventional microcomputer bus such as an Apple Macintosh II NuBus. The apparatus includes a matrix of multipliers to do real-time video processing permitting special effects such as fading between at least two video image sources, as well as a priority resolver to control display on a pixel by pixel basis of more than ten sources based upon dynamically programmable priority. In addition, a programmable multiple range thresholder, a hardware window generator capable of generating multiple simultaneous windows, a color look up table and optional image capture capabilities are provided. The apparatus also provides for a light pen input, genlocking and a range of special video effects including zooming, mosaicing, panning and blending.

Abstract: There is provided a video display system operable in a plurality of color resolution and pixel resolution modes while maintaining constant spatial data resolution and having a video display with a plurality of picture elements responsive to picture signals derived from video data in storage blocks which are spatially correlated with respective picture element blocks of the video display, and wherein the density of data storage per area of display is constant and the input/output of video data can take place at a constant data rate.

Abstract: There is provided a video game system utilizing a video disk as a read only memory to store digital video image data for use in generating the video game images. The video data is stored by compressing digitized video data using color cell compression and then further compressing the data to multi-level data encoded with partial response coding. The data is retrieved from the video disk ROM by using an inverse of this compression process.

Abstract: A digital color video image digitizing and communication process comprises the steps of dividing digital data representing the overall picture information for a video image into sets of data representing shape and color information for blocks of the image with the color information including the major colors for the corresponding block and the variation in shades or hues of the colors within the block, separating, according to a predefined encoding procedure, each of the data sets into a first group of data representing the major colors constituting the block and a second group of data comprising the remaining data in the data sets and representing the shades of the major colors within the block, storing the second group of data in a color table, processing the second group of data and transmitting the first group of data and the processed second group of data to a decoder capable of processing received data according to a predefined decoding procedure, processing the received first data group to display the

Abstract: A subscriber cable television system uses predominantly digital signal processing techniques and has extremely high security and an increased capacity for transmitting program and customer data to individual decoder units. For ease of data handling, two-channel audio, video, and high capacity program and customer data are multiplexed for transmission on the composite video signal. The decoder unit employs a system timing circuit which precisely synchronizes the sample times on the received composite video signal to the chroma burst, regardless of whether the video information is for a color or black-and-white program. An improved time-warp and segment scrambling method is disclosed along with means for suppressing the undesirable effects of discontinuities in the scrambled video signal. The digital audio is transmitted as scrambled most significant bits in low resolution samples and unscrambled least significant bits in a high resolution remainder sample.

Abstract: A subscriber cable television system uses predominantly digital signal processing techniques and has extremely high security and an increased capacity for transmitting program and customer data to individual decoder units. For ease of data handling, two-channel audio, video, and high capacity program and customer data are multiplexed for transmission on the composite video signal. The decoder unit employs a system timing circuit which precisely synchronizes the sample times on the received composite video signal to the chroma burst, regardless of whether the video information is for a color or black-and-white program. An improved time-warp and segment scrambling method is disclosed along with means for suppressing the undesirable effects of discontinuities in the scrambled video signal. The digital audio is transmitted as scrambled most significant bits in low resolution samples and unscrambled least significant bits in a high resolution remainder sample.

Abstract: A joystick position transducer system for use in a video game. The joystick position transducer includes a pair of inductors each having a movable slug, the slugs being coupled to a control handle or knob by means of respective linkages. The control handle or knob is responsive to external stimulus to move the slugs, the inductors providing respective first and second analog signals which are proportional to the position of the slugs. Coupled to each of the inductors is an oscillator and counter for providing a pulsed output, the pulse width of which is proportional to the position of the respective slug. A controller, such as a computer control or logic sequence circuit, is provided to control the oscillators and counters. The controller is further responsive to the counter outputs to control a display presentation on a video display coupled thereto.