Abstract: In one embodiment, a system includes a first device rendering image data, a second device storing the image data, and a display panel that displays the image data stored in the memory. The first device renders multiple frames of the image data, compresses the multiple frames into a single superframe, and transports the single superframe. The second device receives the single superframe, decompresses the single superframe into the multiple frames of image data, and stores the image data on a memory of the second device.

Abstract: The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may be configured to receive a frame of the image data having a plurality of pixels acquired using a digital image sensor. The image processing pipeline may then be configured to determine a first plurality of correction factors that may correct each pixel in the plurality of pixels for fixed pattern noise. The first plurality of correction factors may be determined based at least in part on fixed pattern noise statistics that correspond to the frame of the image data. After determining the first plurality of correction factors, the image processing pipeline may be configured to configured to apply the first plurality of correction factors to the plurality of pixels, thereby reducing the fixed pattern noise present in the plurality of pixels.

Abstract: In one embodiment, a system includes a first device rendering image data, a second device storing the image data, and a display panel that displays the image data stored in the memory. The first device renders multiple frames of the image data, compresses the multiple frames into a single superframe, and transports the single superframe. The second device receives the single superframe, decompresses the single superframe into the multiple frames of image data, and stores the image data on a memory of the second device.

Abstract: The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may be configured to receive a frame of the image data having a plurality of pixels acquired using a digital image sensor. The image processing pipeline may then be configured to determine a first plurality of correction factors that may correct each pixel in the plurality of pixels for fixed pattern noise. The first plurality of correction factors may be determined based at least in part on fixed pattern noise statistics that correspond to the frame of the image data. After determining the first plurality of correction factors, the image processing pipeline may be configured to configured to apply the first plurality of correction factors to the plurality of pixels, thereby reducing the fixed pattern noise present in the plurality of pixels.

Abstract: Some embodiments relate to sharpening segments of an image differently based on content in the image. Content based sharpening is performed by a content image processing circuit that receives luminance values of an image and a content map. The content map identifies categories of content in segments of the image. Based on one or more of the identified categories of content, the circuit determines a content factor associated with a pixel. The content factor may also be based on a texture and/or chroma values. A texture value indicates a likelihood of a category of content and is based on detected edges in the image. A chroma value indicates a likelihood of a category of content and is based on color information of the image. The circuit receives the content factor and applies it to a version of the luminance value of the pixel to generate a sharpened version of the luminance value.

Abstract: Some embodiments relate to sharpening segments of an image differently based on content in the image. Content based sharpening is performed by a content image processing circuit that receives luminance values of an image and a content map. The content map identifies categories of content in segments of the image. Based on one or more of the identified categories of content, the circuit determines a content factor associated with a pixel. The content factor may also be based on a texture and/or chroma values. A texture value indicates a likelihood of a category of content and is based on detected edges in the image. A chroma value indicates a likelihood of a category of content and is based on color information of the image. The circuit receives the content factor and applies it to a version of the luminance value of the pixel to generate a sharpened version of the luminance value.

Abstract: In one embodiment, a system includes a first device rendering image data, a second device storing the image data, and a display panel that displays the image data stored in the memory. The first device renders multiple frames of the image data, compresses the multiple frames into a single superframe, and transports the single superframe. The second device receives the single superframe, decompresses the single superframe into the multiple frames of image data, and stores the image data on a memory of the second device.

Abstract: Embodiments relate to image signal processors (ISP) that include binner circuits that down-sample an input image. An input image may include a plurality of pixels. The output image of the binner circuit may include a reduced number of pixels. The binner circuit may include a plurality of different operation modes. In a bin mode, the binner circuit may blend a subset of input pixel values to generate an output pixel quad. In a skip mode, the binner circuit may select one of the input pixel values as the output pixel. The selection may be performed randomly to avoid aliasing. In a luminance mode, the binner circuit may take a weighted average of a subset of pixel values having different colors. In a color value mode, the binner circuit may select one of the colors in a subset of pixel values as an output pixel value.

Abstract: Embodiments relate to image signal processors (ISP) that include binner circuits that down-sample an input image. An input image may include a plurality of pixels. The output image of the binner circuit may include a reduced number of pixels. The binner circuit may include a plurality of different operation modes. In a bin mode, the binner circuit may blend a subset of input pixel values to generate an output pixel quad. In a skip mode, the binner circuit may select one of the input pixel values as the output pixel pixel. The selection may be performed randomly to avoid aliasing. In a luminance mode, the binner circuit may take a weighted average of a subset of pixel values having different colors. In a color value mode, the binner circuit may select one of the colors in a subset of pixel values as an output pixel value.

Abstract: Methods and apparatus for focusing in virtual reality (VR) or augmented reality (AR) devices based on gaze tracking information are described. Embodiments of a VR/AR head-mounted display (HMD) may include a gaze tracking system for detecting position and movement of the user's eyes. For AR applications, gaze tracking information may be used to direct external cameras to focus in the direction of the user's gaze so that the cameras focus on objects at which the user is looking. For AR or VR applications, the gaze tracking information may be used to adjust the focus of the eye lenses so that the virtual content that the user is currently looking at on the display has the proper vergence to match the convergence of the user's eyes.

Abstract: The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may be configured to receive a frame of the image data having a plurality of pixels acquired using a digital image sensor. The image processing pipeline may then be configured to determine a first plurality of correction factors that may correct each pixel in the plurality of pixels for fixed pattern noise. The first plurality of correction factors may be determined based at least in part on fixed pattern noise statistics that correspond to the frame of the image data. After determining the first plurality of correction factors, the image processing pipeline may be configured to configured to apply the first plurality of correction factors to the plurality of pixels, thereby reducing the fixed pattern noise present in the plurality of pixels.

Abstract: Embodiments relate to image signal processors (ISP) that include binner circuits that down-sample an input image. An input image may include a plurality of pixels. The output image of the binner circuit may include a reduced number of pixels. The binner circuit may include a plurality of different operation modes. In a bin mode, the binner circuit may blend a subset of input pixel values to generate an output pixel quad. In a skip mode, the binner circuit may select one of the input pixel values as the output pixel pixel. The selection may be performed randomly to avoid aliasing. In a luminance mode, the binner circuit may take a weighted average of a subset of pixel values having different colors. In a color value mode, the binner circuit may select one of the colors in a subset of pixel values as an output pixel value.

Abstract: Some embodiments relate to sharpening segments of an image differently based on content in the image. Content based sharpening is performed by a content image processing circuit that receives luminance values of an image and a content map. The content map identifies categories of content in segments of the image. Based on one or more of the identified categories of content, the circuit determines a content factor associated with a pixel. The content factor may also be based on a texture and/or chroma values. A texture value indicates a likelihood of a category of content and is based on detected edges in the image. A chroma value indicates a likelihood of a category of content and is based on color information of the image. The circuit receives the content factor and applies it to a version of the luminance value of the pixel to generate a sharpened version of the luminance value.

Abstract: The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may be configured to receive a frame of the image data having a plurality of pixels acquired using a digital image sensor. The image processing pipeline may then be configured to determine a first plurality of correction factors that may correct each pixel in the plurality of pixels for fixed pattern noise. The first plurality of correction factors may be determined based at least in part on fixed pattern noise statistics that correspond to the frame of the image data. After determining the first plurality of correction factors, the image processing pipeline may be configured to configured to apply the first plurality of correction factors to the plurality of pixels, thereby reducing the fixed pattern noise present in the plurality of pixels.

Abstract: Methods and apparatus for focusing in virtual reality (VR) or augmented reality (AR) devices based on gaze tracking information are described. Embodiments of a VR/AR head-mounted display (HMD) may include a gaze tracking system for detecting position and movement of the user's eyes. For AR applications, gaze tracking information may be used to direct external cameras to focus in the direction of the user's gaze so that the cameras focus on objects at which the user is looking. For AR or VR applications, the gaze tracking information may be used to adjust the focus of the eye lenses so that the virtual content that the user is currently looking at on the display has the proper vergence to match the convergence of the user's eyes.

Abstract: Embodiments relate to image signal processors (ISP) that include binner circuits that down-sample an input image. An input image may include a plurality of pixels. The output image of the binner circuit may include a reduced number of pixels. The binner circuit may include a plurality of different operation modes. In a bin mode, the binner circuit may blend a subset of input pixel values to generate an output pixel quad. In a skip mode, the binner circuit may select one of the input pixel values as the output pixel pixel. The selection may be performed randomly to avoid aliasing. In a luminance mode, the binner circuit may take a weighted average of a subset of pixel values having different colors. In a color value mode, the binner circuit may select one of the colors in a subset of pixel values as an output pixel value.

Abstract: Embodiments relate to image signal processors (ISP) that include one or more auto-focus circuits. Each of the auto-focus circuits may be connected to an image sensor and may be separate from a statistics circuit and other image processing pipelines of the ISP. An image sensor may include one or more focus pixels that are used to generate data for auto-focusing. The auto-focus circuit may extract the focus pixel values and generate a signal to control the lens position of the image sensor. Each image sensor may include a separate auto-focus circuit. When other image processing pipelines of the ISP are processing the image data from one image sensor, the auto-focus circuit for another image sensor may continue to generate focus signals that control the lens position of the other image sensor. The other image sensor may be in standby but may continue to remain in focus.

Abstract: Methods and apparatus for focusing in virtual reality (VR) or augmented reality (AR) devices based on gaze tracking information are described. Embodiments of a VR/AR head-mounted display (HMD) may include a gaze tracking system for detecting position and movement of the user's eyes. For AR applications, gaze tracking information may be used to direct external cameras to focus in the direction of the user's gaze so that the cameras focus on objects at which the user is looking. For AR or VR applications, the gaze tracking information may be used to adjust the focus of the eye lenses so that the virtual content that the user is currently looking at on the display has the proper vergence to match the convergence of the user's eyes.

Abstract: In an embodiment, an electronic device may be configured to capture still frames during video capture but may capture the still frames in the 4×3 aspect ratio and at higher resolution than the 16×9 aspect ratio video frames. The device may interleave high resolution, 4×3 frames and lower resolution 16×9 frames in the video sequence, and may capture the nearest higher resolution, 4×3 frame when the user indicates the capture of a still frame. Alternatively, the device may display 16×9 frames in the video sequence, and then expand to 4×3 frames when a shutter button is pressed. The device may capture the still frame and return to the 16×9 video frames responsive to a release of the shutter button.

Abstract: An image signal processor may include a pixel defect correction component that tracks defect history for frames captured by an image sensor and applies the history when identifying and correcting defective pixels in a frame. The component maintains a defect pixel location table that includes a defect confidence value for pixels of the image sensor. The component identifies defective pixels in a frame, for example by comparing each pixel's value to the values of its neighbor pixels. If a pixel is detected as defective, its defect confidence value may be incremented. Otherwise, the value may be decremented. If a pixel's defect confidence value is over a defect confidence threshold, the pixel is considered defective and thus may be corrected. If a pixel's defect confidence value is under the threshold, the pixel is considered not defective and thus may not be corrected even if the pixel was detected as defective.