Abstract: An image signal processor of a device, apparatus, or computing system that includes a camera capable of capturing image data may apply piecewise perspective transformations to image data received from the camera's image sensor. A scaling unit of an Image Signal Processor (ISP) may perform piecewise perspective transformations on a captured image to correct for rolling shutter artifacts and to provide video image stabilization. Image data may be divided into a series of horizontal slices and perspective transformations may be applied to each slice. The transformations may be based on motion data determined in any of various manners, such as by using gyroscopic data and/or optical-flow calculations. The piecewise perspective transforms may be encoded as Digital Difference Analyzer (DDA) steppers and may be implemented using separable scalar operations. The image signal processor may not write the received image data to system memory until after the transformations have been performed.

Abstract: An image signal processor of a device, apparatus, or computing system that includes a camera capable of capturing video clips may be configured to apply non-warping and warping transformations (including image stabilization transformations) to image data received from the camera's image sensor. The transformations may be applied piecewise to pixels of the image data gathered into multiple line buffers. The line buffers may be divided into vertical slices, each of which may be processed in parallel by a respective portion of the image signal processor circuitry (or circuitry of an output rescaling unit thereof). The image signal processor may not write the received image data to system memory until after the transformations have been performed. By leveraging existing filters and scalers of the image signal processor, rather than a CPU or GPU, image stabilization may be provided that consumes half of the bandwidth and power consumed by existing techniques.

Abstract: An input rescale module that performs cross-color correlated downscaling of sensor data in the horizontal and vertical dimensions. The module may perform a first-pass demosaic of sensor data, apply horizontal and vertical scalers to resample and downsize the data in the horizontal and vertical dimensions, and then remosaic the data to provide horizontally and vertically downscaled sensor data as output for additional image processing. The module may, for example, act as a front end scaler for an image signal processor (ISP). The demosaic performed by the module may be a relatively simple demosaic, for example a demosaic function that works on 3×3 blocks of pixels. The front end of module may receive and process sensor data at two pixels per clock (ppc); the horizontal filter component reduces the sensor data down to one ppc for downstream components of the input rescale module and for the ISP pipeline.

Abstract: An input rescale module for an image signal processor (ISP) that downscales sensor data in the horizontal and vertical dimensions. The module may demosaic the sensor data to generate RGB data. Horizontal filtering may be applied to horizontally downsize the RGB data. The RGB data is converted to YCC, chroma 4:4:4. The chroma 4:4:4 is then horizontally filtered to generate chroma 4:2:2. Dropping chrominance data by going to 4:2:2 may reduce hardware area cost and power usage in the vertical scaler. Vertical filtering may be applied separately to luma and chroma to vertically downsize the YCC data. Chroma may be filtered with stronger filters than luma. The chroma 4:2:2 data may then be horizontally interpolated to generate chroma 4:4:4 data. The YCC data is converted back to RGB, and the RGB data is remosaiced to generate downsampled sensor format data.

Abstract: The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may detect and correct a defective pixel of image data acquired using an image sensor. The image processing pipeline may receive an input pixel of the image data acquired using the image sensor. The image processing pipeline may then identify a set of neighboring pixels having the same color component as the input pixel and remove two neighboring pixels from the set of neighboring pixels thereby generating a modified set of neighboring pixels. Here, the two neighboring pixels correspond to a maximum pixel value and a minimum pixel value of the set of neighboring pixels.

Abstract: The present disclosure relates generally systems and methods for image data processing. In certain embodiments, a method for transferring the image data may include processing the image data in an image sensor by correcting one or more defective pixels in the image data based on a one-dimensional defective pixel correction algorithm; horizontally demosaicing the image data based on a one-dimensional horizontal demosaic algorithm; or a combination thereof. After processing the image data, the method may include horizontally downscaling the processed image data in the image sensor and vertically downscaling the horizontally downscaled image data in an image signal processor to be displayed for viewing on a display device.

Abstract: Data for data elements (e.g., pixels) can be stored in an addressable storage unit that can store a number of bits that is not a whole number multiple of the number of bits of data per data element. Similarly, a number of the data elements can be transferred per unit of time over a bus, where the width of the bus is not a whole number multiple of the number of bits of data per data element. Data for none of the data elements is stored in more than one of the storage units or transferred in more than one unit of time. Also, data for multiple data elements is packaged contiguously in the storage unit or across the width of the bus.

Abstract: Described are methods and systems for processing data. A motion estimator uses a block of an input frame of video data and a block of a reference frame of video data to generate motion vectors according to a first encoding scheme. A motion compensator produces half pel motion vectors from the motion vectors according to a second encoding scheme that is different from the first encoding scheme.

Abstract: The video stabilization method can generate output data for an output frame from input data of an input frame according to a perspective transform of a transform matrix. The input data used for the perspective transform can be obtained from a buffer of a predetermined depth. The transform matrix can be altered when the input data required for the transform exceeds the depth of the buffer.

Abstract: An image signal processor of a device, apparatus, or computing system that includes a camera capable of capturing video clips may be configured to apply non-warping and warping transformations (including image stabilization transformations) to image data received from the camera's image sensor. The transformations may be applied piecewise to pixels of the image data gathered into multiple line buffers. The line buffers may be divided into vertical slices, each of which may be processed in parallel by a respective portion of the image signal processor circuitry (or circuitry of an output rescaling unit thereof). The image signal processor may not write the received image data to system memory until after the transformations have been performed. By leveraging existing filters and scalers of the image signal processor, rather than a CPU or GPU, image stabilization may be provided that consumes half of the bandwidth and power consumed by existing techniques.

Abstract: The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may detect and correct a defective pixel of image data acquired using an image sensor. The image processing pipeline may receive an input pixel of the image data acquired using the image sensor. The image processing pipeline may then identify a set of neighboring pixels having the same color component as the input pixel and remove two neighboring pixels from the set of neighboring pixels thereby generating a modified set of neighboring pixels. Here, the two neighboring pixels correspond to a maximum pixel value and a minimum pixel value of the set of neighboring pixels.

Abstract: The video stabilization method can generate output data for an output frame from input data of an input frame according to a perspective transform of a transform matrix. The input data used for the perspective transform can be obtained from a buffer of a predetermined depth. The transform matrix can be altered when the input data required for the transform exceeds the depth of the buffer.

Abstract: The present disclosure relates generally systems and methods for image data processing. In certain embodiments, a method for transferring the image data may include processing the image data in an image sensor by correcting one or more defective pixels in the image data based on a one-dimensional defective pixel correction algorithm; horizontally demosaicing the image data based on a one-dimensional horizontal demosaic algorithm; or a combination thereof. After processing the image data, the method may include horizontally downscaling the processed image data in the image sensor and vertically downscaling the horizontally downscaled image data in an image signal processor to be displayed for viewing on a display device.

Abstract: Data for data elements (e.g., pixels) can be stored in an addressable storage unit that can store a number of bits that is not a whole number multiple of the number of bits of data per data element. Similarly, a number of the data elements can be transferred per unit of time over a bus, where the width of the bus is not a whole number multiple of the number of bits of data per data element. Data for none of the data elements is stored in more than one of the storage units or transferred in more than one unit of time. Also, data for multiple data elements is packaged contiguously in the storage unit or across the width of the bus.

Abstract: Described are methods and systems for processing data. A motion estimator uses a block of an input frame of video data and a block of a reference frame of video data to generate motion vectors according to a first encoding scheme. A motion compensator produces half pel motion vectors from the motion vectors according to a second encoding scheme that is different from the first encoding scheme.