DATA PROCESSING APPARATUS FOR TRANSMITTING/RECEIVING COMPRESSION-RELATED INDICATION INFORMATION VIA CAMERA INTERFACE AND RELATED DATA PROCESSING METHOD

Abstract

A data processing apparatus has a compressor and an output interface. The compressor generates a compressed multimedia data by compressing a multimedia data according to a compression algorithm. The output interface records indication information in an output bitstream, and outputs the output bitstream via a camera interface, wherein the output bitstream is derived from the compressed multimedia data, and the indication information is set in response to the compression algorithm employed by the compressor. Another data processing apparatus has a de-compressor and an input interface. The de-compressor de-compresses a compressed multimedia data derived from an input bitstream. The input interface receives the input bitstream via a camera interface, parses indication information included in the input bitstream, and configures the de-compressor to employ a de-compression algorithm as indicated by the indication information.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. provisional application No. 61/729,426, filed on Nov. 23, 2012 and incorporated herein by reference.

BACKGROUND

The disclosed embodiments of the present invention relate to transmitting and receiving multimedia data over a camera interface, and more particularly, to a data processing apparatus for transmitting/receiving compression-related indication information via a camera interface and related data processing method.

A camera interface is disposed between a first chip and a second chip to transmit multimedia data from the first chip to the second chip for further processing. For example, the first chip may include a camera module, and the second chip may include an image signal processor (ISP). The multimedia data may include image data (i.e., a single still image) or video data (i.e., a video sequence composed of images). When a camera sensor with a higher resolution is employed in the camera module, the multimedia data transmitted over the camera interface would have a larger data size/data rate, which increases the power consumption of the camera interface inevitably. If the camera module and the ISP are both located at a portable device (e.g., a smartphone) powered by a battery device, the battery life is shortened due to the increased power consumption of the camera interface. Thus, there is a need for an innovative design which can effectively reduce the power consumption of the camera interface.

SUMMARY

In accordance with exemplary embodiments of the present invention, a data processing apparatus for transmitting/receiving compression-related indication information via a camera interface and related data processing method are proposed.

According to a first aspect of the present invention, an exemplary data processing apparatus is disclosed. The exemplary data processing apparatus has a compressor and an output interface. The compressor is arranged for generating a compressed multimedia data by compressing a multimedia data according to a compression algorithm. The output interface is arranged for recording indication information in an output bitstream and outputting the output bitstream via a camera interface, wherein the output bitstream is derived from the compressed multimedia data, and the indication information is set in response to the compression algorithm employed by the compressor.

According to a second aspect of the present invention, an exemplary data processing apparatus is disclosed. The exemplary data processing apparatus has a de-compressor and an input interface. The de-compressor is arranged for de-compressing a compressed multimedia data derived from an input bitstream. The input interface is arranged for receiving the input bitstream via a camera interface, parsing indication information included in the input bitstream, and configuring the de-compressor to employ a de-compression algorithm as indicated by the indication information.

According to a third aspect of the present invention, an exemplary data processing method is disclosed. The exemplary data processing method includes at least the following steps: generating a compressed multimedia data by compressing a multimedia data according to a compression algorithm; and recording indication information in an output bitstream and outputting the output bitstream via a camera interface, wherein the output bitstream is derived from the compressed multimedia data, and the indication information is set in response to the compression algorithm.

According to a fourth aspect of the present invention, an exemplary data processing method is disclosed. The exemplary data processing method includes at least the following steps: de-compressing a compressed multimedia data derived from an input bitstream; and receiving the input bitstream via a camera interface, parsing indication information included in the input bitstream, and configuring the de-compressing step to employ a de-compression algorithm as indicated by the indication information.

These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a data processing system according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a data structure of the output bitstream generated from the camera module to the ISP according to an embodiment of the present invention.

FIG. 3 is a diagram illustrating an example of information handshaking between the camera module and the ISP.

FIG. 4 is a flowchart illustrating a control and data flow of the data processing system shown in FIG. 1 when interface compression is enabled.

FIG. 5 is a block diagram illustrating a data processing system according to another embodiment of the present invention.

FIG. 6 is a diagram illustrating a data structure of the output bitstream generated from the camera module to the ISP according to another embodiment of the present invention.

FIG. 7 is a flowchart illustrating a control and data flow of the data processing system shown in FIG. 5 when interface compression is enabled.

DETAILED DESCRIPTION

Certain terms are used throughout the description and following claims to refer to particular components. As one skilled in the art will appreciate, manufacturers may refer to a component by different names. This document does not intend to distinguish between components that differ in name but not function. In the following description and in the claims, the terms “include” and “comprise” are used in an open-ended fashion, and thus should be interpreted to mean “include, but not limited to . . . ”. Also, the term “couple” is intended to mean either an indirect or direct electrical connection. Accordingly, if one device is coupled to another device, that connection may be through a direct electrical connection, or through an indirect electrical connection via other devices and connections.

The present invention proposes applying data compression to a multimedia data and then transmitting a compressed multimedia data over a camera interface. As the data size/data rate of the compressed multimedia data is smaller than that of the original un-compressed multimedia data, the power consumption of the camera interface is reduced correspondingly. Besides, the de-compression algorithm employed by a receiving end which receives and de-compresses a compressed multimedia data is required to be compliant with the compression algorithm employed by a transmitting end which generates and transmits the compressed multimedia data; otherwise, the receiving end fails to correctly de-compress the compressed multimedia data. Therefore, the present invention further proposes transmitting/receiving the compression-related indication information via the camera interface, such that the de-compression algorithm of the receiving end is properly configured based on the compression-related indication information. Further details will be described as below.

FIG. 1 is a block diagram illustrating a data processing system according to an embodiment of the present invention. The data processing system 100 includes a plurality of data processing apparatuses such as a camera module 102 and an image signal processor (ISP) 104. The camera module 102 and the ISP 104 may be different chips, and the camera module 102 communicates with the ISP 104 via a camera interface 103. In this embodiment, the camera interface 103 may be a camera serial interface (CSI) standardized by a Mobile Industry Processor Interface (MIPI).

The camera module 102 is coupled to the camera interface 103, and supports un-compressed data transmission and compressed data transmission. When the camera module 102 is used to transmit un-compressed data to the ISP 104, the camera module 102 generates the un-compressed multimedia data D1 according to an input multimedia data derived from capturing the incident light, and transmits the un-compressed multimedia data D1 over the camera interface 103. When the camera module 102 is used to transmit compressed data to the ISP 104, the camera module 102 generates a compressed multimedia data D1′ according to the input multimedia data derived from capturing the incident light, and transmits the compressed multimedia data D1′ over the camera interface 103. By way of example, but not limitation, the input multimedia data obtained by capturing the incident light and used for generating the un-compressed multimedia data D1/compressed multimedia data D1′ may be an image data or a video data.

As shown in FIG. 1, the camera module 102 includes a camera controller 112, an output interface 114 and a processing circuit 116. The processing circuit 116 includes circuit elements required for processing the input multimedia data to generate the un-compressed data D1 or the compressed data D1′. For example, the processing circuit 116 has a compressor 117 and other circuitry 118, where the other circuitry 118 may have a lens module, a camera sensor, a camera buffer, multiplexer(s), etc. The lens module guides the incident light to the camera sensor, and the camera sensor obtains the input multimedia data correspondingly. The input multimedia data obtained by the camera sensor may be a single captured image or a video sequence composed of a plurality of captured images. Besides, the input multimedia data obtained by the camera sensor may be single view data for 2D display or multiple view data for 3D display. The camera buffer is arranged to buffer the input multimedia data obtained by the camera sensor. The compressor 117 performs data compression. The multiplexer receives the un-compressed multimedia data D1 and the compressed multimedia data D1′, and selectively outputs the un-compressed multimedia data D1 or the compressed multimedia data D1′ according to the operation mode of the camera module 102. For example, the camera controller 112 controls the operation of the camera module 102. When the camera module 102 is operated under a compression mode, the multiplexer outputs the compressed multimedia data D1′; and when the camera module 102 is operated under a non-compression mode, the multiplexer outputs the un-compressed multimedia data D1. As the present invention focuses on the handshaking mechanism between the camera module 102 and the ISP 104, further description of the other circuit 118 is omitted here for brevity.

The output interface 114 is arranged for packing/packetizing the un-compressed multimedia data D1/compressed multimedia data D1′ into an output bitstream according to the transmission protocol of the camera interface 103, and transmits the output bitstream to the ISP 104 via the camera interface 103. When the compression mode of the camera module 102 is enabled, the compressor 117 further sets indication information INFO in response to the compression algorithm employed. Besides, the output interface 114 further records the indication information INFO in the output bitstream. In this way, when the output bitstream is transmitted from the camera module 102 to the ISP 104, the compression-related indication information (e.g., the indication information INFO) is also transmitted via the camera interface 103.

Regarding the ISP 104, it communicates with the camera module 102 via the camera interface 103. In this embodiment, the ISP 104 is coupled to the camera interface 103, and supports un-compressed data reception and compressed data reception. When the camera module 102 transmits the un-compressed data D1 to the ISP 104, the ISP 104 is operated under a non-decompression mode to receive an un-compressed data D2 from the camera interface 103, generate a processed multimedia data according to the un-compressed data D2, and send the processed multimedia data to one or multiple processors, such as a display processor 106_1, a video processor 106_2, and/or a graphic processor 106_3, for further processing. When the camera module 102 transmits the compressed data D1′ to the ISP 104, the ISP 104 is operated under a de-compression mode to receive a compressed multimedia data D2′ from the camera interface 103, generate a processed multimedia data according to the compressed data D2′, and send the processed multimedia data to one or multiple processors, such as the display processor 106_1, the video processor 106_2, and/or the graphic processor 106_3, for further processing. If there is no error introduced during the data transmission, the un-compressed data D1 transmitted under the non-compression mode should be identical to the un-compressed data D2 received under the non-decompression mode, and the compressed data D1′ transmitted under the compression mode should be identical to the compressed data D2′ received under the de-compression mode.

As shown in FIG. 1, the ISP 104 includes an ISP controller 122, an input interface 124 and a processing circuit 126. The processing circuit 126 may include circuit elements required for generating the processed multimedia data according to an internal transmission mode (e.g., an on-the-fly mode or an off-line mode). For example, the other circuitry 128 may have a write direct memory access (DMA) controller, a read DMA controller, an image processor, multiplexers, etc. The de-compressor 127 generates the de-compressed multimedia data by de-compressing the compressed multimedia data D2′ according to a de-compression algorithm, where the compressed multimedia data D2′ is derived from an input bitstream received from the camera interface 103. The write DMA controller is arranged for storing a multimedia data into an external image buffer of the ISP 104, where the multimedia data may be an un-compressed multimedia data, a compressed multimedia data or a de-compressed multimedia data. The read write DMA controller is arranged for reading a buffered multimedia data from the external image buffer of the ISP 104. The image processor is arranged to support several image processing operations, including resizing/scaling, rotation, quality enhancement, etc., where the processed multimedia data is generated from the image processor. The multiplexers control interconnections of the de-compressor 127, the image processor, the write DMA controller, and the read DMA controller. As the present invention focuses on the handshaking mechanism between the camera module 102 and the ISP 104, further description of the other circuitry 128 is omitted here for brevity.

The input interface 124 is arranged for receiving the input bitstream from the camera interface 103, and un-packing/un-packetizing the input bitstream into un-compressed multimedia data D2/compressed multimedia data D2′ according to the transmission protocol of the camera interface 103. When the de-compression mode of the ISP 104 is enabled, the input interface 124 is further arranged for parsing the indication information INFO included in the input bitstream, and configuring the de-compressor 127 to employ a de-compression algorithm as indicated by the indication information INFO. In addition to the compressed data transmission between the camera module 102 and the ISP 104, information handshaking is realized through the camera interface 103. Therefore, with the help of the indication information INFO supplied from the camera module 102, the de-compressor 127 of the ISP 104 is properly configured to employ an adequate de-compression algorithm needed for de-compressing the compressed data D2′ correctly.

In one exemplary design, the output interface 114 records the indication information INFO by setting a command set in a payload portion of the output bitstream transmitted over the camera interface 103, and the input interface 124 obtains the indication information INFO by parsing a command set in a payload portion of the input bitstream received from the camera interface 103. Please refer to FIG. 2, which is a diagram illustrating a data structure of the output bitstream generated from the camera module 102 to the ISP 104 according to an embodiment of the present invention. The information handshaking between the camera module 102 and the ISP 104 may be realized by defining a set of commands in the transmitted payload. For example, these commands can be specified in vendor-specific commands based on MIPI camera command set (CCS) specification, where each command in a command set may be an 8-bit code, and the command set can be used to communicate between the camera module 102 and the ISP 104 about the compression capability and compression/de-compression algorithms they support. Please refer to FIG. 3, which is a diagram illustrating an example of information handshaking between the camera module 102 and the ISP 104. In this example, the compressor 117 supports four compression algorithms, and the de-compressor 127 supports four de-compression algorithms compliant with the compression algorithms, respectively. The different compression algorithms may handle different compression ratios, different compression unit sizes, different color formats (e.g. Bayer or RGBW), and/or different bit depths (8-bit, 10-bit or 12-bit). Hence, the indication information INFO is adjusted when at least one of a compression ratio, a compression unit size, a color format, and a bit depth of a compression algorithm employed is changed. The camera module 102 may check a de-compression capability of the ISP 104 by sending a request to the ISP 104 through the camera interface 103, and the ISP 104 may inform the camera module 102 of its de-compression capability by sending a response to the camera module 102 through the camera interface 103. In this way, the camera module 102 can detect whether the ISP 104 has the ability of performing data de-compression, and can further detect what kinds of de-compression algorithms the ISP 104 supports if the ISP 104 is equipped with the de-compression capability. In this example, the compressor 117 at the camera module 102 is configured to use the compression algorithm #4, and the indication information INFO is set correspondingly. The indication information INFO is carried by the command set transmitted from the camera module 102 to the ISP 104 via the camera interface 103. Hence, the ISP 104 receives the indication information INFO through the camera interface 103, and configures the de-compressor 127 to employ the de-compression algorithm #4 as indicated by the indication information INFO.

FIG. 4 is a flowchart illustrating a control and data flow of the data processing system 100 shown in FIG. 1 when interface compression is enabled. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown in FIG. 4 The exemplary control and data flow may be briefly summarized by following steps.

Step 400: Start.

Step 402: Configure the compressor 117 to employ a compression algorithm.

Step 404: Set the indication information INFO in response to the compression algorithm employed by the compressor 117.

Step 406: Record the indication information INFO in an output bitstream. For example, the indication information INFO is recorded by setting a command set in a payload portion of the output bitstream.

Step 408: Transmit the output bitstream over the camera interface 103.

Step 410: Receive an input bitstream from the camera interface 103.

Step 412: Parse the indication information INFO included in the input bitstream. For example, the indication information INFO is obtained by parsing a command set in a payload portion of the input bitstream.

Step 414: Configure the de-compressor 127 to employ a de-compression algorithm as indicated by the indication information INFO.

Step 416: End.

It should be noted that steps 402-408 are performed by the camera module 102, and steps 410-414 are performed by the ISP 104. As a person skilled in the art can readily understand details of each step shown in FIG. 4 after reading above paragraphs, further description is omitted here for brevity.

It should be noted that the same handshaking mechanism may be applied to a data processing system equipped with virtual channel capability. Please refer to FIG. 5 in conjunction with FIG. 1. FIG. 5 is a block diagram illustrating a data processing system according to another embodiment of the present invention. The data processing system 500 includes a channel detector 502, a channel controller 504, and a plurality of data processing apparatuses such as the aforementioned camera module 102 and ISP 104 shown in FIG. 1. The output stream generated from the camera module 102 may be composed of interleaved data streams with different data formats. For example, the camera module 102 may have multiple camera sensors and support different compression algorithms. Hence, a first data stream included in the output stream is generated based on a first compression algorithm and an input multimedia data obtained by a first camera sensor using a first color format (e.g., Bayer); and a second data stream included in the output stream is generated based on a second compression algorithm and an input multimedia data obtained by a second camera sensor using a second color format (e.g., RGBW). To make a receiving end able to distinguish between the first data stream and the second stream interleaved in the output stream, the camera module 102 assigns different channel identification (ID) numbers to the first stream and the second stream.

The virtual channel is to provide separate channels (e.g., 506_1 and 506_2) for different data flows that are interleaved in the data stream transmitted over the camera interface 103. Thus, the receiving end will monitor the channel ID numbers and de-multiplex the interleaved data streams (e.g., the aforementioned first data stream and second data stream) to their appropriate channels 506_1, 506_2. More specifically, the channel controller 504 is arranged to direct data streams to individual channels 506_1, 506_2 under the control of the channel detector 502. In this embodiment, the output interface 114 of the camera module 102 is further arranged for recording a channel ID number ID_CH in a header portion of the output bitstream to indicate which one of the channels 506_1, 506_2 should be connected for receiving the output bitstream. FIG. 6 is a diagram illustrating a data structure of the output bitstream generated from the camera module 102 to the ISP 104 according to another embodiment of the present invention. As mentioned above, the information handshaking between the camera module 102 and the ISP 104 is realized by defining a set of commands in the transmitted payload. In this embodiment, the virtual channel capability of the data processing system 500 is realized by defining the channel ID number in the transmitted header. For example, the channel 506_1 is connected for data transmission/reception when ID_CH=#1, and the channel 506_2 is connected for data transmission/reception when ID_CH=#2.

The channel detector 502 is implemented for receiving an input bitstream from the camera interface 103, and identifying the channel ID number ID_CH from the header portion of the input bitstream. Next, the channel controller 504 dispatches the input bitstream, including the header portion and the payload portion shown in FIG. 6, to a selected channel with the channel ID number ID_CH. The information handshaking as illustrated in FIG. 3 for configuring the compression and de-compression algorithms is also performed between the camera module 102 and the ISP 104. Further description is omitted here for brevity.

FIG. 7 is a flowchart illustrating a control and data flow of the data processing system 500 shown in FIG. 5 when interface compression is enabled. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown in FIG. 7. The exemplary control and data flow may be briefly summarized by following steps.

Step 700: Start.

Step 702: Configure the compressor 117 to employ a compression algorithm.

Step 704: Set the indication information INFO in response to the compression algorithm employed by the compressor 117.

Step 706: Set the channel ID number ID_CH indicative of which channel should be connected for data transmission.

Step 708: Record the channel ID number ID_CH and the indication information INFO in an output bitstream. For example, the indication information INFO is recorded by setting a command set in a payload portion of the output bitstream, and the channel ID number ID_CH is recorded by setting a header portion of the output bitstream.

Step 710: Transmit the output bitstream over the camera interface 103.

Step 712: Get the channel ID number ID_CH from an input bitstream. For example, the channel ID number ID_CH is obtained by parsing a header portion of the input bitstream.

Step 714: Dispatch the input bitstream to a selected channel with the channel ID number ID_CH.

Step 716: Receive the input bitstream dispatched from the channel controller 504.

Step 718: Parse the indication information INFO included in the input bitstream. For example, the indication information INFO is obtained by parsing a command set in a payload portion of the input bitstream.

Step 720: Configure the de-compressor 127 to employ a de-compression algorithm as indicated by the indication information INFO.

Step 722: End.

It should be noted that steps 702-710 are performed by the camera module 102, step 712 is performed by the channel detector 502, step 714 is performed by the channel controller 504, and steps 716-720 are performed by the ISP 104. As a person skilled in the art can readily understand details of each step shown in FIG. 7 after reading above paragraphs, further description is omitted here for brevity.

Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.

Claims

1. A data processing apparatus comprising:

a compressor, arranged for generating a compressed multimedia data by compressing a multimedia data according to a compression algorithm; and

an output interface, arranged for recording indication information in an output bitstream and outputting the output bitstream via a camera interface, wherein the output bitstream is derived from the compressed multimedia data, and the indication information is set in response to the compression algorithm employed by the compressor.

2. The data processing apparatus of claim 1, wherein the camera interface is a camera serial interface (CSI) standardized by a Mobile Industry Processor Interface (MIPI).

3. The data processing apparatus of claim 1, wherein the output interface records the indication information by setting a command set in a payload portion of the output bitstream.

4. The data processing apparatus of claim 3, wherein a first end of the camera interface is coupled to the output interface of the data processing apparatus, and a second end of the camera interface is allowed to be coupled to another data processing apparatus via any of a plurality of channels; the output interface further records a channel identification (ID) number in a header portion of the output bitstream to indicate which one of the channels is connected for receiving the output bitstream.

5. A data processing apparatus comprising:

a de-compressor, arranged for de-compressing a compressed multimedia data derived from an input bitstream; and

an input interface, arranged for receiving the input bitstream via a camera interface, parsing indication information included in the input bitstream, and configuring the de-compressor to employ a de-compression algorithm as indicated by the indication information.

6. The data processing apparatus of claim 5, wherein the camera interface is a camera serial interface (CSI) standardized by a Mobile Industry Processor Interface (MIPI).

7. The data processing apparatus of claim 5, wherein the input interface obtains the indication information by parsing a command set in a payload portion of the input bitstream.

8. The data processing apparatus of claim 7, wherein one end of the camera interface is allowed to be coupled to the data processing apparatus via any of a plurality of channels; and a specific channel selected from the channels according to a channel identification (ID) number recorded in a header portion of the input bitstream is connected for receiving the input bitstream.

9. A data processing method comprising:

generating a compressed multimedia data by compressing a multimedia data according to a compression algorithm; and

recording indication information in an output bitstream and outputting the output bitstream via a camera interface, wherein the output bitstream is derived from the compressed multimedia data, and the indication information is set in response to the compression algorithm.

10. The data processing method of claim 9, wherein the camera interface is a camera serial interface (CSI) standardized by a Mobile Industry Processor Interface (MIPI).

11. The data processing method of claim 9, wherein the step of recording the indication information in the output bitstream comprises:

recording the indication information by setting a command set in a payload portion of the output bitstream.

12. The data processing method of claim 11, wherein a first end of the camera interface is coupled to the output bitstream, and a second end of the camera interface is allowed to be coupled to a data processing apparatus via any of a plurality of channels; and the data processing method further comprises:

recording a channel identification (ID) number in a header portion of the output bitstream to indicate which one of the channels is connected for receiving the output bitstream.

13. The data processing method of claim 9, wherein the indication information is adjusted when at least one of a compression ratio, a compression unit size, a color format, and a bit depth of the compression algorithm is changed.

14. A data processing method comprising:

de-compressing a compressed multimedia data derived from an input bitstream; and

receiving the input bitstream via a camera interface, parsing indication information included in the input bitstream, and configuring the de-compressing step to employ a de-compression algorithm as indicated by the indication information.

15. The data processing method of claim 14, wherein the camera interface is a camera serial interface (CSI) standardized by a Mobile Industry Processor Interface (MIPI).

16. The data processing method of claim 14, wherein the step of parsing the indication information included in the input bitstream comprises:

obtaining the indication information by parsing a command set in a payload portion of the input bitstream.

17. The data processing method of claim 16, wherein one end of the camera interface is allowed to be coupled to any of a plurality of channels; and the data processing method further comprises:

receiving the input bitstream from the camera interface via a specific channel selected from the channels according to a channel identification (ID) number recorded in a header portion of the input bitstream.

18. The data processing method of claim 14, wherein the indication information is adjusted when at least one of a compression ratio, a compression unit size, a color format, and a bit depth of the compression algorithm is changed.