Method and system for wireless communication of uncompressed video information
A method and system for communication uncompressed video over a wireless communication medium is provided. Pixels in an uncompressed video frame are grouped into groups of pixels for wireless transmission from a transmitter to a receiver, wherein the neighboring pixels in each group possess (have) high spatial correlation. Upon receiving the pixel groups, the receiving station reconstructs the video frame from the received pixel groups, wherein upon detecting an erroneous pixel in a group, information from one or more neighboring pixels in that pixel group are used in place of the erroneous pixel. Further, FEC encoding is utilized recovering bit-errors.
Latest Samsung Electronics Patents:
- Organic electroluminescence device and heterocyclic compound for organic electroluminescence device
- Video decoding method and apparatus, and video encoding method and apparatus
- Organic light-emitting device
- Security device including physical unclonable function cells, operation method of security device, and operation method of physical unclonable function cell device
- Case for mobile electronic device
The present invention relates to wireless communication and in particular, to wireless communication of video information.
BACKGROUND OF THE INVENTIONWith the proliferation of high quality video, an increasing number of electronic devices (e.g., consumer electronics (CE) devices) utilize high-definition (HD) video. Conventionally, most systems compress HD content, which can be around 1 Gbps (gigabits per second) in bandwidth, to a fraction of its size to allow for transmission between devices. However, with each compression and subsequent decompression of the signal, some data can be lost and the picture quality can be degraded.
Uncompressed video pixels exhibit high spatial correlation, wherein neighboring pixels have almost similar values. This spatial correlation allows reusing a neighboring pixel in place of an erroneously received pixel. However, because sharp edges of a picture can result in a sudden change in neighboring pixel values this assumption of spatial correlation is not always true.
The present invention provides a method of communication uncompressed video information over a wireless communication medium. In one embodiment, pixels in an uncompressed video frame are grouped into logical groups of pixels, such that the neighboring pixels in each logical group possess (have) high spatial correlation.
Grouping pixels includes grouping sub-samples of each set of neighboring pixels into a corresponding macropixel, and logically grouping each set of macropixels with similar component values into a corresponding spatial pixel group. Further, in one example, grouping each set of macropixels includes using the most significant bit (MSB) of each component in each macropixel in determining which adjacent N macropixels have similar MSBs. Such N macropixels are then logically grouped into a spatial pixel group. As such, in this example, all macropixels in a spatial pixel group have the same pair-wise MSBs for components such as luminance and chrominance.
The logical groups of pixels are then transmitted from a transmitting wireless station to a receiving wireless station, over the wireless communication medium. Upon receiving the logical groups, the receiving station attempts reconstruction of a video frame from the received pixels. Upon detecting an erroneous pixel in a logical group, the receiver utilizes information from one or more pixels in that pixel group in place of the erroneous pixel. Each spatial pixel group includes delimiters for identifying boundaries of a received spatial pixel group. Reconstructing the video frame from the received logical groups further includes using a spatially correlated macropixel in the spatial pixel group, which is correctly received, in place of the erroneous pixel in that spatial pixel group.
For bit-error recovery, forward error correction (FEC) information is provided within each spatial pixel group, and transmitted with the spatial pixel group. In one example, FEC information for each spatial pixel group includes placing the MSBs of all macropixels in the spatial pixel group, into a selected macropixel in the spatial pixel group, and performing forward error correction (FEC) for the MSBs in the selected macropixel to obtain FEC information. The FEC information is transmitted in the spatial pixel group. Transmitting the FEC information can further include using the MSB position of macropixels in the spatial pixel groups other than the selected macropixel to store the FEC information for the MSBs stored in the selected macropixel. Reconstructing the video frame further includes the steps of receiving the transmitted FEC information, and using the FEC information to recover errors in the received spatial groups.
These and other features, aspects and advantages of the present invention will become understood with reference to the following description, appended claims and accompanying figures.
In the drawings, like references refer to like elements.
DETAILED DESCRIPTION OF THE INVENTIONThe present invention provides a method and system for wireless communication of uncompressed HD video content. In one embodiment, pixel reuse and forward error correction (FEC) are implemented in wireless communication.
One implementation of the present invention involves transmission of high-quality uncompressed video (e.g., 1080p RGB format) over a wireless channel which provides sufficient bandwidth to support the uncompressed video with limited retransmissions. An example application for transmission of uncompressed HD video signals in a 60 GHz frequency band such as in WirelessHD (WiHD) is described below. WiHD is an industry-led effort to define a wireless digital network interface specification for wireless HD digital signal transmission on the 60 GHz frequency band, (e.g., for CE devices).
Each uncompressed video frame 212 at the transmitter is divided into multiple scan lines. Each scan line includes an integer number of pixels, each pixel including multiple component values (e.g., CY, CB, CR). Each component includes either a color component (chrominance) or a luminance component of a video pixel. Quantization for pixel depth (bits per sample (bitplane)), may be, e.g., 8-bit, 10-bit, 12-bit, or 16-bit values. Considering an 8-bit quantization and 60 frames/second, one second of uncompressed video (e.g., 1080 p RGB format) can be expressed as 60×3×8×1920×1080=2.98 Gbits. This requires a large data rate which places significant constraints on retransmissions, indicating retransmissions should be avoided to ensure continuous playback at the receiver.
A transmitter spatial grouping function 214 implements a spatial grouping process that utilizes spatial correlations between pixels of uncompressed video frame 212, to provide information to a receiver spatial processing function 220 for determining which correctly received pixels to reuse in place of erroneous pixels. This allows the receiver spatial processing function 220 to perform error recovery by selectively reusing correctly received pixels in place of erroneous pixels, in the absence of retransmissions.
In one embodiment, the transmitter spatial grouping function 214 provides the receiver spatial processing function 220 with information indicating neighboring pixels in the video frame 212 that have similar pixel components. Based on such information, the receiver spatial processing function 220 can efficiently determine which pixels to reuse for display in place of an erroneous pixel (i.e., reuse a pixel that is similar in component values CY, CB, CR to an erroneous pixel).
Since each pixel component includes either color information (chrominance) or luminance information, and the human eye is more sensitive to luminance, then less chrominance information can be used for display. This is known as sub-sampling.
Using sub-sampling at the transmitter, adjacent pixels from a macropixel 254 can be reused at the receiver in place of an erroneous pixel. Since uncompressed video has high spatial correlation, in the event of an erroneous received pixel, the receiver 204 can instead selectively use one of the correctly received and decoded, spatially co-located, macropixels 254. Macropixel size depends on the sub-sampling scheme being used (e.g.,
The transmitter spatial grouping function 214 performs spatial grouping of the macropixels into spatial pixel groups. In one implementation, the spatial grouping function 214 uses perceptually important information bits, such as the most significant bits (MSBs), of each pixel component in each macropixel, to determine which adjacent N macropixels have similar values for logical grouping in a spatial pixel group. As a result, all macropixels in a spatial pixel group have the same pair-wise MSBs for luminance and chrominance components (i.e., all CY components have the same MSBs, all CR components have the same MSBs and all CB components have the same MSBs).
The spatial pixel groups are formed at the transmitter 204 by the spatial grouping function 214 for each uncompressed video frame 212. A new spatial pixel group is started at the beginning of a new video frame. The spatial grouping function 214 delimits each spatial pixel group so that the spatial processing function 220 at the receiver 204 can identify different spatial pixel groups.
Specifically, the spatial grouping function 214 indicates the boundary of each spatial pixel group using a delimiter known to both the transmitter and the receiver. The receiver spatial processing function 220 uses the delimiter to identify each spatial pixel group. The delimiters provide the receiver with information for selectively reusing correctly received pixels in place of erroneous pixels.
When the receiver detects an erroneous pixel, the receiver utilizes the first delimiter before, and the first delimiter after the erroneous pixel, to identify the boundaries of the spatial pixel group to which the erroneous pixel belongs (this applies for all cases except for the very first spatial pixel group, which does not have a delimiter at the beginning). The receiver then selectively uses information from the macropixels in the identified spatial pixel group in place of the erroneous pixel while, e.g., playing back the received video.
For example, in
-
- Step 302: Obtain a new frame of sub-sampled video pixels for transmission.
- Step 304: Since for each video frame the first spatial pixel group starts from the beginning of the video frame, it is determined if a current macropixel is at the start of the new frame. If yes, go to step 322 to start a new spatial pixel group, otherwise a spatial pixel group is already active (current).
- Step 306: For each luminance component (CY) of the current macropixel, compare the MSB of the component to the MSB of a corresponding component in the spatial pixel group to determine a match.
- Step 308: If the MSBs match, then go to step 310, otherwise go to step 320.
- Step 310: For each chrominance component (CB) of the current macropixel, compare the MSB of the component to the MSB of a corresponding component in the spatial pixel group.
- Step 312: If the MSBs match, then go to step 314, otherwise go to step 320.
- Step 314: For each chrominance component (CR) of the current macropixel, compare the MSB of the component to the MSB of a corresponding component in the spatial pixel group, to determine a match
- Step 316: If the MSBs match, then go to step 318, otherwise go to step 322.
- Step 318: Add the macropixel to the spatial pixel group. Go to step 324.
- Step 320: Add a robust delimiter to terminate the current spatial pixel group.
- Step 322: Start a new spatial pixel group.
- Step 324: Determine if all macropixels in the current video frame are processed. If yes, go to step 326, otherwise go to step 328 to process next macropixel in the frame.
- Step 326: Add a robust delimiter to terminate the current spatial pixel group. Go to step 302.
- Step 328: Obtain the next macropixel in the frame. Go to step 304 to begin processing the macropixel.
Each video frame with the delimited spatial pixel groups is encoded and wirelessly transmitted as packets from the transmitter 202 to the receiver 204. At the receiver 204, the received packets of video data (video transmissions) are decoded, wherein the spatial processing function 220 performs macropixel processing on the information in the decoded packets to identify delimiters for each spatial pixel group in each video frame.
If the spatial processing function 220 finds that a pixel in a spatial pixel group is erroneous, then one of the correctly received and decoded, spatially co-located, or neighboring macropixels in that spatial pixel group are selectively used in place of the erroneous pixel, to regenerate an uncompressed video frame 224. Such selective reuse of pixels at the receiver reduces the need for retransmission of pixels from the transmitter, while maintaining quality for the received video information.
-
- Step 352: Start macropixel processing.
- Step 354: In each macropixel, check if the MSBs of CY and CB and CR components match with the very first macropixel in the spatial pixel group. If yes, go to step 358, otherwise go to step 356.
- Step 356: Reuse one of the neighboring pixel for display in place of an erroneous pixel in the spatial pixel group.
- Step 358: End macropixel processing.
A packet structure is used for data transmission between wireless stations. A communication module 209 in the transmission station 202 (
Referring back to
The FEC function 216 is applied within a spatial pixel group 256 (
FEC encoding normally requires redundant data, hence, the total data length is increased. In one implementation of the present invention, one or more MSB positions are used to include FEC data. Therefore, FEC is included without increasing the video data packet length. In another example, video data packet length is reduced by withholding certain MSBs from transmission.
Then, the MSBs in the first macropixel 254-1 of the spatial pixel group 256B are protected using FEC. This is achieved by: determining FEC data for the MSBs in the first macropixel 254-1 of the spatial pixel group 256B, and using the MSB position of all other macropixels 254-2, . . . , 254-M in the spatial pixel group 256B to carry said FEC data for the MSBs in said first macropixel 254-1. As the spatial pixel group 256B has M macropixels 254 (4:2:2 sub-sampling), and each macropixel includes four pixel components, then K*O*4 bits are available (K represents the total number of MSBs in the spatial pixel group 256B and O represents the MSBs in each pixel component) for the FEC over the MSB bits, when all macropixels except the first macropixel do not carry the MSBs.
Referring to
As is known to those skilled in the art, the aforementioned example architectures described above, according to the present invention, can be implemented in many ways, such as program instructions for execution by a processor, as logic circuits, as an application specific integrated circuit, as firmware, etc. The present invention has been described in considerable detail with reference to certain preferred versions thereof; however, other versions are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the preferred versions contained herein.
Claims
1. A method of wireless communication of video information, comprising the steps of:
- grouping pixels in an uncompressed video frame into logical groups of pixels, such that the neighboring pixels in each logical group posses high spatial correlation, including the steps of: grouping sub-samples of components of a set of neighboring pixels into a macropixel; logically grouping macropixels with similar component values into a spatial pixel group; and transmitting each spatial pixel group over a wireless communication medium.
2. The method of claim 1 further including:
- receiving the transmitted spatial pixel groups;
- for an erroneous pixel in a received spatial pixel group, utilizing component values from one or more selected neighboring pixels in that spatial pixel group, in place of the erroneous pixel.
3. The method of claim 1 further including:
- reconstructing a video frame from the received spatial pixel groups, and upon detecting an erroneous pixel in a spatial pixel group, utilizing component values from one or more selected neighboring pixels in that spatial pixel group in place of the erroneous pixel.
4. The method of claim 1 wherein grouping each set of macropixels with similar component values into a corresponding spatial pixel group further includes:
- using one or more perceptually important bits of each component in each macropixel in determining which adjacent N macropixels have similar corresponding perceptually important bits; and
- logically grouping said N macropixels into a spatial pixel group.
5. The method of claim 1 wherein grouping each set of macropixels with similar component values into a corresponding spatial pixel group further includes:
- using one or more most significant bits of each component in each macropixel in determining which adjacent N macropixels have similar most significant bits; and
- logically grouping said N macropixels into a spatial pixel group.
6. The method of claim 5 wherein all macropixels in a spatial pixel group have the same pair-wise most significant bits for luminance and chrominance components.
7. The method of claim 6 wherein grouping each set of macropixels with similar component values into a spatial pixel group further includes the step of designating each spatial pixel group with delimiters that indicate the boundaries of each spatial pixel group.
8. The method of claim 2 wherein utilizing component values from one or more selected neighboring pixels in that spatial pixel group in place of an erroneous pixel further includes:
- using component values of a correctly received spatially correlated macropixel in the spatial pixel group, in place of the erroneous pixel in that spatial pixel group.
9. The method of claim 8 further including identifying the boundaries of the spatial pixel group based on said delimiters.
10. The method of claim 1 further including:
- performing forward error correction (FEC) within each spatial pixel group to obtain FEC data; and
- transmitting the FEC data along with the spatial pixel group.
11. The method of claim 10 wherein transmitting the FEC data further includes:
- replacing one or more perceptually important bits in a spatial pixel group with FEC data to protect the remaining perceptually important bits in the spatial pixel group.
12. The method of claim 11 wherein transmitting the FEC data further includes:
- replacing one or more most significant bits in a spatial pixel group with FEC data to protect the remaining most significant bits in the spatial pixel group.
13. The method of claim 10 wherein transmitting the FEC data for each spatial pixel group further includes:
- placing the MSBs of all macropixels in the spatial pixel group into a selected macropixel in the spatial pixel group;
- performing FEC for the MSBs in the selected macropixel to obtain FEC data; and
- transmitting the FEC data in the spatial pixel group.
14. The method of claim 13 wherein transmitting the FEC data in the spatial pixel group further includes:
- using the MSB position of macropixels in the spatial pixel group other than a selected macropixel to store the FEC data for the MSBs in the selected macropixel for transmitting the FEC data in the spatial pixel group.
15. The method of claim 10 further including:
- receiving the transmitted spatial pixel groups and the FEC data; and
- using the FEC data to recover errors in the received spatial groups.
16. The method of claim 11 further include using one or more perceptually important bit positions to include FEC data, such that FEC data is included in packet transmission while maintaining data packet length.
17. The method of claim 11 further including withholding certain spatially important bits from transmission.
18. The method of claim 1 wherein transmitting each spatial pixel group includes transmitting each spatial pixel group over a 60 GHz frequency band wireless channel.
19. The method of claim 18 wherein the video information comprises uncompressed high-definition video information.
20. A wireless station for communication of video information, comprising:
- a spatial grouper configured for grouping pixels in an uncompressed video frame into logical groups of pixels, by: grouping sub-samples of components of a set of neighboring pixels into a macropixel; and logically grouping macropixels with similar component values into a spatial pixel group;
- a communication module configured for transmitting each spatial pixel group over a wireless communication medium.
21. The wireless station of claim 20 wherein the spatial grouper is further configured for using a perceptually important bit of each component in each macropixel in determining which adjacent N macropixels have similar corresponding perceptually important bits, and logically grouping said N macropixels into a spatial pixel group.
22. The wireless station of claim 20 wherein the spatial grouper is further configured for using one or more most significant bits of each component in each macropixel in determining which adjacent N macropixels have similar most significant bits, and logically grouping said N macropixels into a spatial pixel group.
23. The wireless station of claim 22 wherein all macropixels in a spatial pixel group have the same pair-wise most significant bits for luminance and chrominance components.
24. The wireless station of claim 23 wherein the spatial grouper is further configured for designating each spatial pixel group with delimiters that indicate the boundaries of each spatial pixel group.
25. The wireless station of claim 20 further including an encoder configured for performing forward error correction (FEC) encoding within each spatial pixel group to obtain the FEC data, wherein the communication module is configured for transmitting the FEC data along with the spatial pixel group.
26. The wireless station of claim 25 wherein one or more perceptually important bits in a spatial pixel group with FEC data to protect the remaining perceptually important bits in the spatial pixel group.
27. The wireless station of claim 26 wherein one or more most significant bits in a spatial pixel group are replaced with FEC data to protect the remaining most significant bits in the spatial pixel group.
28. The wireless station of claim 25 wherein the MSBs of all macropixels in a spatial pixel group are placed into a selected macropixel in the spatial pixel group, and forward error correction (FEC) is performed for the MSBs in the selected macropixel to obtain the FEC data, such that the FEC data is transmitted in the spatial pixel group.
29. The wireless station of claim 26 wherein one or more perceptually important bit positions are to include FEC data, such that FEC data is included in packet transmission while maintaining data packet length.
30. The wireless station of claim 26 wherein certain spatially important bits are withheld from transmission.
31. The wireless station of claim 20 wherein the communication module is configured for transmitting each spatial pixel group over a 60 GHz frequency band wireless channel.
32. The wireless station of claim 31 wherein the video information comprises uncompressed high-definition video information.
33. A wireless station for communication of video information, comprising:
- a communication module for receiving packets of spatial pixel groups over a wireless communication medium, each spatial group formed by grouping sub-samples of components of a set of neighboring pixels of an uncompressed video frame into a macropixel, and logically grouping macropixels with similar component values into a spatial pixel group; and
- a spatial processor configured for detecting an erroneous pixel in a received spatial pixel group, and utilizing component values from one or more selected neighboring pixels in that spatial pixel group, in place of the erroneous pixel.
34. The wireless station of claim 33 wherein the spatial processor is further configured for reconstructing a video frame from the received spatial pixel groups, and upon detecting an erroneous pixel in a spatial pixel group, utilizing component values from one or more selected neighboring pixels in that spatial pixel group in place of the erroneous pixel.
35. The wireless station of claim 33 wherein the spatial processor is further configured for using component values of a correctly received spatially correlated macropixel in the spatial pixel group in place of the erroneous pixel in that spatial pixel group.
36. The wireless station of claim 33 wherein the spatial processor is configured for identifying boundaries of the spatial pixel group based on delimiters therein.
37. The wireless station of claim 33 further including:
- a decoder configured for performing forward error correction (FEC) decoding on received spatial pixel groups with FEC data;
- such that the spatial processor is further configured for using the FEC data to recover errors in the received spatial groups.
38. The wireless station of claim 33 wherein the communication module is configured for receiving each spatial pixel group over a 60 GHz frequency band wireless channel.
39. The wireless station of claim 38 wherein the video information comprises uncompressed high-definition video information.
Type: Application
Filed: Aug 29, 2007
Publication Date: Mar 5, 2009
Patent Grant number: 8243823
Applicant: Samsung Electronics Co., Ltd. (Suwon City)
Inventors: Harkirat Singh (Santa Clara, CA), Huai-Rong Shao (San Jose, CA), Chiu Ngo (San Francisco, CA)
Application Number: 11/897,087
International Classification: H03M 13/47 (20060101); H04N 7/26 (20060101);