Codeword assignment for intra chroma mode signalling for HEVC
Intra prediction is used in state-of-the-art video coding standards such as AVC. The intra prediction modes are coded into the bitstream. Luma and chroma components could potentially have different prediction modes. For chroma components, there are 5 different modes defined in AVC: vertical, horizontal, DC, diagonal down right, and “same as luma”. Statistics show that the “same as luma” mode is frequent used, but in AVC, this mode is encoded using more bits than other modes during entropy coding, therefore the coding efficiency is decreased. Accordingly, a modified binarization/codeword assignment for chroma intra mode signaling is able to be utilized for high efficiency video coding (HEVC), the next generation video coding standard.
Latest SONY CORPORATION Patents:
- Methods, terminal device and infrastructure equipment using transmission on a preconfigured uplink resource
- Surface-emitting semiconductor laser
- Display control device and display control method for image capture by changing image capture settings
- Image display device to display a plurality of viewpoint images
- Retransmission of random access message based on control message from a base station
This application is a continuation of U.S. patent application Ser. No. 15/866,542, filed Jan. 10, 2018, which is a continuation of U.S. patent application Ser. No. 14/326,334, filed Jul. 8, 2014, now U.S. Pat. No. 9,955,156, issued Apr. 24, 2018, which is a continuation of U.S. patent application Ser. No. 13/038,157, filed Mar. 1, 2011 (now abandoned), which claims priority under 35 U.S.C. § 119(e) of the U.S. Patent Application Ser. No. 61/432,970, filed Jan. 14, 2011 and U.S. Patent Application Ser. No. 61/432,957, filed Jan. 14, 2011.
FIELD OF THE INVENTIONThe present invention relates to the field of image processing. More specifically, the present invention relates to a modified codeword assignment for intra chroma mode signaling for High Efficiency Video Coding (HEVC).
BACKGROUND OF THE INVENTIONA standard binarization/Variable Length Coding (VLC) codeword assignment scheme used in Advanced Video Coding (AVC) is shown in Table 1. In A VC, the prediction mode for an intra coded block is signaled in the bitstream to the decoder. Possible chroma prediction modes/directions are vertical, horizontal, DC, diagonal and “same as luma.” The modes are each assigned a codeword from O through 4 and are signaled using truncated unary code in the bitstream.
A statistical study shows that there is a high probability for the chroma components to use the sample prediction direction as the luma component of the block. However, the “same as luma” mode is assigned the longest codeword in Table 1 (4 bits), which is not efficient. On the other hand, redundancy also exists between mode 4 and mode 0-3, when the luma prediction mode is one of vertical, horizontal, DC and diagonal down right. Therefore, a better design of the codeword table is needed.
SUMMARY OF THE INVENTIONIntra prediction is used in state-of-the-art video coding standards such as AVC. The intra prediction modes are coded into the bitstream. Luma and chroma components could potentially have different prediction modes. For chroma components, there are 5 different modes defined in AVC: vertical, horizontal, DC, diagonal down right, and “same as luma.” Statistics show that the “same as luma” mode is frequently used, but in AVC, this mode is encoded using more bits than other modes during entropy coding, therefore the coding efficiency is decreased.
Accordingly, a modified binarization/codeword assignment for chroma intra mode signaling is able to be utilized for high efficiency video coding (HEVC), the next generation video coding standard.
In one aspect, a method programmed in a controller of a device comprises programming a codeword assignment and performing intra coding prediction using the codeword assignment. The codeword assignment comprises five modes including vertical, horizontal, DC, diagonal down right and same as luma. A codeword for the same as luma mode includes the fewest bits of the five modes. The method further comprises determining if the mode of the same as luma is the same as one of the other modes, and if the mode of the same as luma is the same as one of the other modes, then reconfiguring the codeword assignment so that the same mode is removed and remaining mode codewords are reduced in bits. The intra coding prediction is used to decode an image. The codeword assignment is programmed in a memory of the device. The codeword assignment includes programming hardware logic. The device is selected from the group consisting of a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular/mobile telephone, a smart appliance, a gaming console, a digital camera, a digital camcorder, a camera phone, an iPod®/iPhone/iPad, a video player, a DVD writer/player, a Blu-Ray® writer/player, a television and a home entertainment system.
In another aspect, an apparatus for implementing a codeword assignment comprises a programming module for programming a codeword assignment wherein the codeword assignment includes a same as luma codeword with a fewest bits of codewords and an intra coding prediction module for performing intra coding prediction using the codeword assignment. The codeword assignment comprises five modes including vertical, horizontal, DC, diagonal down right and the same as luma. The apparatus further comprises a comparison module for determining if a same as luma mode value is equal to another codeword mode value. The apparatus further comprises a reconfiguring module for reconfiguring the codeword assignment, if the luma mode value is equal to another codeword mode value. Reconfiguring the codeword assignment includes removing the same mode and reducing bits in remaining mode codewords. The intra coding prediction is used to decode an image. The codeword assignment is programmed in a memory of the device. The codeword assignment includes programming hardware logic. The apparatus is selected from the group consisting of a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular/mobile telephone, a smart appliance, a gaming console, a digital camera, a digital camcorder, a camera phone, an iPod®/iPhone/iPad, a video player, a DVD writer/player, a Blu-ray® writer/player, a television and a home entertainment system.
In another aspect, an apparatus comprises a memory for storing an application, the application for programming a codeword assignment including five modes including vertical, horizontal, DC, diagonal down right and same as luma, wherein same as luma includes a fewest bits of the modes and performing intra coding prediction using the codeword assignment and a processing component coupled to the memory, the processing component configured for processing the application. The apparatus further comprises determining if the mode of the same as luma is the same as one of the other modes, and if the mode of the same as luma is the same as one of the other modes, then reconfiguring the codeword assignment so that the same mode is removed and remaining mode codewords are reduced in bits. The intra coding prediction is used to decode an image. The apparatus is selected from the group consisting of a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular/mobile telephone, a smart appliance, a gaming console, a digital camera, a digital camcorder, a camera phone, an iPod®/iPhone/iPad, a video player, a DVD writer/player, a Blu-ray® writer/player, a television and a home entertainment system.
Since the texture pattern of luma and chroma components are correlated, often times the mode used for the luma prediction is also good for the chroma prediction. Thus, the “same as luma” mode is likely to be the best mode. Therefore, a redefined binarization/Variable Length Coding (VLC) scheme is shown in Table 2. As shown, the “same as luma” mode has the shortest codeword (only 1 bit) in the improved scheme.
Furthermore, chroma intra modes are not mutually exclusive. For example, if the luma intra mode is vertical, then the “same as luma” is the same as vertical. Therefore, codeword space reduction is able to be used to further improve coding efficiency, such that if the luma mode is one of vertical, horizontal, DC or diagonal down right, the corresponding mode is removed from the chroma mode list, and accordingly, the codeword space of the truncated unary code is reduced from 0 through 4 to 0 through 3. Table 3 shows a complete list of codewords.
In some embodiments, the modified codeword assignment application(s) 430 include several applications and/or modules. Modules include a programming module for programming a modified codeword assignment, an intra coding prediction module for performing the intra coding prediction using the modified codeword assignment, a comparison module for determining if the “same as luma” mode value is equal to one of the mode codewords and a reconfiguring module for reconfiguring the codeword assignment. In some embodiments, modules include one or more sub-modules as well. In some embodiments, fewer or additional modules are able to be included.
Examples of suitable computing devices include a personal computer, a laptop computer, a computer workstation, a server, a mainframe computer, a handheld computer, a personal digital assistant, a cellular/mobile telephone, a smart appliance, a gaming console, a digital camera, a digital camcorder, a camera phone, an iPod®/iPhone/iPad, a video player, a DVD writer/player, a Blu-Ray® writer/player, a television, a home entertainment system or any other suitable computing device.
To utilize the modified codeword assignment, a user acquires a video/image such as on a digital camcorder, and while or after the video is acquired, or when displaying the video, the modified codeword assignment is utilized in the intra coding prediction aspect of encoding the video. The modified codeword assignment is able to be implemented automatically without user involvement.
In operation, the modified codeword assignment enables more efficient prediction. A simple modified codeword assignment is able to be used where “same as luma” has the shortest codeword, or a slightly more complex codeword assignment is able to be used where the codeword assignment change depending on whether “same as luma” mode is the same as one of the other modes. In some embodiments, the luma prediction is checked at the encoder, the decoder or both.
Claims
1. A decoding device, comprising:
- circuitry configured to: execute a debinarization process on same-as-luma intra prediction mode for a chroma component based on a codeword assignment, wherein in the same-as-luma intra prediction mode for the chroma component, an intra prediction mode for the chroma component is same as an intra prediction mode for a luma component corresponding to the chroma component, the intra prediction mode for the chroma component is assigned to one bit in case the intra prediction mode for the chroma component is the same-as-luma intra prediction mode, and the one bit of the same-as-luma intra prediction mode for the chroma component is a shortest codeword among a plurality of codewords associated with a plurality of intra prediction modes for the chroma component.
2. The decoding device according to claim 1, wherein the plurality of intra prediction modes for the chroma component includes five intra prediction modes.
3. The decoding device according to claim 2, wherein the five intra prediction modes include at least one of a vertical mode, a horizontal mode, or a DC mode.
4. The decoding device according to claim 1, wherein the codeword assignment for the same-as-luma intra prediction mode is modified based on a mode value of the same-as-luma intra prediction mode.
5. The decoding device according to claim 4, wherein the circuitry is further configured to execute, based on the modified codeword assignment, an intra coding prediction process for the same-as-luma intra prediction mode.
6. The decoding device according to claim 5, wherein the circuitry is further configured to decode a bit stream based on the execution of the intra coding prediction process.
7. A decoding method, comprising:
- executing a debinarization process on a same-as-luma intra prediction mode for a chroma component based on a codeword assignment, wherein in the same-as-luma intra prediction mode for the chroma component, an intra prediction mode for the chroma component is same as an intra prediction mode for a luma component corresponding to the chroma component, the intra prediction mode for the chroma component is assigned to one bit in case the intra prediction mode for the chroma component is the same-as-luma intra prediction mode, and the one bit of the same-as-luma intra prediction mode for the chroma component is a shortest codeword among a plurality of codewords associated with a plurality of intra prediction modes for the chroma component.
8. The decoding method according to claim 7, wherein the plurality of intra prediction modes for the chroma component includes five intra prediction modes.
9. The decoding method according to claim 8, wherein the five intra prediction modes include at least one of a vertical mode, a horizontal mode, or a DC mode.
10. The decoding method according to claim 7, wherein the codeword assignment for the same-as-luma intra prediction mode is modified based on a mode value of the same-as-luma intra prediction mode.
11. The decoding method according to claim 10, further comprising executing, based on the modified codeword assignment, an intra coding prediction process for the same-as-luma intra prediction mode.
12. The decoding method according to claim 11, further comprising decoding an image based on the execution of the intra coding prediction process.
8755435 | June 17, 2014 | Guo et al. |
8861594 | October 14, 2014 | Kim et al. |
20030048208 | March 13, 2003 | Karczewicz |
20060056518 | March 16, 2006 | Conklin |
20070025441 | February 1, 2007 | Ugur et al. |
20080260028 | October 23, 2008 | Lamy-Bergot |
20080310507 | December 18, 2008 | Ye et al. |
20090175334 | July 9, 2009 | Ye et al. |
20120177112 | July 12, 2012 | Guo et al. |
20130272401 | October 17, 2013 | Seregin et al. |
1784015 | June 2006 | CN |
1812582 | August 2006 | CN |
10-1872663 | June 2018 | KR |
10-1912567 | October 2018 | KR |
- Tabatabai et al. “HM2 Chroma Intra Coding Description,” joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 3rd Meeting: Guangzhou, CN, Oct. 7-15, 2010 Document: JCTVC-E395. (Year: 2010).
- Office Action for KR Patent Application No. 10-2018-0125318, dated Dec. 19, 2018, 06 pages of Office Action and 05 pages of English Translation.
- Yu Liu,“Analysis of Coding Tools in HEVC Test Model (HM 1.0)—Intra Prediction”, H265.net, Witness the development of H.265, Dec. 1, 2010, http://www.h265.netf201 0/12/analysis-of-coding-tools-in-hevc-test, Apr. 3, 2014.
- Chen, et al., “Chroma Intra Prediction by Reconstructed Luma Samples”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, JCTVC-C206, 3rd Meeting: Guangzhou, CN, Oct. 7-15, 2010, 07 pages.
- Wiegand, et al., “WD1: Working Draft 1 of High-Efficiency Video Coding”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, JCTVC-C403, 3rd Meeting: Guangzhou, CN, Oct. 7-15, 2010, 137 pages.
- Office Action for KR Patent Application No. 2017-0034598, dated Apr. 28, 2017, 04 pages of Office Action and 03 pages of English Translation.
- Tabatabai, et al., “HM2 Chroma Intra Coding Description,” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1 /SC29/WG11, 3rd Meeting: Guangzhou, CN, Oct. 7-15, 2010, Document: JCTVC-E395, 03 pages.
- Office Action received for Korean Patent Application No. 10-2013-0066403, dated May 31, 2016, 07 pages of Office Action and 3 pages of English Translation.
- Notice of Allowance for U.S. Appl. No. 14/326,334, dated Dec. 19, 2017, 08 pages.
- Advisory Action for U.S. Appl. No. 14/326,334, dated Jun. 22, 2017, 06 pages.
- Non-Final Rejection for U.S. Appl. No. 14/326,334, dated Sep. 14, 2017, 10 pages.
- Final Rejection for U.S. Appl. No. 14/326,334, dated Mar. 30, 2017, 12 pages.
- Non-Final Rejection for U.S. Appl. No. 14/326,334, dated Aug. 19, 2016, 07 pages.
- Final Rejection for U.S. Appl. No. 13/038,157, dated Aug. 15, 2014, 09 pages.
- Non-Final Rejection for U.S. Appl. No. 13/038,157, dated Apr. 25, 2014, 09 pages.
- Final Rejection for U.S. Appl. No. 13/038,157, dated Oct. 24, 2013, 12 pages.
- Non-Final Rejection for U.S. Appl. No. 13/038,157, dated Aug. 13, 2013, 12 pages.
- Advisory Action for U.S. Appl. No. 13/038,157, dated Dec. 5, 2013, 02 pages.
- Notice of Allowance for U.S. Appl. No. 15/866,542, dated Oct. 15, 2018, 08 pages.
- Non-Final Rejection for U.S. Appl. No. 15/866,542, dated Jun. 13, 2018, 13 pages.
Type: Grant
Filed: Nov 28, 2018
Date of Patent: Oct 29, 2019
Patent Publication Number: 20190098300
Assignee: SONY CORPORATION (Tokyo)
Inventors: Wei Liu (San Jose, CA), Lina Dong (San Jose, CA)
Primary Examiner: Nathnael Aynalem
Application Number: 16/202,686
International Classification: H04N 19/11 (20140101); H04N 19/50 (20140101); H04N 19/176 (20140101); H04N 19/46 (20140101); H04N 19/61 (20140101); H04N 19/593 (20140101); H04N 19/463 (20140101); H04N 19/186 (20140101); H04N 19/23 (20140101); H04N 19/93 (20140101);