METHOD AND APPARATUS FOR PROCESSING THREE DIMENSIONAL VIDEO DATA
A video processing method and apparatus, the video processing method including: decoding two-dimensional video data; adjusting a brightness of the decoded two-dimensional video data; converting the brightness-adjusted two-dimensional video data into three-dimensional video data; and displaying the three-dimensional video data.
Latest Samsung Electronics Patents:
This application claims the benefit of U.S. provisional application No. 61/075,184, filed on Jun. 24, 2008 in the U.S. Patent and Trademark Office and the benefit of Korean Patent Application No. 10-2008-0093371, filed on Sep. 23, 2008 in the Korean Intellectual Property Office, the disclosures of which are incorporated herein in their entirety by reference.
BACKGROUND OF THE INVENTION1. Field of the Invention
Aspects of the present invention relate to a method and an apparatus to process three-dimensional video images, and more particularly, to a method and apparatus to output brightness-adjusted three-dimensional video images.
2. Description of the Related Art
Three-dimensional video technologies are being developed widely nowadays. Three-dimensional video technology provides depth information to two-dimensional images to display realistic three-dimensional images. Thus, three-dimensional video technology is being applied to various fields such as communications, medicine, and broadcasting.
Three-dimensional video technology can be classified into a technology to generate three-dimensional video data from scratch, and a technology to convert two-dimensional video data into three-dimensional video data. Research for these two technologies is being conducted together.
SUMMARY OF THE INVENTIONAspects of the present invention provide a method and apparatus to output brightness-adjusted three-dimensional video images.
According to an aspect of the present invention, there is provided a video processing method including: decoding a current frame of two-dimensional video data; adjusting a brightness of the decoded current frame; converting the brightness-adjusted two-dimensional video data into three-dimensional video data; and displaying the three-dimensional video data.
According to an aspect of the present invention, the adjusting of the brightness may include: adjusting a brightness distribution in a frequency distribution table of a current frame, in which the brightness is divided into 256 grades, from 0 to 255, and a number of pixels for each of the grades constitutes a frequency.
According to an aspect of the present invention, when a brightness level of a pixel having a highest brightness among pixels of the current frame is lower than 255, the adjusting of the brightness distribution may include adjusting the brightness distribution by increasing brightness levels of all of the pixels of the current frame by as much as, or less than, a difference between the brightness level of the pixel having the highest brightness among pixels of the current frame and 255.
According to an aspect of the present invention, when a brightness level of a pixel having a lowest brightness from among pixels of the current frame is N, a brightness level of a pixel having a highest brightness from among the pixels of the current frame is M, and M is less than 255, the adjusting of brightness distribution may include adjusting the brightness distribution so that the brightness level of the pixel having the lowest brightness is N and the brightness level of the pixel having the highest brightness is 255.
According to an aspect of the present invention, the adjusting of the brightness distribution may include: adjusting the brightness distribution so that distances between grades between N and the grade 255 are equal.
According to an aspect of the present invention, when a brightness level of a pixel having a lowest brightness from among pixels of the current frame is N, a brightness level of a pixel having a highest brightness from among pixels of the current frame is M, and K is greater than N and less than M, the adjusting of the brightness distribution may include adjusting the brightness distribution so that a distance between grades between N and K is smaller than a distance between grades between K and M.
According to another aspect of the present invention, there is provided a video processing method including: decoding a current frame of two-dimensional video data; converting the current frame into three-dimensional video data; adjusting the brightness of the three-dimensional video data; and displaying the brightness-adjusted three-dimensional video data.
According to another aspect of the present invention, there is provided a video processing method including: receiving video data; recognizing that the received video data is converted from two-dimensional video data to three-dimensional video data; adjusting a brightness of a current frame of the received video data; and outputting the brightness-adjusted video data.
According to another aspect of the present invention, there is provided a video processing apparatus including: a video data decoder to decode a current frame of two-dimensional video data; a brightness adjuster to adjust the brightness of the current frame; a three-dimensional video converter to convert the current frame to three-dimensional video data; and an output unit to display the three-dimensional video data.
According to still another aspect of the present invention, there is provided a video processing system, including: a video processing apparatus including: a video data decoder to decode a current frame of two-dimensional video data, a brightness adjuster to adjust a brightness of the current frame, a three-dimensional video converter to convert the current frame to three-dimensional video data comprising a left-eye image frame and a right-eye image frame corresponding to the current frame, and an output unit to display the three-dimensional video data and to output a shutter control signal; and shutter glasses including a left-eye lens to receive the left-eye image, a right-eye lens to receive the right-eye image, and a signal receiver to receive the shutter control signal, wherein the left-eye lens closes according to the received shutter control signal when the right-eye lens receives the right-eye image, and the right-eye lens closes according to the received shutter control signal when the left-eye lens receives the left-eye image.
According to yet another aspect of the present invention, there is provided a video processing method of a video processing apparatus, the video processing method including: decoding, by the video processing apparatus, a current frame of two-dimensional video data; adjusting, by the video processing apparatus, a brightness of the decoded current frame; and displaying the brightness-adjusted current frame as three-dimensional video data.
Additional aspects and/or advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
These and/or other aspects and advantages of the invention will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
Reference will now be made in detail to the present embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiments are described below in order to explain the present invention by referring to the figures.
Since the human left eye and right eye are separated from each other in a transverse direction, two-dimensional images seen from the left eye and the right eye are different from each other. The video processing apparatus 110 alternately outputs left eye images and right eye images that are not the same as each other onto a screen using the above principle. The human eye recognizes images as continuous video when the images are seen at a frame rate of 60 Hz or greater. Three-dimensional video is generated by combining images input through the left eye and right eye. Thus, in order for the human eye to recognize the three-dimensional images as continuous video without stopping, the video processing apparatus 110 outputs the images at a frame rate of at least 120 Hz. For example, when the video processing apparatus 110 outputs left eye images and right eye images alternately every 1/120 second, the brain of the viewer combines the different two-dimensional images to generate realistic three-dimensional images having perspective and reality.
When the viewer views the images outputted from the video processing apparatus 110 without any special equipment, the left eye image and the right eye image overlap with each other. To prevent the left eye image and the right eye image from overlapping, the viewer wears the shutter glasses 130. The shutter glasses 130 opens a left lens and closes a right lens, and closes the left lens and opens the right lens alternately in synchronization with the left and right eye images output by the video processing apparatus 110. The IR transmission unit 120 included in the video processing apparatus 110 transmits infrared ray signals to the shutter glasses 130 for the synchronization. The IR receiving unit 140 included in the shutter glasses 130 senses the infrared ray signals transmitted from the IR transmission unit 120, such that the shutter glasses 130 operates in synchronization with the video processing apparatus 110. While described as being infrared, it is understood that the communication between the transmission unit 120 and the glasses 130 can be using other light, radio, or energy frequencies, or can also be done using wired communications.
As described above, the shutter glasses 130 alternately blocks the left eye images and right eye images. However, as a result, the images irradiated to the eyes of the viewer through the shutter glasses 130 are darker than the images that are continuously input into the eyes of the viewer without blocking. In addition, since the shutter glasses 130 are dark, the images irradiated to the eyes of the viewer through the shutter glasses 130 are darker than the images output from the video processing apparatus 110. To prevent this, the video processing apparatus 110 adjusts the brightness of the images before outputting the three-dimensional video images onto the screen.
When the video processing apparatus 110 outputs a left eye image, the shutter glasses 130 opens the left lens and closes the right lens so that the image is irradiated to the eye of the viewer through the left lens.
Since the shutter glasses 130 alternately opens the left lens 130L and the right lens 130R, the left eye and the right eye of the viewer respectively view left eye images and right eye images every 1/120 second. In the shown embodiment, since the video processing apparatus 110 adjusts the brightness of the images before outputting the images in order to compensate for the reduction of brightness while the images pass through the shutter glasses 130, the images irradiated to the eyes of the viewer through the shutter glasses 130 are brighter than the original images, the brightness of which is not adjusted.
In a case where video data is recorded on a disk (such as a DVD, Blu-ray disc, etc.), when the disk is loaded into the video processing apparatus 300, the video data decoder 310 reads the video data from the loaded disk and decodes the read video data. While not required, the image processing apparatus 300 can include a drive to read the disc directly, or can be connected to a separate drive. Moreover, while not required in all embodiments, the video processing apparatus 300 may further include a communication unit (not shown) to communicate with an external server or an external terminal through a communication network and/or any wired/wireless connection (such as universal serial bus, infrared, Bluetooth, etc.). In this case, the video processing apparatus 300 downloads the video data from the external server or the external terminal onto a local storage device (not shown) through the communication unit, and then reads the video data stored in the local storage device and decodes the read video data. However, it is understood that aspects of the present invention are not limited to storing the video data in the local storage. For example, in other aspects, the video processing apparatus 300 may decode the video data transmitted from the external server or the external terminal through the communication unit by using a real-time streaming method. Furthermore, the video processing apparatus 300 may read the video data from any external storage device (such as a flash memory). The video data can be movies, television shows, video games, software interfaces, etc.
In the decoding of the video data, the video data decoder 310 extracts brightness components (or luminance components) Y and chrominance components Cr and Cb from pixels in image frames. The brightness adjuster 320 adjusts a brightness of the image frames using the brightness components Y extracted by the video data decoder 310. The brightness adjuster 320 adjusts the brightness of the image frames by controlling a brightness distribution in a frequency distribution table regarding distribution of luminance components. The frequency distribution table with respect to the brightness components is a histogram showing the brightness on a variance side of a horizontal axis, and the number of pixels in each brightness level on a frequency side of a vertical axis. The brightness is divided into 256 grades from 0 to 255, and the number of pixels corresponding to each level of brightness is denoted as the frequency. Embodiments in which the brightness adjuster 320 adjusts the brightness of the image frame by adjusting the brightness distribution of the frequency distribution table will be described in detail with reference to
The video data input into the video data decoder 310 may be two-dimensional video data or three-dimensional video data. When the video data input into the video data decoder 310 is three-dimensional video data, the brightness adjuster 320 adjusts the brightness of the three-dimensional video data and transmits the brightness-adjusted three-dimensional video data to the output unit 340. When the video data input into the video data decoder 310 is two-dimensional video data, the brightness adjuster 320 adjusts the brightness of the two-dimensional video data and transmits the brightness-adjusted two-dimensional video data to the three-dimensional video converter 330. While shown as having both, it is understood that the apparatus 200 need not have both the capability of converting two dimensional video data to three dimensional video data as well as receiving three-dimensional data as shown.
The three-dimensional video converter 330 generates three-dimensional video data by providing depth to the two-dimensional video data. In this regard, the three-dimensional video converter 330 divides an image frame in the two-dimensional video data into blocks of predetermined sizes, and selects a block, which is the most similar to a predetermined block of the current frame, among blocks of a previous frame. Then, the three-dimensional video converter 330 calculates a motion vector for each of the divided blocks in the current frame, and generates a new frame by combining the selected blocks. However, aspects of the present invention are not limited to the above example. For example, in other aspects, the three-dimensional video converter 330 may generate a depth map using composition information of the image frames and generate a new frame in order to provide depth to the two-dimensional video.
The three-dimensional video converter 330 determines one of the new image frame and the current image frame as the left eye image and the other of the new image frame and the current image frame as the right eye image. Alternatively, the three-dimensional video converter 330 may generate a new left eye image and/or a new right eye image using the new image frame and the current image frame. The three-dimensional video converter 330 generates a three-dimensional format image frame, which includes both the left eye image and the right eye image in a frame, and transmits the generated three-dimensional format image frame to the output unit 340. The output unit 230 outputs the left and right eye images, which are included in the brightness-compensated frame, alternately at least every 1/120 second.
In
As described above, according to aspects of the present invention, the brightness of video images is adjusted before outputting three-dimensional video. Thus, problems caused by dark video images when a viewer watches three-dimensional video using shutter glasses can be overcome.
While not restricted thereto, aspects of the present invention can also be embodied as computer-readable code on a computer-readable recording medium. The computer-readable recording medium is any data storage device that can store data that can be thereafter read by a computer system. Examples of the computer-readable recording medium include read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium can also be distributed over network-coupled computer systems so that the computer-readable code is stored and executed in a distributed fashion. Aspects of the present invention may also be realized as a data signal embodied in a carrier wave and comprising a program readable by a computer and transmittable over the Internet. Moreover, while not required in all aspects, one or more units of the video processing apparatus 300 or 350 can include a processor or microprocessor executing a computer program stored in a computer-readable medium, such as a local storage.
Although a few embodiments of the present invention have been shown and described, it would be appreciated by those skilled in the art that changes may be made in this embodiment without departing from the principles and spirit of the invention, the scope of which is defined in the claims and their equivalents.
Claims
1. A video processing method of a video processing apparatus, the video processing method comprising:
- decoding, by the video processing apparatus, a current frame of two-dimensional video data;
- adjusting, by the video processing apparatus, a brightness of the decoded current frame;
- converting the brightness-adjusted current frame into three-dimensional video data; and
- displaying the converted three-dimensional video data.
2. The video processing method as claimed in claim 1, wherein the adjusting of the brightness comprises:
- adjusting a brightness distribution in a frequency distribution table of the current frame, in which the brightness is divided into 256 grades, from 0 to 255, and a number of pixels of the current frame for each of the grades constitutes a frequency.
3. The video processing method as claimed in claim 2, wherein, when a brightness level of a pixel of the current frame having a highest brightness from among pixels of the current frame is lower than 255, the adjusting of the brightness distribution comprises adjusting the brightness distribution by increasing brightness levels of all of the pixels of the current frame by as much as, or less than, a difference between the brightness level of the pixel having the highest brightness from among the pixels of the current frame and 255 so as to change the brightness of the current frame.
4. The video processing method as claimed in claim 2, wherein, when a brightness level of a pixel having a lowest brightness from among pixels of the current frame is N, a brightness level of a pixel having a highest brightness from among the pixels of the current frame is M, and M is less than 255, the adjusting of the brightness distribution comprises adjusting the brightness distribution so that the brightness level of the pixel having the lowest brightness is N and the brightness level of the pixel having the highest brightness is 255.
5. The video processing method as claimed in claim 4, wherein the adjusting of the brightness distribution so that the brightness level of the pixel having the lower brightness is N and the brightness level of the pixel having the highest brightness is 255 comprises:
- adjusting the brightness distribution so that distances between grades between N and 255 are equal.
6. The video processing method as claimed in claim 2, wherein, when a brightness level of a pixel having a lowest brightness from among pixels of the current frame is N, a brightness level of a pixel having a highest brightness from among the pixels of the current frame is M, and K is greater than N and less than M, the adjusting of the brightness distribution comprises adjusting the brightness distribution so that a distance between grades between N and K is less than a distance between grades between K and M.
7. A computer-readable recording medium encoded with the method of claim 1 and implemented by the image processing apparatus.
8. A video processing method of a video processing apparatus, the video processing method comprising:
- decoding, by the video processing apparatus, a current frame of two-dimensional video data;
- converting, by the video processing apparatus, the current frame of the two-dimensional video data into three-dimensional video data; and
- adjusting a brightness of the converted three-dimensional video data; and
- displaying the brightness-adjusted three-dimensional video data.
9. The video processing method as claimed in claim 8, wherein the adjusting of the brightness comprises:
- adjusting a brightness distribution in a frequency distribution table of the current frame, in which the brightness is divided into 256 grades, from 0 to 255, and a number of pixels of the current frame for each of the grades constitutes a frequency.
10. The video processing method as claimed in claim 9, wherein, when a brightness level of a pixel of the current frame having a highest brightness from among pixels of the current frame is lower than 255, the adjusting of the brightness distribution comprises adjusting the brightness distribution by increasing brightness levels of all of the pixels of the current frame by as much as, or less than, a difference between the brightness level of the pixel having the highest brightness from among the pixels of the current frame and 255 so as to change the brightness of the current frame.
11. The video processing method as claimed in claim 9, wherein, when a brightness level of a pixel having a lowest brightness from among pixels of the current frame is N, a brightness level of a pixel having a highest brightness from among the pixels of the current frame is M, and M is less than 255, the adjusting of the brightness distribution comprises adjusting the brightness distribution so that the brightness level of the pixel having the lowest brightness is N and the brightness level of the pixel having the highest brightness is 255.
12. The video processing method as claimed in claim 11, wherein the adjusting of the brightness distribution so that the brightness level of the pixel having the lowest brightness is N and the brightness level of the pixel having the highest brightness is 255 comprises:
- adjusting the brightness distribution so that distances between grades between N and 255 are equal.
13. The video processing method as claimed in claim 9, wherein, when a brightness level of a pixel having a lowest brightness from among pixels of the current frame is N, a brightness level of a pixel having a highest brightness from among the pixels of the current frame is M, and K is greater than N and less than M, the adjusting of the brightness distribution comprises adjusting the brightness distribution so that a distance between grades between N and K is less than a distance between grades between K and M.
14. A computer-readable recording medium encoded with the method of claim 8 and implemented by the image processing apparatus.
15. A video processing method of a video processing apparatus, the video processing method comprising:
- receiving, by the video processing apparatus, video data;
- recognizing that a current frame of the received video data is converted from two-dimensional video data to three-dimensional video data;
- adjusting, by the video processing apparatus, a brightness of the current frame of the received video data; and
- outputting the brightness-adjusted current frame.
16. The video processing method as claimed in claim 15, wherein the adjusting of the brightness comprises:
- adjusting a brightness distribution in a frequency distribution table of the current frame, in which the brightness is divided into 256 grades, from 0 to 255, and a number of pixels of the current frame for each of the grades constitutes a frequency.
17. The video processing method as claimed in claim 16, wherein, when a brightness level of a pixel of the current frame having a highest brightness from among pixels of the current frame is lower than 255, the adjusting of the brightness distribution comprises adjusting the brightness distribution by increasing brightness levels of all of the pixels of the current frame by as much as, or less than, a difference between the brightness level of the pixel having the highest brightness from among the pixels of the current frame and 255 so as to change the brightness of the current frame.
18. The video processing method as claimed in claim 16, wherein, when a brightness level of a pixel having a lowest brightness from among pixels of the current frame is N, a brightness level of a pixel having a highest brightness from among the pixels of the current frame is M, and M is less than 255, the adjusting of the brightness distribution comprises adjusting the brightness distribution so that the brightness level of the pixel having the lowest brightness is N and the brightness level of the pixel having the highest brightness is 255.
19. The video processing method as claimed in claim 18, wherein the adjusting of the brightness distribution so that the brightness level of the pixel having the lowest brightness is N and the brightness level of the pixel having the highest brightness is 255 comprises:
- adjusting the brightness distribution so that distances between grades between N and 255 are equal.
20. The video processing method as claimed in claim 16, wherein, when a brightness level of a pixel having a lowest brightness from among pixels of the current frame is N, a brightness level of a pixel having a highest brightness from among the pixels of the current frame is M, and K is greater than N and less than M, the adjusting of the brightness distribution comprises adjusting the brightness distribution so that a distance between grades between N and K is less than a distance between grades between K and M.
21. A computer-readable recording medium encoded with the method of claim 15 and implemented by the image processing apparatus.
22. A video processing apparatus comprising:
- a video data decoder to decode a current frame of two-dimensional video data;
- a brightness adjuster to adjust a brightness of the decoded current frame;
- a three-dimensional video converter to convert the decoded current frame to three-dimensional video data; and
- an output unit to output the converted three-dimensional video data having the brightness adjusted current frame to a display.
23. The video processing apparatus as claimed in claim 22, wherein the brightness adjuster adjusts a brightness distribution in a frequency distribution table of the current frame, in which the brightness is divided into 256 grades, from 0 to 255, and a number of pixels of the current frame for each of the grades constitutes a frequency.
24. The video processing apparatus as claimed in claim 23, wherein, when a brightness level of a pixel of the current frame having a highest brightness from among pixels of the current frame is lower than 255, the brightness adjuster adjusts the brightness distribution by increasing brightness levels of all of the pixels of the current frame by as much as, or less than, a difference between the brightness level of the pixel having the highest brightness from among the pixels of the current frame and 255 so as to change the brightness of the current frame.
25. The video processing apparatus as claimed in claim 23, wherein, when a brightness level of a pixel having a lowest brightness from among pixels of the current frame is N, a brightness level of a pixel having a highest brightness from among the pixels of the current frame is M, and M is less than 255, the brightness adjuster adjusts the brightness distribution so that the brightness level of the pixel having the lowest brightness is N and the brightness level of the pixel having the highest brightness is 255.
26. The video processing apparatus as claimed in claim 25, wherein the brightness adjuster adjusts the brightness distribution so that distances between grades between N and 255 are equal.
27. The video processing apparatus as claimed in claim 23, wherein, when a brightness level of a pixel having a lowest brightness from among pixels of the current frame is N, a brightness level of a pixel having a highest brightness from among pixels of the current frame is M, and K is greater than N and less than M, the brightness adjuster adjusts the brightness distribution so that a distance between grades between N and K is less than a distance between grades between K and M.
28. A video processing method of a video processing apparatus, the video processing method comprising:
- decoding, by the video processing apparatus, a current frame of video data;
- adjusting, by the video processing apparatus, a brightness of the decoded current frame; and
- displaying the brightness-adjusted current frame as three-dimensional video data.
29. The video processing method as claimed in claim 28, wherein the decoded current frame is two-dimensional video data.
30. The video processing method as claimed in claim 28, wherein the adjusting of the brightness comprises:
- adjusting a brightness distribution in a frequency distribution table of the current frame, in which the brightness is divided into 256 grades, from 0 to 255, and a number of pixels of the current frame for each of the grades constitutes a frequency.
31. A computer-readable recording medium encoded with the method of claim 28 and implemented by the image processing apparatus.
Type: Application
Filed: Jun 24, 2009
Publication Date: Dec 24, 2009
Applicant: Samsung Electronics Co., Ltd. (Suwon-si)
Inventors: Kil-soo Jung (Osan-si), Hyun-Kwon Chung (Seoul), Dae-Jong Lee (Suwon-si)
Application Number: 12/490,589
International Classification: G09G 5/02 (20060101); H04N 13/00 (20060101); G06T 15/00 (20060101);