METHOD AND SYSTEM FOR HANDLING MULTIPLE 3-D VIDEO FORMATS

Aspects of a method and system for handling multiple 3-D video formats are provided. A video processing system may receive one or more video frames comprising first 3-D view pixel data and second 3-D view pixel data suitable for generating a three-dimensional (3-D) video frame. The video system may be operable to determine an arrangement of the first 3-D view pixel data and the second view pixel data in the one or more video frames. In instances that the determined arrangement is not a desired arrangement, the video processing system may be operable to convert the one or more video frames to the desired arrangement. Either or both of the determined arrangement and the desired arrangement may comprise a series of two single-view frames. Either or both of the determined arrangement and the desired arrangement may comprise a single frame comprising the first 3-D view pixel data and the second 3-D view pixel data.

Skip to: Description  ·  Claims  · Patent History  ·  Patent History
Description
CLAIM OF PRIORITY

This patent application makes reference to, claims priority to and claims benefit from:

U.S. Provisional Patent Application Ser. No. 61/296,851 (Attorney Docket No. 22866US01) filed on Jan. 20, 2010;
U.S. Provisional Patent Application Ser. No. 61/267,729 (Attorney Docket No. 20428US01) filed on Dec. 8, 2009; and
U.S. Provisional Patent Application Ser. No. 61/330,456 (Attorney Docket No. 23028US01) filed on May 3, 2010.

Each of the above stated applications is hereby incorporated herein by reference in its entirety.

INCORPORATION BY REFERENCE

This patent application also makes reference to:

U.S. Provisional patent application Ser. No. ______ (Attorney Docket No. 23437US02) filed on Dec. 8, 2010;
U.S. Provisional patent application Ser. No. ______ (Attorney Docket No. 23438US02) filed on Dec. 8, 2010;
U.S. Provisional patent application Ser. No. ______ (Attorney Docket No. 23439US02) filed on Dec. 8, 2010; and
U.S. Provisional patent application Ser. No. ______ (Attorney Docket No. 23440US02) filed on Dec. 8, 2010.

Each of the above stated applications is hereby incorporated herein by reference in its entirety.

FIELD OF THE INVENTION

Certain embodiments of the invention relate to video processing. More specifically, certain embodiments of the invention relate to a method and system for handling multiple 3-D video formats.

BACKGROUND OF THE INVENTION

Support of three-dimensional (3-D) video presents many complexities that are not addressed in conventional two-dimensional (2D) video processing systems. The rapid growth of 3-D video systems has resulted in inconsistent and inadequate ways of dealing with these complexities.

Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of such systems with some aspects of the present invention as set forth in the remainder of the present application with reference to the drawings.

BRIEF SUMMARY OF THE INVENTION

A system and/or method is provided for handling multiple 3-D video formats, substantially as illustrated by and/or described in connection with at least one of the figures, as set forth more completely in the claims.

These and other advantages, aspects and novel features of the present invention, as well as details of an illustrated embodiment thereof, will be more fully understood from the following description and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is diagram illustrating a video processing system, in accordance with an embodiment of the invention.

FIG. 2 is flow chart illustrating exemplary operation for converting between arrangements of 3-D pixel data, in accordance with an embodiment of the invention.

FIG. 3 is a diagram illustrating various arrangements of one or more frames comprising 3-D pixel data, in accordance with an embodiment of the invention.

FIG. 4A is a diagram illustrating reception and storage of a single-frame-left-right arrangement of 3-D pixel data, in accordance with an embodiment of the invention.

FIG. 4B is a diagram illustrating reception and storage of a single-frame-over-under arrangement of 3-D pixel data, in accordance with an embodiment of the invention.

FIG. 4C is a diagram illustrating reception and storage of a two-frame-sequential arrangement of 3-D pixel data, in accordance with an embodiment of the invention.

FIG. 5A is a diagram illustrating reading 3-D pixel data from memory to generate a left-right-single-frame arrangement of 3-D pixel data, in accordance with an embodiment of the invention.

FIG. 5B is a diagram illustrating reading 3-D pixel data from memory to generate an over-under-single-frame arrangement of 3-D pixel data, in accordance with an embodiment of the invention.

FIG. 5C is a diagram illustrating reading 3-D pixel data from memory to generate a multi-frame arrangement of 3-D pixel data, in accordance with an embodiment of the invention.

FIG. 6 is a flow chart illustrating exemplary steps for 3-D video processing, in accordance with an embodiment of the invention.

 DETAILED DESCRIPTION OF THE INVENTION

Various embodiments of the invention may be found in a method and system for handling multiple 3-D video formats. In various embodiments of the invention, a video processing system may receive one or more video frames comprising pixel data for a first 3-D view and pixel data for a second 3-D view, which may both be suitable for generating a three-dimensional (3-D) video frame. The pixel data for the first 3-D view via may be referred to as the first 3-D view pixel data and the pixel data for the second 3-D view via may be referred to as the second 3-D view pixel data. The video system may be operable to determine an arrangement of the first 3-D view pixel data and the second 3-D view pixel data in the one or more video frames. In instances that the determined arrangement is not a desired arrangement, the video processing system may be operable to convert the one or more video frames to the desired arrangement. Either one or both of the determined arrangement and the desired arrangement may comprise a series of two single-view frames, and each of the single-view frames may comprise one of the first 3-D view pixel data and the second 3-D view pixel data. Either or both of the determined arrangement and the desired arrangement may comprise a single frame comprising the first 3-D view pixel data and the second 3-D view pixel data. The single frame may be arranged such that a left portion of the single frame comprises the first 3-D view pixel data and a right portion of the single frame comprises the second 3-D view pixel data. The single frame may be arranged such that a top portion of the single frame comprises the first 3-D view pixel data and a bottom portion of the single frame comprises the second 3-D view pixel data. The single frame may be arranged such that the first 3-D view pixel data is interleaved with the second 3-D view pixel data.

The converting may comprise writing the first 3-D view pixel data to one or more locations in memory identified by a first one or more pointers and/or writing the second 3-D view pixel data to one or more locations in memory identified by a second one or more pointers. The converting may also comprise reading the first 3-D view pixel data and the second 3-D view pixel data from memory in an order that is different than an order in which the first 3-D view pixel data and the second 3-D view pixel data was written to memory. The video system may receive the first 3-D view pixel data and second 3-D view pixel data via a first switching element that is operable to convey pixel data onto one or more of a plurality of data paths, and via a second switching element that is operable to convey pixel data from the plurality of data paths to memory, to the first switching element, and to a compositor. Which one or more of the data paths the left-view pixel data and the right-view pixel data is conveyed onto may be based on the determined arrangement and the desired arrangement. As utilized herein a “3-D view” refers to one view (i.e., a left view or a right view) of a stereoscopic image, “3-D pixel data” refers to pixel data of one or both views of a stereoscopic image, and 3-D video refers to stereoscopic video.

FIG. 1 is diagram illustrating a video processing system, in accordance with an embodiment of the invention. Referring to FIG. 1, the video processing system 100 comprises video input interface 106, video feeder 108, MPEG feeder 110, multiplexers 112a and 112b, processing paths 1141-114J, bypass paths 1161-116K, loopback paths 1181-118L, capture module 120, compositor 122, memory 124, and the memory 126. Each of J, K, and L is an integer greater than or equal to 1. In various embodiments of the invention, the system 100 may, for example, reside in a set-top box, a television, or a desktop or laptop computer. In an exemplary embodiment of the invention, the system 100 may be implemented in single semiconductor die or “chip.” A chip may comprise, for example, an ASIC or an FPGA. In an exemplary embodiment of the invention, the portion of the system 100 enclosed in the dashed line comprise a single-chip video processor.

Each of the memory 124, and the memory 126 may comprise RAM, ROM, NVRAM, flash, a hard drive, or any other suitable memory device. The memory 124, and memory 126 may be physically distinct memory elements of may be different portions and/or partitions of a single memory device.

The video input interface 106 may comprise suitable logic, circuitry, interfaces, and/or code that may be operable to receive a video stream and convey the pixel data of the video stream to the multiplexer 112a. The video input interface 106 may comprise, for example, a VGA interface, composite video interface, component video interface, HDMI interface, DisplayPort interface, and/or other suitable interface and the video stream into the interface 106 may be formatted accordingly. The received video stream may comprise monoscopic (2-D) video data and/or stereoscopic (3-D) video data. While this application focuses on processing of received 3-D video streams. Exemplary details of processing 2-D video streams are described in U.S. patent application Ser. No. ______ (Attorney Docket No. 23438US02) and in U.S. patent application Ser. No. ______ (Attorney Docket No. 23439US02) each of which is incorporated by reference above.

The video feeder 108 may comprise suitable logic, circuitry, interfaces, and/or code that may be operable to input pixel data corresponding to locally generated graphics to the multiplexer 112a. In this regard, the video feeder 108 may, for example, read pixel data out of the memory 126 and convey the pixel data to the multiplexer 112a.

The MPEG feeder 110 may comprise suitable logic, circuitry, interfaces, and/or code that may be operable to receive an MPEG stream and process the MPEG stream to output pixel data to the multiplexer 112a. In this regard, the MPEG stream may be received via a networking device (not shown).

Each of the multiplexers 112a and 112b may comprise suitable logic, circuitry, interfaces, and/or code that may be operable to route pixel data between any one or more inputs of the multiplexer to any one or more outputs of the multiplexer. Pixel data input to the multiplexer 112a from any one of more of the interface 106, feeder 108, and the feeder 110 may be conveyed to any one or more of the processing paths 1141-114J and/or any one or more of the bypass paths 1161-116K. Each of the processing paths 1141-114J may comprise suitable logic, circuitry, interfaces, and/or code that may be operable to perform one or more processing functions. Exemplary processing functions comprise scaling, subsampling, deinterlacing, blur/sharpen, color adjustment, and noise reduction. Each of the bypass paths 1161-116K may enable pixel data to be conveyed unchanged from the multiplexer 112a to the multiplexer 112b. Each of the loopback paths 1181-118L may enable pixel data to be conveyed from the multiplexer 112b to the multiplexer 112a. In this manner, the loopback paths may, for example, enable processing the same pixel data via multiple ones of the processing paths 1141-114J.

The capture module 120 may comprise suitable logic, circuitry, interfaces, and/or code that may be operable to write 3-D pixel data to the memory 126. The capture module 120 may be operable to write first 3-D view pixel data to the memory 126 utilizing a first one or more memory pointers. The capture module 120 may be operable to write second 3-D view pixel data to the memory 126 utilizing a second one or more memory pointers. First 3-D view pixel data may be left-view data, and second 3-D view pixel data may be right-view pixel data, or visa-versa. For example, left-view pixel data may be captured via a left lens of a video camera and right-view pixel data may be captured via a right lens of the video camera.

The compositor 122 may comprise suitable logic, circuitry, interfaces, and/or code that may be operable to generate an output video stream which may be output via, for example, a VGA output, composite video output, component video output, HDMI output, and/or DisplayPort output. The output video stream may comprise pixel data received from the multiplexer 112b and/or pixel data read from memory 124. In this regard, the compositor 122 may be operable to concurrently present pixel data from the memory 124 and pixel data from the multiplexer 112b. For example, graphics from the memory 124 may be overlaid on the pixel data from the multiplexer 112b. The output video stream may comprise monoscopic (2-D) video data and/or stereoscopic (3-D) video data. While this application focuses on outputting 3-D video streams. Exemplary details of processing 2-D video streams are described in U.S. patent application Ser. No. ______ (Attorney Docket No. 23438US02) and in U.S. patent application Ser. No. ______ (Attorney Docket No. 23439US02) each of which is incorporated by reference above.

In operation, one or more 3-D video frames may be input to the system 100 via one or more of the interface 106, the feeder 108, and the feeder 110. Each of the one or more input frames may comprise live-action images and/or computer-generated images. The input frame(s) may comprise left-view pixel data and right-view pixel data. The arrangement of the input frame(s) may correspond to any one of the arrangements described below with respect to FIG. 3. The arrangement of the input frame(s) may be determined in any of a variety of ways. For example, the system 100 may determine the arrangement of the input frame(s) based on the source from which the one or more frames was received, based on a state of one or more control signals in the system 100, and/or based on inspection the input frame(s).

The multiplexer 112a may convey the received frame(s) to the multiplexer 112b via one or more of the processing paths 1141-114J and/or one or more of the bypass paths 1161-116K. In this regard, the frame(s) may make multiple passes from the multiplexer 112a to the multiplexer 112b and thus may traverse one or more of the loopback paths 1181-118L.

Upon arriving at the multiplexer 112b, after one or more traversals of one or more of the processing paths 1141-114J, bypass paths 1161-116K, and/or loopback paths 1181-118L, the frame(s) may be conveyed to the capture module 120. The capture module 120 may write the left-view pixel data and right-view pixel data to the memory 126. The left-view pixel data may be written to one or more memory locations identified by a first one or more pointers. The right-view pixel data may be written to one or more memory locations identified by a second one or more pointers.

Subsequently, the feeder module 108 may read the left-view and right-view pixel data from the memory 126 to generate one or more output frames. The first one or more pointers and the second one or more pointers may be utilized for reading the pixel data out from the memory 126. The order in which the pixel data is read from memory may depend on the arrangement of the input frame(s) and the desired arrangement of the output frame(s). In this regard, the arrangement of the output frame read from the memory 126 may correspond to any of the arrangements described below with respect to FIG. 3. Thus, in instances that the arrangement of the input frame(s) is the same as the arrangement of the output frame(s), the pixel data may be read out of the memory 126 in the same order in which it was written to the memory 126. Conversely, in instances that the arrangement of the input frame(s) is the different than the arrangement of the output frame(s), the pixel data may be read out of the memory 126 in a different order than which it was written to the memory 126.

The output frame(s) may be conveyed to the compositor 122. In some instances, prior to being conveyed to the compositor 122, the output frame(s) may be conveyed to the multiplexer 112a for one or more traversals of one or more of the processing paths 1141-114J, bypass paths 1161-116K, and/or loopback paths 1181-118L.

The compositor 122 may process the output frame(s) to make the output frame(s) suitable for insertion into a video stream. The video stream may be formatted so as to be compatible with one or more video standards such as VGA, composite video, component video, HDMI, and/or DisplayPort. Processing of the output frame(s) may comprise combining the output frame(s) from the multiplexer 112b with pixel data from memory 124. For example, graphics may be read from the memory 124 and overlaid on the output frame(s) from the multiplexer 112b.

FIG. 2 is flow chart illustrating exemplary operation for converting between arrangements of 3-D pixel data, in accordance with an embodiment of the invention. Referring to FIG. 2, the exemplary steps begin with step 202 in which an input frame is conveyed to the multiplexer 112a.

In step 204, it is determined whether the input frame(s) are to traverse one or more of the processing paths 1141-114J or traverse one or more of the bypass paths 1161-116K. In instances that the input frame(s) are to traverse one or more of the processing paths 1141-114J, then in step 224, processing, such as scaling and/or deinterlacing, may occur.

In step 206, the input frame(s) are conveyed to the multiplexer 112b. In step 208 it is determined whether the input frame(s) are to be looped-back to multiplexer 112a for another traversal of one or more of the processing paths processing paths 1141-114J and/or one or more of the bypass paths 1161-116K. In instances that the input frame(s) are to be looped-back, the exemplary steps may return to step 202. In instances that the input frame(s) are not to be looped-back, the exemplary steps may advance to step 210.

In step 210, the input frame(s) are captured to the memory 126. The first 3-D pixel data of the input frame(s) may be stored to memory location(s) indicated by a first one or more memory pointers. The second 3-D view pixel data of the input frame(s) may be stored to memory location(s) indicated by a second one or more memory pointers.

In step 212, the left-view pixel data and right-view pixel data is read from the memory 126 to generate one or more output frame(s). The order in which the data is read from the memory 126 may depend on the desired arrangement of the output frame(s).

In step 214, it is determined whether the output frame(s) are be processed by one or more of the processing paths 1141-114J. In instances that the output frame(s) are to be processed, then the exemplary steps may advance to step 226.

In step 226, the output frame(s) are communicated to the multiplexer 112a. In step 228, the output frame(s) are conveyed onto one or more of the processing paths 1141-114J for processing, such as scaling and/or noise reduction. In step 230, the output frame(s) may arrive at the multiplexer 112b. In step 232 it may be determined whether the output frame(s) are to be looped-back to multiplexer 112a for another traversal of one or more of the processing paths processing paths 1141-114J and/or one or more of the bypass paths 1161-116K. In instances that the output frame(s) are to be looped-back, the exemplary steps may return to step 226. In instances that the output frame(s) are not to be looped-back, the exemplary steps may advance to step 216.

In step 216, the output frame(s) arrive at the multiplexer 112b. In step 218 the output frame(s) are conveyed to the compositor 122. In step 220, the compositor may process the output frame(s) to make them suitable for insertion into a video stream. Processing the output frame(s) may comprise combining the output frame(s) from the multiplexer 112b with pixel data from memory 124. For example, graphics may be read from the memory 124 and overlaid on the output frame(s) from the multiplexer 112b.

In step 222, the video stream may be communicated to another video device, such as a television or monitor. The video stream may, for example, be formatted in accordance with one or more video standards such as VGA, composite video, component video, HDMI, and/or DisplayPort.

FIG. 3 is a diagram illustrating various arrangements of one or more frames comprising 3-D pixel data, in accordance with an embodiment of the invention. Referring to FIG. 3, there is shown a two-frame-sequential arrangement 302, a left-right-single-frame arrangement 304, an over-under-single-frame arrangement 306, a vertically-interleaved-single-frame arrangement 308, a horizontally-interleaved-single-frame arrangement 310, and a vertically-and-horizontally-interleaved-single-frame arrangement 312. For the following description of the various arrangements, each of N and M may be any positive integer.

The two-frame-sequential arrangement 302 comprises a first frame comprising first 3-D view pixel data and a second frame comprising second 3-D view pixel data. The two frames may be received by the system 100 sequentially. That is, the first frame may be received earlier in time before the second frame.

The left portion of the left-right-single-frame arrangement 304 may comprise first 3-D view pixel data and the right portion of the left-right single-frame arrangement 304 may comprise second 3-D view pixel data. An exemplary 4M×4N left-right-single-frame arrangement is described in table 1 below.

TABLE 1 Left-Right-Single-Frame arrangement Col. 1-2M Col. 2M + 1-4M Lines 1-4N first 3-D view second 3-D view

The top portion of the over-under-single-frame arrangement 306 may comprise first 3-D view pixel data and the bottom portion of the over-under-single-frame arrangement 306 may comprise second 3-D view pixel data. An exemplary 4M×4N over-under-single-frame arrangement is described in table 2 below.

TABLE 2 Over-Under-Single-Frame Arrangement Col. 1-4M Lines 1-2N first 3-D view Lines 2N + 1-4N second 3-D view

The vertically-interleaved-single-frame arrangement 308 may alternate between one or more lines of left-view pixel data and one or more lines of right-view pixel data. An exemplary 4M×4N vertically-interleaved-single-frame arrangement is described in table 3 below.

TABLE 3 Vertically-Interleaved-Single-Frame Arrangement Col. 1-4M Lines 1-N first 3-D view Lines N + 1-2N second 3-D view Lines 2N + 1-3N first 3-D view Lines 3N + 1-4N second 3-D view

The horizontally-interleaved-single-frame arrangement 310 may alternate between one or more columns of left-view pixel data and one or more columns of right-view pixel data. An exemplary 4M×4N horizontally-interleaved-single-frame arrangement is described in table 4 below.

TABLE 4 Horizontally-Interleaved-Single-Frame Arrangement Col. M + Col. 2M + Col. 3M + Col. 1-M 1-2M 1-3M 1-4M Lines 1-4N first 3-D view second first 3-D view second 3-D view 3-D view

In the vertically-and-horizontally-interleaved-single-frame arrangement 312, the first 3-D view and second 3-D view pixel data may be interleaved in both a vertical and horizontal direction. An exemplary 4M×4N horizontally-interleaved-single-frame arrangement is described in table 4 below.

TABLE 5 Vertically-and-Horizontally-Interleaved-Single-Frame Arrangement Col. 1-M Col. M + 1-2M Col. 2M + 1-3M Col. 3M + 1-4M Lines 1-N first 3-D view second 3-D view first 3-D view second 3-D view Lines N + 1-2N second 3-D view first 3-D view second 3-D view first 3-D view Lines 2N + 1-3N first 3-D view second 3-D view first 3-D view second 3-D view Lines 3N + 1-4N second 3-D view first 3-D view second 3-D view first 3-D view

FIG. 4A is a diagram illustrating reception and storage of a single-frame-left-right arrangement of 3-D pixel data, in accordance with an embodiment of the invention. Referring to FIG. 4A, there is shown a 4×2 left-right-single-frame arrangement 402 being written to the memory 126. The pixel data may be written to the memory 126 in the order in which it was received. In this regard, the pixel data may be received line by line with each line being received from left to right. That is, pixel data may arrive in the following order: column 1 line 1, column 2 line 1, column 3 line 1, column 4 line 1, column 1 line 2, column 2, line 2, column 3, line 3, column 4 line 4. First 3-D view pixel data may be written to the location(s) 150a of the memory 126 and second 3-D view pixel data may be written to the location(s) 150b of the memory 126. The location(s) 150a may be identified by a first one or more memory pointers and the location(s) 150b may be identified by a second one or more memory pointers. Although the first 3-D view pixel data and second 3-D view pixel data are depicted as being written to the same memory, the invention is not so restricted. For example, first 3-D view pixel data may be written to a first memory and second 3-D view pixel data may be written to a second memory.

FIG. 4B is a diagram illustrating reception and storage of a single-frame-over-under arrangement of 3-D pixel data, in accordance with an embodiment of the invention. Referring to FIG. 4B, there is shown a 2×4 over-under-single-frame arrangement 406 being written to the memory 126. The pixel data may be written to the memory 126 in the order in which it was received. In this regard, the pixel data may be received line by line, with each line being received from left to right. That is, pixel data may arrive in the following order: column 1 line 1, column 2 line 1, column 1 line 2, column 2 line 2, column 1 line 3, column 2, line 3, column 1, line 4, column 2 line 4. First 3-D view pixel data may be written to the location(s) 150a of the memory 126 and second 3-D view pixel data may be written to the location(s) 150b of the memory 126. The location(s) 150a may be identified by a first one or more memory pointers and the location(s) 150b may be identified by a second one or more memory pointers. Although the first 3-D view pixel data and second 3-D view pixel data are depicted as being written to the same memory, the invention is not so restricted. For example, first 3-D view pixel data may be written to a first memory and second 3-D view pixel data may be written to a second memory.

FIG. 4C is a diagram illustrating reception and storage of a two-frame-sequential arrangement of 3-D pixel data, in accordance with an embodiment of the invention. Referring to FIG. 4C, there is shown a 2×2 frame 408a comprising first 3-D view pixel data and 2×2 frame 408b comprising second 3-D view pixel data being written to the memory 126. The pixel data may be written to the memory 126 in the order in which it was received. In this regard, frame 408a may be received before frame 408b. Each frame may be received line by line, with each line being received from left to right. That is, pixel data may arrive in the following order: column 1 line 1 of frame 408a, column 2 line 1 of frame 408a, column 1 line 2 of frame 408a, column 2 line 2 of frame 408a, column 1 line 1 of frame 408b, column 2 line 1 of frame 408b, column 1 line 2 of frame 408b, and column 2 line 2 of frame 408b. First 3-D view pixel data may be written to the location(s) 150a of the memory 126 and second 3-D view pixel data may be written to the location(s) 150b of the memory 126. The location(s) 150a may be identified by a first one or more memory pointers and the location(s) 150b may be identified by a second one or more memory pointers. Although the first 3-D view pixel data and second 3-D view pixel data are depicted as being written to the same memory, the invention is not so restricted. For example, first 3-D view pixel data may be written to a first memory and second 3-D view pixel data may be written to a second memory.

FIG. 5A is a diagram illustrating reading 3-D pixel data from memory to generate a left-right-single-frame arrangement of 3-D pixel data, in accordance with an embodiment of the invention. Referring to FIG. 5A, there is shown a 4×2 left-right-single-frame arrangement 502 being read from the memory 126. The pixel data may be read from memory line by line, with each line being read from left to right. That is, pixel data may be read in the following order: first 3-D view pixel data may be read out for column 1 line 1 and column 2 line 1, second 3-D view pixel data may be read out for column 3 line 1 and column 4 line 1, first 3-D view pixel data may be read out for column 1 line 2 and column 2, line 2, and second view pixel data may be read out for column 3, line 3 and column 4 line 4.

FIG. 5B is a diagram illustrating reading 3-D pixel data from memory to generate an over-under-single-frame arrangement of 3-D pixel data, in accordance with an embodiment of the invention. Referring to FIG. 5B, there is shown a 2×4 over-under-single-frame arrangement 506 being read from the memory 126. The pixel data may be read from memory line by line, with each line being read from left to right. That is, pixel data may be read in the following order: first 3-D view pixel data may be read out for column 1 line 1, then column 2 line 1, then column 1 line 2, then column 2 line 2. Second 3-D view pixel data may then be read out for column 1 line 3, then column 2, line 3, then column 1, line 4, then column 2 line 4.

FIG. 5C is a diagram illustrating reading 3-D pixel data from memory to generate a two-frame-sequential arrangement of 3-D pixel data, in accordance with an embodiment of the invention. Referring to FIG. 5C, there is shown 2×2 frames 508a and 508b being read from the memory 126. The pixel data may be read from memory line by line, with each line being read from left to right. That is, pixel data may be read in the following order: first 3-D view pixel data may be read out for column 1 line 1 of frame 508a, then column 2 line 1 of frame 508a, then column 2 line 1 of frame 508a, then column 2 line 2 of frame 508a. Subsequently, second 3-D view pixel data may be read out for column 1 line 1 of frame 508b, then column 2 line 1 of frame 508b, then column 2 line 1 of frame 508b, then column 2 line 2 of frame 508b.

FIG. 6 is a flow chart illustrating exemplary steps for 3-D video processing, in accordance with an embodiment of the invention. Referring to FIG. 6, after start step 602, the system 100 may be configured to select an output frame arrangement. The output frame arrangement may be selected based on, for example, the device(s) from which the system 100 receives a video stream and/or the device(s) to which the system 100 outputs a video stream. In step 606, the system 100 may receive one or more frames comprising 3-D pixel data. The arrangement of the input frame(s) may be any of the arrangements described with respect to FIG. 3. In step 608, the system 100 may determine the arrangement of the input frame(s). This determination may be based on, for example, an inspection of the input frame(s), the source from which the frame(s) were received, and/or based on a pre-configuration of the system 100. In step 610, the input frame(s) may be processed, if necessary. The processing may comprise, for example, scaling, de-interlacing, noise reduction, and/or chroma subsampling. In step 612, the pixel data of the input frame(s) may be stored to memory, with first 3-D view pixel data being stored to one or more locations identified by a first one or more pointers and second 3-D view pixel data being stored to one or more locations identified by a second one or more pointers. In step 614, the pixel data may be read from memory to generate one or more output frames. The arrangement of the output frame(s) may be any of the arrangements described with respect to FIG. 3. In step 616, the output frame(s) may be processed, if necessary. The processing may comprise, for example, scaling, de-interlacing, noise reduction, and/or chroma subsampling. In step 618, the output frame may be processed for insertion into a video stream, and inserted into the video stream. The video stream may be communicated to, for example, a television or monitor.

Various aspects of the invention may be found in a method and system for handling multiple 3-D video formats. In an exemplary embodiment of the invention, a video processing system 100 may receive one or more video frames, such one or more of the frames 402, 406, 408a and 408b, comprising first 3-D view pixel data and second 3-D view pixel data suitable for generating a three-dimensional (3-D) video frame. The video system 100 may be operable to determine an arrangement of the first 3-D view pixel data and the second view pixel data in the one or more video frames, where the arrangement corresponds to one of the arrangements 302-312. In instances that the determined arrangement is not a desired arrangement, the video processing system 100 may be operable to convert the one or more video frames to the desired arrangement. Either or both of the determined arrangement and the desired arrangement may be the arrangement 302 and may comprise a series of two single-view frames, each of the single-view frames comprising one of the first 3-D view pixel data and the second 3-D view pixel data.

Either or both of the determined arrangement and the desired arrangement may be one of the arrangements 304-312 and comprise a single frame comprising the first 3-D view pixel data and the second 3-D view pixel data. The single frame may be arranged as arrangement 304 and a left portion of the single frame may comprise the first 3-D view pixel data and a right portion of the single frame may comprise the second 3-D view pixel data. The single frame may be arranged as arrangement 306 and a top portion of the single frame may comprise the first 3-D view pixel data and a bottom portion of the single frame may comprise the second 3-D view pixel data. The single frame may be arranged as one of arrangements 308, 310, and 312 and may comprise the first 3-D view pixel data interleaved with the second 3-D view pixel data.

The converting may comprise writing the first 3-D view pixel data to one or more locations 150a in the memory 126 identified by a first one or more pointers. The converting may comprise writing the second 3-D view pixel data to one or more locations 150b in the memory 126 identified by a second one or more pointers. The converting may comprise reading the first 3-D view pixel data and the second 3-D view pixel data from memory in an order that is different than an order in which the first 3-D view pixel data and the second 3-D view pixel data was written to memory. The video processing system 100 may receive the first 3-D view pixel data and second 3-D view pixel data via a first switching element 112a that is operable to convey pixel data onto one or more of a plurality of data paths 1141-114J and/or 1161-116K, and via a second switching element 112b that is operable to convey pixel data from the plurality of data paths 1141-114J and/or 1161-116K to the memory 126 (via capture module 120), to the first switching element 112a, and to the compositor 122. Which one or more of the data paths 1141-114J, 1161-116K and/or 1181-118L and/or the left-view pixel data and the right-view pixel data is conveyed onto may be based on the determined arrangement and the desired arrangement.

Other embodiments of the invention may provide a non-transitory computer readable medium and/or storage medium, and/or a non-transitory machine readable medium and/or storage medium, having stored thereon, a machine code and/or a computer program having at least one code section executable by a machine and/or a computer, thereby causing the machine and/or computer to perform the steps as described herein for handling multiple 3-D video formats.

Accordingly, the present invention may be realized in hardware, software, or a combination of hardware and software. The present invention may be realized in a centralized fashion in at least one computer system, or in a distributed fashion where different elements are spread across several interconnected computer systems. Any kind of computer system or other apparatus adapted for carrying out the methods described herein is suited. A typical combination of hardware and software may be a general-purpose computer system with a computer program that, when being loaded and executed, controls the computer system such that it carries out the methods described herein.

The present invention may also be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein, and which when loaded in a computer system is able to carry out these methods. Computer program in the present context means any expression, in any language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after either or both of the following: a) conversion to another language, code or notation; b) reproduction in a different material form.

While the present invention has been described with reference to certain embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the present invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the present invention without departing from its scope. Therefore, it is intended that the present invention not be limited to the particular embodiment disclosed, but that the present invention will include all embodiments falling within the scope of the appended claims.

Claims

1. A method comprising:

in a video processing device: receiving one or more video frames, said received one or more frames comprising first 3-D view pixel data and second 3-D view pixel data for generating a three-dimensional (3-D) video frame; determining an arrangement of said first 3-D view pixel data and said second 3-D view pixel data in said one or more video frames; and in instances that said determined arrangement is not a desired arrangement, converting said one or more video frames to said desired arrangement.

2. The method of claim 1, wherein:

one of said determined arrangement and said desired arrangement comprises a series of two single-view frames, each of said single-view frames comprising one of said first 3-D view pixel data and said second 3-D view pixel data; and
one of said determined arrangement and said desired arrangement comprises a single frame, said single frame comprising said first 3-D view pixel data and said second 3-D view pixel data.

3. The method of claim 2, wherein said single frame is arranged such that a left portion of said single frame comprises said first 3-D view pixel data and a right portion of said single frame comprises said second 3-D view pixel data.

4. The method of claim 2, wherein said single frame is arranged such that a top portion of said single frame comprises said first 3-D view pixel data and a bottom portion of said single frame comprises said second 3-D view pixel data.

5. The method of claim 1, wherein:

one of said determined arrangement and said desired arrangement comprises a left-right single frame, a left portion of said left-right single frame comprising said first 3-D view pixel data and a right portion of said left-right single frame comprising said second 3-D view pixel data; and
one of said determined arrangement and said desired arrangement comprises an over-under single frame, a top portion of said over-under single frame comprising said first 3-D view pixel data and a bottom portion of said over-under single frame comprising said second 3-D view pixel data.

6. The method of claim 1, wherein:

one of said determined arrangement and said desired arrangement comprises an interleaved single frame, said interleaved single frame comprising said first 3-D view pixel data interleaved with said second 3-D view pixel data; and
one of said determined arrangement and said desired arrangement comprises a left-right single frame, a left portion of said left-right single frame comprising said first 3-D view pixel data and a right portion of said left-right single frame comprising said second 3-D view pixel data.

7. The method of claim 1, wherein:

one of said determined arrangement and said desired arrangement corresponds to an interleaved single frame, said interleaved single frame comprising said first 3-D view pixel data interleaved with said second 3-D view pixel data; and
one of said determined arrangement and said desired arrangement corresponds to a over-under single frame, a top portion of said over-under single frame comprising said first 3-D view pixel data and a bottom portion of said over-under single frame comprises said second 3-D view pixel data.

8. The method of claim 1: wherein said converting comprises:

writing said first 3-D view pixel data to one or more locations in memory identified by a first one or more pointers;
writing said second 3-D view pixel data to one or more locations in memory identified by a second one or more pointers; and
reading said first 3-D view pixel data and said second 3-D view pixel data from memory in an order that is different than an order in which said first 3-D view pixel data and said second 3-D view pixel data was written to memory.

9. The method according to claim 1, comprising receiving said first 3-D view pixel data and second 3-D view pixel data via:

a first switching element that is operable to convey pixel data onto one or more of a plurality of data paths; and
a second switching element that is operable to convey pixel data from said plurality of data paths to memory, to said first switching element, and to a compositor.

10. The method according to claim 9, comprising selecting which one or more of said data paths onto which said pixel data for 3-D left view and said pixel data 3-D right view is conveyed based on said determined arrangement and said desired arrangement.

11. A system comprising:

one or more circuits for use in a video processing system, said one or more circuits being operable to: receive one or more video frames, said one or more frames comprising first 3-D view pixel data and second 3-D view pixel data suitable for generating a three-dimensional (3-D) video frame; determine an arrangement of said first 3-D view pixel data and said second view pixel data in said one or more video frames; and in instances that said determined arrangement is not a desired arrangement, convert said one or more video frames to said desired arrangement.

12. The system of claim 11, wherein:

one of said determined arrangement and said desired arrangement comprises a series of two single-view frames, each of said single-view frames comprising one of said first 3-D view pixel data and said second 3-D view pixel data; and
one of said determined arrangement and said desired arrangement comprises a single frame, said single frame comprising said first 3-D view pixel data and said second 3-D view pixel data.

13. The system of claim 12, wherein said single frame is arranged such that a left portion of said single frame comprises said first 3-D view pixel data and a right portion of said single frame comprises said second 3-D view pixel data.

14. The system of claim 12, wherein said single frame is arranged such that a top portion of said single frame comprises said first 3-D view pixel data and a bottom portion of said single frame comprises said second 3-D view pixel data.

15. The system of claim 11, wherein:

one of said determined arrangement and said desired arrangement comprises a left-right single frame, a left portion of said left-right single frame comprising said first 3-D view pixel data and a right portion of said left-right single frame comprising said second 3-D view pixel data; and
one of said determined arrangement and said desired arrangement comprises an over-under single frame, a top portion of said over-under single frame comprising said first 3-D view pixel data and a bottom portion of said over-under single frame comprising said second 3-D view pixel data.

16. The system of claim 11, wherein:

one of said determined arrangement and said desired arrangement comprises an interleaved single frame, said interleaved single frame comprising said first 3-D view pixel data interleaved with said second 3-D view pixel data; and
one of said determined arrangement and said desired arrangement comprises a left-right single frame, a left portion of said left-right single frame comprising said first 3-D view pixel data and a right portion of said left-right single frame comprising said second 3-D view pixel data.

17. The system of claim 11, wherein:

one of said determined arrangement and said desired arrangement corresponds to an interleaved single frame, said interleaved single frame comprising said first 3-D view pixel data interleaved with said second 3-D view pixel data; and
one of said determined arrangement and said desired arrangement corresponds to a over-under single frame, a top portion of said over-under single frame comprising said first 3-D view pixel data and a bottom portion of said over-under single frame comprises said second 3-D view pixel data.

18. The system of claim 11: wherein said converting comprises:

writing said first 3-D view pixel data to one or more locations in memory identified by a first one or more pointers;
writing said second 3-D view pixel data to one or more locations in memory identified by a second one or more pointers;
reading said first 3-D view pixel data and said second 3-D view pixel data from memory in an order that is different than an order in which said first 3-D view pixel data and said second 3-D view pixel data was written to memory.

19. The system according to claim 11, wherein said one or more circuits are operable to receive said first 3-D view pixel data and second 3-D view pixel data via:

a first switching element that is operable to convey pixel data onto one or more of a plurality of data paths;
a second switching element that is operable to convey pixel data from said plurality of data paths to: memory, to said first switching element, and to a compositor.

20. The system according to claim 19, wherein said one or more circuits are operable to select which one or more of said data paths onto which said left-view pixel data and said right-view pixel data is conveyed based on said determined arrangement and said desired arrangement.

Patent History
Publication number: 20110134211
Type: Application
Filed: Dec 8, 2010
Publication Date: Jun 9, 2011
Inventors: Darren Neuman (Palo Alto, CA), Jason Herrick (Pleasanton, CA), Christopher Payson (Bolton, MA), Qinghua Zhao (Cupertino, CA)
Application Number: 12/963,212
Classifications
Current U.S. Class: Stereoscopic (348/42); Stereoscopic Television Systems; Details Thereof (epo) (348/E13.001)
International Classification: H04N 13/00 (20060101);