Methods and systems for compressing a video stream with minimal loss after subsampled decoding
Transcoding of a video stream to reduce the size of the video stream with little, if any, loss in video quality after subsampling. After accessing a video stream of video pictures (i.e., video frames or fields), the blocks of the video picture are each subject to matrix premultiplication and postmultiplication. Such matrix multiplication does degrade the video quality if subsampling was not to occur. However, the premultiplication and postmultiplication matrices are calculated based on the subsampling matrices that will be used to ultimately subsample the video stream such that after subsampling eventually occurs, the matrix multiplications result in minimal loss of video quality.
Latest Microsoft Patents:
 Roaming between network access points based on dynamic criteria
 Extending sharing options of local computing resources
 Video coding/decoding with subblock transform sizes and adaptive deblock filtering
 Innovations in block vector prediction and estimation of reconstructed sample values within an overlap area
 Motion estimation for screen remoting scenarios
Description
CROSSREFERENCE TO RELATED APPLICATIONS
The present application is a continuation application of commonlyassigned U.S. patent application Ser. No. 09/886,693 filed Jun. 18, 2001, entitled “Methods and Systems for Compressing a Video Stream with Minimal Loss After Subsampled Decoding” and which is incorporated herein by reference.
BACKGROUND OF THE INVENTION
1. The Field of the Invention
The present invention relates to the field of video processing. In particular, the present invention relates to the compression of a video stream when it is known that the video stream is to be subsampled for minimal loss after subsampled decoding.
2. Background and Related Art
Video constitutes a series of images that, when displayed above a certain rate, gives the illusion to a human viewer that the image is moving. Video is now a widespread medium for communicating information whether it be a television broadcast, a taped program, or the like. More recently, digital video has become popular.
An uncompressed digital video stream has high bandwidth and storage requirements. For example, the raw storage requirement for uncompressed CCIR601 resolution 4:2:2: serial digital video is approximately 20 megabytes per second of video. In addition, associated audio and data channels also require bandwidth and storage. From a transmission bandwidth perspective, 20 megabytes per second is much faster than conventional transmission techniques can practicably support. In addition, from a storage perspective, a twohour movie would occupy approximately 144 Gigabytes of memory, well above the capabilities of a conventional Digital Versatile Disk (DVD). Therefore, what were desired were systems and methods for compressing (or coding) digital video in a way that maintains a relatively high degree of fidelity with the original video once uncompressed (or decoded).
One conventional highquality compression standard is called MPEG2, which is based on the principle that there is a large degree of visual redundancy in video streams. By removing much of the redundant information, the video storage and bandwidth requirements are significantly reduced.
Under the MPEG2 standard, there are three classes of pictures, Ipictures, Ppictures and Bpictures. While MPEG2 allows for a number of display orders for groups of pictures, the display order illustrated in
The Ipictures are “intracoded” meaning that they can be restructured without reference to any other picture in the video stream.
The Ppictures are “intercoded” meaning that they may only be restructured with reference to another reference picture. Typically, the Ppicture may include motion vectors that represent estimated motion with respect to the reference picture. The Ppicture may be reconstructed using the immediately preceding Ipicture or Ppicture as a reference. In
Bpictures are also intercoded. The Bpicture is typically reconstructed using the immediately preceding Ipicture or Ppicture as a reference, and the immediately subsequent Ipicture or Ppicture as a reference. For example, the reconstruction of Bpicture B14 uses Ppicture P13 and Ipicture I16 as references.
If the digital picture 201 is to be a Ppicture, the encoding process is similar as for Ipictures with several notable exceptions. If a Ppicture, the digital picture is passed first to the motion estimator 202. For each macroblock (i.e., 16×16 pixel array) in the Ppicture, the motion estimator 202 finds a close match to the macroblock in the reference picture. The motion estimator 202 then represents the macroblock in the Ppicture as a motion vector representing the motion between the macroblock in the Ppicture and the close match 16×16 pixel array in the reference picture. In addition to the motion vector, a difference macroblock is calculated representing the difference between the macroblock in the Ppicture and the close match 16×16 pixel array in the reference frame. A macroblock represented as a difference with corresponding motion vectors is typically smaller than a macroblock represented without motion vectors. Discrete cosine transformation and quantization are then performed on just the difference representation of the Ppicture. Then, the difference information is combined with the motion vectors before variable length coding is performed.
Bpictures are encoded similar to how Ppictures are encoded, except that motion may be estimated with reference to a prior reference picture and a subsequent reference picture.
In this manner, MPEG2 combines the functionality of motion compensation, discrete cosine transformation, quantization, and variable length coding to significantly reduce the size of a video stream with some generally acceptable reduction in video quality. Despite conventional standards such as MPEG2 that provide significant compression to a video stream, it is desirable to reduce the bandwidth requirements of the video stream even more to maximize network and storage performance.
One way to further reduce the bandwidth requirements is to compress the video stream even beyond the compression performed during the original MPEG2 encoding processes. However, this results in a loss of video information and thus degrades the quality of the video stream to a certain extent. Therefore, what are desired are systems and methods for further compressing a video stream with less, if any, loss of video information.
BRIEF SUMMARY OF THE INVENTION
The present invention extends to both methods and systems for transcoding a video stream so as to reduce the size of the video stream with little, if any, degradation of video quality after subsampling. The video steam includes a number of video pictures such as frames or fields and may be stored in memory or accessed from a transmission. In addition, each video picture includes one or more blocks. These blocks are the fundamental unit upon which subsampling may be performed. For example, under the MPEG2 standard, developed by the Moving Pictures Experts Group, subsampling may be performed on blocks of 8 pixels by 8 pixels.
The video management system accesses one of the video pictures from the video stream. Then, for at least one block of the video picture, the video management system represents the block as a matrix of pixel values. Then, the block matrix is premultiplied by a premultiplication matrix and postmultiplied by a postmultiplication matrix. The premultiplication matrix is generated from a subsample matrix that represents the subsampled decoding in one direction. The postmultiplication matrix is generated from a subsample matrix that represents the subsampled decoding in a substantially perpendicular direction.
The premultiplication matrix and the postmultiplication matrix are structured so that the block of pixels is altered in a manner that subsampling of the altered block of pixels results in the same subsampled image as subsampling of the original block of pixels. The premultiplication matrix and the postmultiplication matrix are also designed to decrease the size of the encoded version of the block of pixels.
This strategic altering of blocks of pixels may be repeated for each block in the video picture and for each video picture in the video stream that is to be subject to subsampled decoding. Accordingly, the memory and bandwidth requirements of the video stream may be substantially reduced with the satisfaction that the reduction comes at minimal cost in video quality assuming that the video stream is to ultimately be subsample decoded. In one aspect of the invention, the further compressed video stream is sent to a subsample decoder where it is subsampled and presented on a display device.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the invention. The features and advantages of the invention may be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. These and other features of the present invention will become more fully apparent from the following description and appended claims, or may be learned by the practice of the invention as set forth hereinafter.
BRIEF DESCRIPTION OF THE DRAWINGS
In order to describe the manner in which the aboverecited and other advantages and features of the invention can be obtained, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments thereof, which are illustrated in the appended drawings. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
Subsampling is a process that reduces the dimensions of a video image such as when the video stream is to be displayed in a reducedsize pictureinpicture display. The present invention extends to both methods and systems for reducing the size of the video stream with minimal, if any, effect on the video quality as displayed after subsampling. A video management system accesses a video stream by receiving the video stream from a video channel, or by accessing a memory where the video stream is stored. Once the video management system determines that only a reducedsize version of the video stream is ultimately to be displayed as when the video stream is to be subject to subsampling, the video management system compresses each picture (e.g., frame or field) of the video frame. Although this compression would cause loss of picture quality if the picture were to be displayed in its full size, this compression is performed in such a manner that there is little, if any, loss in video quality as displayed after subsampling. Any loss in video quality would be primarily due to requantization, and finiteprecision effects inherent in computer processing.
Embodiments within the scope of the present invention include computerreadable storage media having computerexecutable instructions or data structures stored thereon. Such computerreadable media can be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computerreadable media can comprise RAM, ROM, EEPROM, CDROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store desired program code means in the form of computerexecutable instructions or data structures and which can be accessed by a general purpose or special purpose computer.
When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a computer, the computer properly views the connection as a computerreadable transmission medium. Combinations of the above should also be included within the scope of computerreadable media. Computerexecutable instructions comprise, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions.
The precise operating environment in which the principles of the present invention are implemented is not important to the present invention. The principles of the present invention may be implemented in any operating environment that is able to implement the principles of the present invention. For example, given suitable software and/or adaptation, generalpurpose computers, specialpurpose computers or special purpose processing devices (whether now developed or to be developed in the future) might implement the principles of the present invention. In addition, the principles of the present invention may be implemented by software, hardware, firmware or any combination thereof.
As will be described in further detail below, the principles of the present invention are most advantageous when the video processing in accordance with the present invention is followed by subsampling. The environment discussed below with respect to
The video management system 310 includes a memory 341 that may store the computerexecutable instructions described above, and a processor 342 that is coupled to the memory 341 through, for example, a bus 343 so as to be able to execute the computerexecutable instructions. The video management system 310 also includes a video decoder 344 that decodes video in accordance with a video decoding standard such as, for example, MPEG. A transcoder 345 operates to reduce the memory and bandwidth requirements of the video 302 and may do such by implementing the principles of the present invention described herein. If the video decoder 344 and the transcoder 345 are implemented at least partially in hardware, the video decoder 344 and the transcoder 345 would be coupled to the bus 343 as shown in
While
First, the video stream is accessed (act 401). Then, for at least one of the blocks in a video picture of the video stream, the size of the encoded block is reduced without substantially reducing image quality as measure after subsampling (act 402). Preferably, all of the blocks in all of the video pictures in the video stream are compressed when it is known that the video stream is to be ultimately subject to subsampled decoding. The processing of an already encoded video stream (or any already encoded data component for that matter) into a different encoded video stream is often referred to as “transcoding” since the video stream is moved from one encoded state to another. The compression of the encoded video stream to generated a more compressed video stream with minimal, if any, loss in video quality after subsampling is one example of transcoding and will be referred to herein as “subsampled transcoding” although subsampled decoding is still needed after the subsampled transcoding in order to display the reduced size image. In the context of
In order to generate the reduced size blocks, the block of pixels is represented as a matrix (act 403). The block of pixels may either be represented by a “spatial domain” matrix or by a “transform domain” matrix. A spatial domain matrix of a block of pixels means that the element values of the matrix are specific pixel values that are laid out spatially in the matrix according to the position of the corresponding pixel in the block. Thus, the element in row 3, column 2 of the spatial domain matrix represents a pixel value corresponding to a pixel in row 3, column 2 of the block of pixels. A transform domain matrix of a block of pixels is less intuitive and is represented by performing a transform on the spatial domain matrix. Each element in the transform domain matrix represents a discrete transform relationship between the pixel values in the spatial domain matrix. For example, if the transform domain matrix is a frequency domain matrix, one common discrete frequency relationship is defined by the wellknown Discrete Cosine Transform (DCT) operation. Before describing a suitable premultiplication matrix and postmultiplication matrix that are suitable for acts 404 and 405, respectively, the mathematical relationship between transform domain matrices and spatial domain matrices will now be briefly described followed by a mathematical description of how subsampling typically occurs.
A transform domain matrix A may be generated by performing premultiplication and postmultiplication on a corresponding spatial domain matrix P. This operation is represented in matrix form by the following equation 1:
1:
A=D×P×E (1)

 where,
 P is the spatial domain matrix corresponding to the transform domain matrix A;
 A is the transform domain matrix corresponding to the spatial domain matrix P;
 D is the transform matrix for the vertical direction; and
 E is the transform matrix for the horizontal direction.
If the spatial domain matrix P is, for example, an 8by8 matrix where each element represents a pixel component, the matrices D, E and A are also 8by8 matrices. There is no requirement that the matrices D and E be unitary or symmetric. Also there is no requirement that the D and E represent the same transform. In one case, D could represent a Discrete Cosine Transform (DCT) matrix in the vertical direction, while E represents D transpose (i.e., the DCT matrix in the horizontal direction). However, in another example D could represent a wavelet transform matrix in the vertical direction, while D represents the DCT matrix in the horizontal direction.
Conversely, the spatial domain matrix P may be generated from a transform domain matrix by performing an inverse matrix transform on the transform domain matrix. This inverse operation is represented in matrix form by the following equation 2:
P=D^{−1}×A×E^{−1} (2)
Subsampling of the spatial domain matrix P occurs by premultiplying the matrix P by a subsampling matrix that defines the subsampling in one direction such as when performing horizontal subsampling. The resulting subsampled matrix may then be postmultiplied by the transpose of another subsampling matrix that defines the subsampling in a substantially perpendicular direction as when performing vertical subsampling. This subsampling is performed on the spatial domain matrix P as illustrated by the following equation 3:
p=S×P×T′ (3)

 where,
 p is the subsampled spatial domain matrix of the spatial domain matrix P;
 S is the subsampling matrix that is used for horizontal subsampling; and
 T′ is the transpose of the subsampling matrix T that is used for vertical subsampling.
Rewriting equation 3 by substituting the value of matrix P from equation 2 results in the following equation 4:
p=S×D^{−1}×A×E^{−1}×T′ (4)
Each of these matrices may conceptually be split into four separate quadrants based on the subsample size. For example, the matrix A may be rewritten as the following equation 5:

 where,
 A_{TL }is a matrix component having a size that is proportional to the matrix A by the same ratio as the proportion of the subsampled picture to the original picture;
 A_{TR }is a matrix component that resides to the right of the matrix A_{TL};
 A_{BL }is a matrix component that resides below the matrix A_{TL}; and
 A_{BR }is a matrix component that resides below the matrix A_{TR }and to the right of the matrix A_{BL}.
Similarly, the top two components of the matrix may be combined and the bottom two components may be combined so that the matrix A is defined as in the following equation 6:

 where,
 A_{T }is a matrix component defined by the combination of A_{TL }and A_{TR}; and
 A_{B }is a matrix component defined by the combination of A_{BL }and A_{BR}.
For instance, if the matrix A is an 8 row by 8 column matrix, and the subsampling cuts each dimension size (horizontal and vertical) in half, the matrix component ATL would be a 4 row by 4 column matrix. Consequently, the other matrix components ATR, ABL, and ABR would also be 4 row by 4 column matrices. In this case, the matrix components AT and AB would each be 4 rows by 8 columns. This subsample ratio is used as an example in the following description although the present invention works with other subsampling ratios. For example, if the picture were subsampled by 75%, each 8 row by 8 column matrix would be reduced to a mere 2 row by 2 column matrix. In this latter case, the matrix component ATL would be a 2 row by 2 column matrix. Consequently, the matrix components ATR would be a 2 row by 6 column matrix, the matrix component ABL would be a 6 row by 2 column matrix, and the matrix component ABR would be a 6 row by 6 column matrix. In this case, the matrix component AT would be two rows by eight columns and the matrix component AB would be 6 rows by 8 columns. Subsampling may also occur in just one direction, horizontal or vertical, with no subsampling occurring in the other direction.
Thus, the size of the matrix components is important for the transcoder to know when performing the subsample transcoding in accordance with the present invention since the subsampling ratio used to perform subsample transcoding by the transcoder 345 should be the same as the subsampling ratio used to perform subsampled decoding at the video node 320. This knowledge may be inferred by the transcoder 440. For example, if the video stream corresponds to the reducedsize image of a pictureinpicture display, the reduced size image may always be a certain size (e.g., half the size for each dimension). Thus, the transcoder 440 may infer that if subsampled decoding is to occur at all, it is at a subsampling ratio of 50% in each direction.
Referring to
In accordance with the principles of the present invention, the transform domain matrix A is converted into a matrix a that has zero values in all but its upper left component aTL. Specifically, matrix a may be represented by the following equation 7.

 where,
 Z represents matrix components having zero values for all elements.
Since the matrix a has many zero values, coding methods such as Huffman variable length coding reduce the coded representation of the matrix a significantly as compared to the coded representation of the matrix A. Thus, the size of the coded video stream is significantly reduced when converting matrix A to the matrix a for each block in each picture of the video stream.
In accordance with the principles of the present invention, the matrix A is converted into the matrix a in such a manner that subsampled decoding of the matrix a results in the same pixel block (i.e., matrix p defined by equation 4) as subsampled decoding the matrix A. Specifically, the following equation 8 holds true:
S×D^{−1}×A×E^{−1}×T′=S×D^{−1}×a×E^{−1}×T′ (8)
As mentioned above, the matrix a only has the potential for nonzero elements in its upper left matrix component aTL. Thus, once one determines what the matrix component aTL should be, one has also determined what the matrix a should be. The inventors have discovered that an appropriate matrix component aTL that will cause equation 8 to be satisfied so that subsampled decoding of the matrix a results in substantially the same picture as subsampled decoding of the matrix A is defined by the following equation 9:
a_{TL}=(m1)^{−1}×S×P×T′×(n1)^{−1} (9)
In equation 9, the matrix (m1)^{−1}×S represents an example of the premultiplication matrix that the block matrix P is premultiplied by in act 404 of
In equation 9, the matrix P represents the spatial domain representation of a block of pixels and, in a typical example, is an 8 row by 8 column matrix.
The matrix S is the vertical subsampling matrix. For example, if each dimension of the picture is cut in half when subsampling, and the block P has 8 rows, the matrix S would be a 4 row by 8 column matrix.
The matrix T′ is the transpose of the matrix T. The matrix T is the horizontal subsampling matrix. For example, if each dimension of the picture is cut in half when subsampling, and the block P has 8 columns, the matrix T′ would be an 8 row by 4 column matrix.
The matrix (m1)−1 is the multiplication inverse of matrix m1 such that (m1)−1×m1=I where “I” is the identity matrix. Matrix m1 equals S×(D−1)left, where (D−1)left is the left portion of the inverse of D. As an illustrative but nonlimiting example of the dimension of m1, if S is a four row by eight column matrix and (D−1)left is an eight row by four column matrix, the matrix m1 and the matrix (m1)−1 are both four row by four column matrices.
The matrix (n1)−1 is the multiplication inverse of matrix n1 such that (n1)−1×n1=I where “I” is the identity matrix. Matrix n1 equals (E−1)top×T′. As an illustrative but nonlimiting example of the size of n1, if (E−1)top is a four row by eight column matrix and T′ is an eight row by four column matrix, the matrix n1 and the matrix (n1)−1 are both four row by four column matrices.
The dimension of the resulting premultiplication matrix (m1)−1×S by which the matrix P is premultiplied according to equation 9 is obtained as follows. The number of rows in this matrix is equal to the number of rows in S and the number of columns in this matrix is equal to the number of rows in P (the latter typically being 8).
The dimension of the resulting postmultiplication matrix T′×(n1)^{−1 }by which the matrix P is postmultiplied according to equation 9 is obtained as follows. The number of rows in this matrix is equal to the number of columns in P (which is typically 8), and the number of columns is equal to the number of rows in T.
To continue with the illustrative but nonlimiting example of the dimensions of the matrices involved, in this example, equation 9 results in the premultiplication of an eight row by eight column matrix by a four row by eight column premultiplication matrix, and in the postmultiplication of the eight row by eight column matrix by an eight row by four column premultiplication matrix. In this example, the result is a four row by four column matrix that constitutes the potential nonzero values of the eight row by eight column matrix a.
Referring to
Since the transcoded video stream is smaller after variable length coding than the original encoded video stream, less memory is required to store the video stream if the video stream is stored. Also, less network bandwidth is required to transmit the video stream if the video stream is transmitted over the network. According, the memory and network bandwidth needed to handle the video stream are reduced. The subsample transcoding described above would result in a loss of image quality if subsampled decoding was not to occur. However, if it is known that the video stream is to ultimately be subsample decoded, the video stream may be subsampled transcoded in accordance with the present invention with the assurance that the subsampled transcoding will result in no lost image quality after subsampled decoding. Accordingly, although some additional processing is required to perform the subsampled transcoding, the principles of the present invention allow for reduced memory and bandwidth requirements with no cost in terms of loss of video quality after subsampled transcoding.
The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
Claims
1. In a video management system configured to receive a video stream containing one or more video pictures that are each divided into blocks, wherein the video management system is to provide a representation of the one or more video pictures to a subsample decoder for subsampling, a method of reducing the size of the one or more frames with minimal, if any, effect on the video quality generated from the one or more frames after subsampling, the method comprising the following:
 accessing a video picture that is to be provided to a video node having a subsample decoder;
 obtaining information specifying a subsampling ratio to be performed by the subsample decoder; and
 in response to the obtained information, for at least one block of the video picture, performing subsample transcoding of the at least one block, thereby reducing the size of the at least one block to generate at least one reduced size block in such a way that subsampled decoding the at least one reduced size block results in substantially the same reduced size image as subsampled decoding the at least one block prior to the subsample transcoding, wherein reducing the size of the block includes the following: representing the block as a matrix; premultiplying the block matrix by a premultiplication matrix, the premultiplication matrix generated from a first subsample matrix that represents the subsampled decoding in a first direction; and postmultiplying the block matrix by a postmultiplication matrix generated from a second subsample matrix that represents the subsampled decoding in a second direction that is substantially perpendicular to the first direction.
2. The method in accordance with claim 1, wherein the method further comprises the following:
 providing the video picture with its one or more reduced size blocks to the subsample decoder.
3. The method in accordance with claim 1, wherein the method further comprises the following:
 subsampled decoding the video picture.
4. The method in accordance with claim 3, wherein the method further comprises the following:
 displaying the subsample decoded video picture on a display device.
5. The method in accordance with claim 3, further comprising the following:
 displaying the subsample decoded video picture on a display device as a reduced sized picture of a pictureinpicture display.
6. A computer system comprising a processor and storage media storing computerexecutable instructions which, when executed, perform the method recited in claim 1.
7. The method of claim 1, further comprising:
 receiving an indication over a network that the subsample decoder is to operate on the video stream.
8. The method of claim 1, further comprising:
 receiving the first and second subsample matrix over a network from the subsample decoder.
9. The method of claim 1, wherein the first subsample matrix is a vertical subsample matrix that represents the vertical subsampling that is to be performed by the subsample decoder, and wherein the second subsample matrix is a horizontal subsample matrix that represents the horizontal subsampling that is to be performed by the subsample decoder, the method further comprising the following:
 generating the premultiplication matrix by performing the following:
 determining a first inverse matrix that represents the multiplication inverse of a matrix that results from the multiplication of a first transform matrix times the vertical subsampling matrix; and
 multiplying the first inverse matrix by the vertical subsampling matrix to generate the premultiplication matrix.
10. The method in accordance with claim 9, wherein determining a first inverse matrix that represents the multiplication inverse of a matrix that results from the multiplication of a first transform matrix times the vertical subsampling matrix comprises the following:
 determining a first inverse matrix that represents the multiplication inverse of a matrix that results from the multiplication of a vertical discrete cosign transform matrix times the vertical subsampling matrix.
11. The method in accordance with claim 9, wherein determining a first inverse matrix that represents the multiplication inverse of a matrix that results from the multiplication of a first transform matrix times the first subsampling matrix comprises the following:
 determining a first inverse matrix that represents the multiplication inverse of a matrix that results from the multiplication of a vertical wavelet transform matrix times the first subsampling matrix.
12. The method in accordance with claim 9, further comprising the following:
 generating the postmultiplication matrix by performing the following:
 determining a second inverse matrix that represents the multiplication inverse of a matrix that results from the multiplication of a second transform matrix times the transpose of the horizontal subsampling matrix; and
 multiplying the transpose of the horizontal subsampling matrix by the second inverse matrix to generate the postmultiplication matrix.
13. The method in accordance with claim 12, wherein determining a second inverse matrix that represents the multiplication inverse of a matrix that results from the multiplication of a second transform matrix times the horizontal subsampling matrix comprises the following:
 determining a second inverse matrix that represents the multiplication inverse of a matrix that results from the multiplication of a horizontal discrete cosign transform matrix times the horizontal subsampling matrix.
14. The method in accordance with claim 12, wherein determining a second inverse matrix that represents the multiplication inverse of a matrix that results from the multiplication of a second transform matrix times the horizontal sub sampling matrix comprises the following:
 determining a second inverse matrix that represents the multiplication inverse of a matrix that results from the multiplication of a horizontal wavelet transform matrix times the horizontal subsampling matrix.
15. The method in accordance with claim 12, further comprising the following:
 horizontal subsampling the video picture using the horizontal subsampling matrix; and
 vertical subsampling the video picture using the vertical subsampling matrix.
16. The method in accordance with claim 1, wherein the at least one block of the video picture comprises all the blocks in the video picture.
17. The method in accordance with claim 1, wherein the act of accessing a video picture that is to be subsampled comprises the following:
 accessing a video frame that is to be subsampled.
18. The method in accordance with claim 1, wherein the act of accessing a video picture that is to be subsampled comprises the following:
 accessing a video frame that is to be subsampled.
19. A method as recited in claim 1, wherein the information specifying the subsampling ratio is provided by the video node.
20. A method as recited in claim 19, wherein the information specifying the subsampling ratio is provided upon determining that the subsampling ratio cannot be inferred.
21. A method as recited in claim 1, wherein the information specifying the subsampling ratio corresponds to a reducedsize in which the video picture will be displayed by the video node.
22. A method as recited in claim 1, wherein subsampled decoding of the at least one block causes the video image to be rendered without image loss that would otherwise occur if the image were to be rendered without the subsampled decoding.
23. A computerreadable storage media having stored computerexecutable instructions for implementing a method in a video management system configured to receive a video stream containing one or more video pictures that are each divided into blocks, wherein the video management system is to provide a representation of the one or more video pictures to a subsample decoder for subsampling, wherein the method is for reducing the size of the one or more frames with minimal, if any, effect on the video quality generated from the one or more frames after subsampling, the method comprising the following:
 accessing a video picture that is to be provided to a video node having a subsample decoder;
 obtaining information specifying a subsampling ratio to be performed by the subsample decoder; and
 in response to the obtained information, for at least one block of the video picture, performing subsample transcoding of the at least one block, thereby reducing the size of the at least one block to generate at least one reduced size block in such a way that subsampled decoding the at least one reduced size block results in substantially the same reduced size image as subsampled decoding the at least one block prior to the subsample transcoding, wherein reducing the size of the block includes the following: representing the block as a matrix: premultiplying the block matrix by a premultiplication matrix, the premultiplication matrix generated from a first subsample matrix that represents the subsampled decoding in a first direction; and postmultiplying the block matrix by a postmultiplication matrix generated from a second subsample matrix that represents the subsampled decoding in a second direction that is substantially perpendicular to the first direction.
Referenced Cited
U.S. Patent Documents
4942457  July 17, 1990  Keesen 
5193003  March 9, 1993  Kondo 
5253059  October 12, 1993  Ansari 
5949485  September 7, 1999  Oh 
6243421  June 5, 2001  Nakajima 
6310915  October 30, 2001  Wells et al. 
6333952  December 25, 2001  Lim et al. 
6563876  May 13, 2003  Boyce 
6671322  December 30, 2003  Vetro et al. 
7170932  January 30, 2007  Vetro et al. 
7263231  August 28, 2007  Jiang et al. 
Other references
 Yue Yu Jian Zhou and Chang Wen Chen, “A Fast Block Motion Estimation Algorithm Based on Combined Subsamplings on Pixels and Search Candidates” Image and Video Communications and Processing 2000, Proceedings of SPIE, Vo. 3974 (2000), pp. 835843.
 Myung Jun Kim, Byung Cheol Song, Sung Kyu Jang, and Jong Beom Ra, “An Efficient Video Down Conversation Algorithm Using Modified IDCT Basis Functions” 0780354672/99@ 1999 IEEE, pp. 914918.
 Robert Mokry and Dimitris Anastassiou, “Minimal Error Drift in Frequency Scalability for MotionCompensating DCT Coding”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 4, No. 4, Aug. 1994, pp. 392406.
 Office Action dated Aug. 4, 2004 cited in U.S. Appl. No. 09/886,693.
 Notice of Allowance dated Dec. 1, 2004 cited in U.S. Appl. No. 09/886,693.
Patent History
Type: Grant
Filed: Oct 19, 2004
Date of Patent: Dec 1, 2009
Patent Publication Number: 20050053153
Assignee: Microsoft Corporation (Redmond, WA)
Inventors: Shankar Moni (San Jose, CA), John A. Tardif (San Jose, CA)
Primary Examiner: Gims S Phillippe
Attorney: Workman Nydegger
Application Number: 10/968,529
Classifications
International Classification: H04N 7/12 (20060101); G06K 9/36 (20060101);