Patents by Inventor Yaowu Xu

Yaowu Xu has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11405631
    Abstract: Decoding a current frame includes identifying a first reference frame and a second reference frame for decoding the current frame; storing reference motion vectors of reference blocks of the first reference frame, where other reference frames are used to decode the first reference frame; identifying motion trajectories that pass through the current frame by projecting the reference motion vectors of the reference blocks of the first reference frame onto the current frame using at least a third reference frame of the other reference frames, where the projecting identifies, for a first current block of the current frame a corresponding first reference block in the first reference frame, and a corresponding reference motion vector of the reference motion vectors is associated with the corresponding first reference block; and projecting the corresponding reference motion vector onto the second reference frame to obtain a second reference block in the second reference frame.
    Type: Grant
    Filed: August 3, 2020
    Date of Patent: August 2, 2022
    Assignee: GOOGLE LLC
    Inventors: Jingning Han, Yaowu Xu, James Bankoski, Jia Feng
  • Patent number: 11405646
    Abstract: A processor is configured to maintain, for encoding current values related to the transform coefficients a first line buffer and a second line buffer. The current values are arranged along a current scan-order anti-diagonal line. The first line buffer includes first values of a first scan-order anti-diagonal line. The second line buffer includes second values of a second scan-order anti-diagonal line. The processor is further configured to interleave the first values and the second values in a destination buffer; select, using the destination buffer, a probability distribution for coding a current value of the current values; entropy encode, in a compressed bitstream, the current value using the probability distribution; and replace, for coding values of an immediately subsequent scan-order anti-diagonal line to the current scan-order anti-diagonal line, one of the second line buffer or the first line buffer with the current scan-order anti-diagonal line.
    Type: Grant
    Filed: November 30, 2020
    Date of Patent: August 2, 2022
    Assignee: GOOGLE LLC
    Inventors: Jingning Han, James Zern, Linfeng Zhang, Ching-Han Chiang, Yaowu Xu
  • Patent number: 11375240
    Abstract: Video coding using a constructed reference frame may include generating, by a processor in response to instructions stored on a non-transitory computer readable medium, an encoded video and outputting an output bitstream. Generating the encoded video includes receiving an input video stream, generating a constructed reference frame, generating an encoded constructed reference frame by encoding the constructed reference frame, including the encoded constructed reference frame in an output bitstream such that the constructed reference frame is a non-showable frame, generating an encoded frame by encoding a current frame from the input video stream using the constructed reference frame as a reference frame, and including the encoded frame in the output bitstream.
    Type: Grant
    Filed: December 17, 2018
    Date of Patent: June 28, 2022
    Assignee: GOOGLE LLC
    Inventors: James Bankoski, Paul Wilkins, Yaowu Xu
  • Patent number: 11350102
    Abstract: Decoding a current block of a current frame includes selecting a first reference frame for forward inter prediction of the current frame; selecting a second reference frame for backward inter prediction of the current frame; generating an optical flow reference frame portion for inter prediction of the current block by performing an optical flow estimation using the first reference frame and the second reference frame, where the optical flow estimation produces a respective motion field for pixels of the current block; and performing a prediction process for the current block using the optical flow reference frame portion by: using a motion vector used to encode the current block to identify a reference block; adjusting boundaries of the reference block using a subpixel interpolation filter length; and identifying blocks encompassing pixels within the adjusted boundaries of the reference block.
    Type: Grant
    Filed: May 5, 2020
    Date of Patent: May 31, 2022
    Assignee: GOOGLE LLC
    Inventors: Yaowu Xu, Bohan Li, Jingning Han
  • Publication number: 20220132116
    Abstract: Video coding may include identifying an input frame from an input video stream, generating, by a processor, an output bitstream by encoding the input frame, wherein encoding the input frame includes dividing the input frame into at least a first contiguous area of the input frame and a second contiguous area of the input frame, generating first encoded data by encoding the first contiguous area of the input frame, generating second encoded data by encoding the second contiguous area of the input frame, wherein encoding the second contiguous area of the input frame is performed independently of encoding the first contiguous area of the input frame, and including the first encoded data and the second encoded data in the output bitstream, and outputting the output bitstream.
    Type: Application
    Filed: January 7, 2022
    Publication date: April 28, 2022
    Inventors: Eric Ameres, James Bankoski, Scott Lavarnway, Yaowu Xu, Dan Miller, Adrian Grange, Paul Wilkins
  • Patent number: 11284107
    Abstract: An optical flow reference frame is generated that can be used for inter prediction of blocks of a current frame in a video sequence. A first (e.g., forward) reference frame and a second (e.g., backward reference frame) are used in an optical flow estimation that produces a respective motion field for pixels of the current frame. The motion fields are used to warp the reference frames to the current frame. The warped reference frames are blended to forms the optical flow reference frame.
    Type: Grant
    Filed: August 22, 2017
    Date of Patent: March 22, 2022
    Assignee: GOOGLE LLC
    Inventors: Yaowu Xu, Bohan Li, Jingning Han
  • Patent number: 11240498
    Abstract: Video coding may include identifying an input frame from an input video stream, generating, by a processor, an output bitstream by encoding the input frame, wherein encoding the input frame includes dividing the input frame into at least a first contiguous area of the input frame and a second contiguous area of the input frame, generating first encoded data by encoding the first contiguous area of the input frame, generating second encoded data by encoding the second contiguous area of the input frame, wherein encoding the second contiguous area of the input frame is performed independently of encoding the first contiguous area of the input frame, and including the first encoded data and the second encoded data in the output bitstream, and outputting the output bitstream.
    Type: Grant
    Filed: March 24, 2020
    Date of Patent: February 1, 2022
    Assignee: GOOGLE LLC
    Inventors: Eric Ameres, James Bankoski, Scott Lavarnway, Yaowu Xu, Dan Miller, Adrian Grange, Paul Wilkins
  • Publication number: 20220014744
    Abstract: Generating, by a processor in response to instructions stored on a non-transitory computer readable medium, a reconstructed frame, may include generating a reconstructed block of the reconstructed frame by decoding from an encoded bitstream. Decoding may include decoding a value from the encoded bitstream, identifying, in accordance with the value, a probability distribution for generating the reconstructed block, wherein the value indicates the probability distribution among a plurality of probability distributions determined independently of generating the reconstructed frame, entropy decoding an encoded prediction mode from the encoded bitstream using the probability distribution to identify a prediction mode for generating the reconstructed block, generating a prediction block in accordance with the prediction mode; combining the prediction block and a reconstructed residual block to obtain the reconstructed block, and including the reconstructed block in the reconstructed frame.
    Type: Application
    Filed: June 7, 2021
    Publication date: January 13, 2022
    Inventors: Yaowu Xu, Paul Gordon Wilkins, James Bankoski
  • Patent number: 11218737
    Abstract: Asymmetric probability model updating and entropy coding includes using different numbers of bits for storing probabilities of a probability model and for entropy coding symbols using that probability model. The probabilities of a probability model are updated according to values of syntax elements decoded from a bitstream. The probabilities are associated with possible values of the syntax elements and are stored using a first bit precision. Based on the updated probabilities, a second bit precision to use to entropy decode the syntax elements is determined. The second bit precision is less than the first bit precision. The syntax elements are then entropy decoded using the second bit precision, such as to produce quantized transform coefficients, which may be further processed and output to an output video stream. Using the first bit precision to entropy decode the syntax elements results in a lower compression throughput than using the second bit precision.
    Type: Grant
    Filed: July 23, 2018
    Date of Patent: January 4, 2022
    Assignee: GOOGLE LLC
    Inventors: Jingning Han, Aki Kuusela, Stanislav Vitvitskyy, Yaowu Xu, Ching-Han Chiang, Dake He
  • Patent number: 11153588
    Abstract: An apparatus for video coding using dual deblocking filter thresholds may include a processor generating a reconstructed frame by decoding an encoded bitstream and outputting the reconstructed frame. Decoding may include generating a decoded block by decoding a portion of the encoded bitstream, identifying a first deblocking threshold index from the encoded bitstream, identifying a second deblocking threshold index from the encoded bitstream, generating a reconstructed block based on the decoded block, and including the reconstructed block in the reconstructed frame. Generating the reconstructed block may include deblocking based on the first deblocking threshold index and the second deblocking threshold index.
    Type: Grant
    Filed: April 2, 2020
    Date of Patent: October 19, 2021
    Assignee: GOOGLE LLC
    Inventors: Jingning Han, Yaowu Xu
  • Patent number: 11102477
    Abstract: A sign value of a DC coefficient of a current block is determined using neighbor blocks of the current block. First and second sign values are identified as respectively corresponding to an above neighbor block of the current block and a left neighbor block of the current block. A context value is calculated by applying a first weighting value to the first sign value and a second weighting value to the second sign value. The first weighting value is based on a boundary between the above neighbor block and the current block, and the second weighting value is based on a boundary between the left neighbor block and the current block. A probability value is selected based on the context value. The sign value of the DC coefficient of the current block is then determined using the probability model.
    Type: Grant
    Filed: April 2, 2020
    Date of Patent: August 24, 2021
    Assignee: Google LLC
    Inventors: Yaowu Xu, Jingning Han, Ching-Han Chiang
  • Patent number: 11039138
    Abstract: A system, apparatus, and method for encoding and decoding a video image having a plurality of frames is disclosed. Encoding and decoding the video image can include selecting, for a current block, a prediction mode from a plurality of prediction modes; identifying, for the current block, a quantization value; selecting, for the current block, a probability distribution from a plurality of probability distributions based on the identified quantization value using a processor; and entropy encoding the selected prediction mode using the selected probability distribution.
    Type: Grant
    Filed: March 8, 2012
    Date of Patent: June 15, 2021
    Assignee: GOOGLE LLC
    Inventors: Yaowu Xu, Paul Gordon Wilkins, James Bankoski
  • Patent number: 11025950
    Abstract: A motion-field based reference frame is rendered that can be used to perform a prediction process for a first frame of a video sequence to be predicted. A first reference frame from the video sequence for forward inter prediction of the first frame is determined, as is a second reference frame from the video sequence for backward inter prediction of the first frame. A respective motion field determined for blocks of the first frame is used to determine parameters forming an affine homographic model. A current block of a reconstructed reference frame is rendered at a co-located position within a motion field-based reference frame by applying the affine homographic model to the current block. An affine homographic model may be determined for each reconstructed reference frame block to render the motion-field based reference frame.
    Type: Grant
    Filed: February 12, 2018
    Date of Patent: June 1, 2021
    Assignee: GOOGLE LLC
    Inventors: Yaowu Xu, Jingning Han, Ching-Han Chiang
  • Publication number: 20210144364
    Abstract: A motion field estimate determined using motion vector information of two or more reference frames of a current/encoded frame is used to derive a motion vector for inter-prediction of the current/encoded frame. Motion trajectory information, including concatenated motion vectors and locations of the current/encoded frame at which those concatenated motion vectors point, is determined by concatenating motion vectors of the reference frames. A motion field estimate is determined using the motion trajectory information and, in some cases, by interpolating unavailable motion vectors using neighbors. The motion field estimate is used to determine a co-located reference frame for the current/encoded frame, and an inter-prediction process is performed for the current/encoded frame using a motion vector derived using the co-located reference frame.
    Type: Application
    Filed: November 5, 2020
    Publication date: May 13, 2021
    Inventors: Bohan Li, Yaowu Xu, Jingning Han
  • Patent number: 10986361
    Abstract: Decoding a video sequence may include generating decoded video data by decoding encoded video data from an encoded bitstream, wherein decoding the encoded video data includes identifying candidate reference motion vectors used to decode previously decoded blocks, identifying reconstructed pixel values from a block spatially adjacent to a current block in the current frame, for each candidate reference motion vector, determining a respective score based on a difference between the reconstructed pixel values and pixel values obtained using the candidate reference motion vector, identifying a candidate reference motion vector having the minimal score from the candidate reference motion vectors as the reference motion vector, decoding a motion vector from the encoded bitstream using the reference motion vector, and decoding the current block from the encoded bitstream using the motion vector.
    Type: Grant
    Filed: June 28, 2019
    Date of Patent: April 20, 2021
    Assignee: GOOGLE LLC
    Inventors: Adrian William Grange, Ronald Sebastiaan Bultje, Yaowu Xu, Paul Gordon Wilkins
  • Publication number: 20210112270
    Abstract: Dynamic motion vector referencing is used to predict motion within video blocks. A motion trajectory is determined for a current frame including a video block to encode or decode based on a reference motion vector used for encoding or decoding one or more reference frames of the current frame. One or more temporal motion vector candidates are then determined for predicting motion within the video block based on the motion trajectory. A motion vector is selected from a motion vector candidate list including the one or more temporal motion vector candidates and used to generate a prediction block. The prediction block is then used to encode or decode the video block. The motion trajectory is based on an order of video frames indicated by frame offset values encoded to a bitstream. The motion vector candidate list may include one or more spatial motion vector candidates.
    Type: Application
    Filed: December 23, 2020
    Publication date: April 15, 2021
    Inventors: Jingning Han, James Bankoski, Yaowu Xu
  • Publication number: 20210084336
    Abstract: A processor is configured to maintain, for encoding current values related to the transform coefficients a first line buffer and a second line buffer. The current values are arranged along a current scan-order anti-diagonal line. The first line buffer includes first values of a first scan-order anti-diagonal line. The second line buffer includes second values of a second scan-order anti-diagonal line. The processor is further configured to interleave the first values and the second values in a destination buffer; select, using the destination buffer, a probability distribution for coding a current value of the current values; entropy encode, in a compressed bitstream, the current value using the probability distribution; and replace, for coding values of an immediately subsequent scan-order anti-diagonal line to the current scan-order anti-diagonal line, one of the second line buffer or the first line buffer with the current scan-order anti-diagonal line.
    Type: Application
    Filed: November 30, 2020
    Publication date: March 18, 2021
    Inventors: Jingning Han, James Zern, Linfeng Zhang, Ching-Han Chiang, Yaowu Xu
  • Patent number: 10951885
    Abstract: A method for using an alternate frame reference (ARF) includes selecting an anchor frame and video frames, where the anchor frame includes an anchor block, and the anchor block includes an anchor pixel; identifying, for the anchor block of the anchor frame, respective reference blocks in the video frames; determining, for the anchor pixel and using an anchor patch, respective distances between the anchor pixel and respective co-located reference pixels of the respective reference blocks, where the anchor patch includes anchor patch pixels, and a respective distance, of the respective distances, between the anchor pixel and a respective co-located reference pixel is determined using the anchor patch pixels and co-located reference pixels; determining, using the respective distances, respective weights; determining, using the respective weights, an ARF pixel that is co-located with the anchor pixel; and encoding, in a compressed bitstream, the ARF.
    Type: Grant
    Filed: August 30, 2018
    Date of Patent: March 16, 2021
    Assignee: GOOGLE LLC
    Inventors: Jingning Han, Paul Wilkins, Yaowu Xu
  • Patent number: 10951921
    Abstract: Generating encoded image data using adjustable per-symbol entropy coding probability updating may include generating a portion of the encoded image data in accordance with a value of a probability update indicator for the portion indicating whether per-symbol entropy coding probability updating is disabled for the portion, and including the value of the probability update indicator and the entropy coded image data in an output bitstream. Generating decoded image data using adjustable per-symbol entropy coding probability updating may include obtaining a value of a probability update indicator for a portion of the decoded image data, the value of the probability update indicator for the portion indicating whether per-symbol entropy coding probability updating is disabled for the portion, and generating decoded image data for the portion in accordance with the value of the probability update indicator for the portion.
    Type: Grant
    Filed: January 30, 2020
    Date of Patent: March 16, 2021
    Assignee: GOOGLE LLC
    Inventors: Yaowu Xu, Hui Su
  • Patent number: 10880573
    Abstract: Dynamic motion vector referencing is used to predict motion within video blocks. A motion trajectory is determined for a current frame including a video block to encode or decode based on a reference motion vector used for encoding or decoding one or more reference frames of the current frame. One or more temporal motion vector candidates are then determined for predicting motion within the video block based on the motion trajectory. A motion vector is selected from a motion vector candidate list including the one or more temporal motion vector candidates and used to generate a prediction block. The prediction block is then used to encode or decode the video block. The motion trajectory is based on an order of video frames indicated by frame offset values encoded to a bitstream. The motion vector candidate list may include one or more spatial motion vector candidates.
    Type: Grant
    Filed: August 15, 2017
    Date of Patent: December 29, 2020
    Assignee: GOOGLE LLC
    Inventors: Jingning Han, James Bankoski, Yaowu Xu