Patents by Inventor Jingning Han

Jingning Han has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10771803
    Abstract: A method for decoding an encoded block of an encoded frame includes selecting motion vectors corresponding to blocks of an extended collocated processing unit in a first reference frame; identifying a block of the blocks, such that the encoded block is a projection, using a motion vector of the block that refers to a third reference frame, onto the encoded frame; on condition that a block is identified, determining a temporal motion vector candidate for the encoded block in a second reference frame using a motion vector of the identified block and respective display orders of the encoded frame, the first reference frame, the second reference frame, and the third reference frame, and adding the temporal motion vector candidate to a motion vector candidate list; selecting a motion vector from the motion vector candidate list; generating a prediction block using the selected motion vector for decoding the encoded block.
    Type: Grant
    Filed: April 9, 2019
    Date of Patent: September 8, 2020
    Assignee: GOOGLE LLC
    Inventors: Jingning Han, Yaowu Xu, James Bankoski, Jia Feng
  • Publication number: 20200275098
    Abstract: A memory-efficient filtering approach is used to code images and video. A buffer having a fixed size based on a size of processing units to use for filtering a video frame is allocated. For each of the processing units, pre-filtered pixel values are copied from a respective region of the video frame to the buffer based on a writing point for the video frame and an offset applied to the writing point, filtering is performed against the pre-filtered pixel values from the buffer to produce filtered pixel values, and the filtered pixel values are written to the video frame based on the writing point and the offset. The filtering may be performed using a loop restoration tool, such as where the pre-filtered pixel values are output from a constrained directional enhancement filter (CDEF) tool. Alternatively, the filtering may be performed using the CDEF tool or another coding tool.
    Type: Application
    Filed: April 27, 2020
    Publication date: August 27, 2020
    Inventors: Cheng Chen, Jingning Han
  • Publication number: 20200267391
    Abstract: Decoding a current block of a current frame includes selecting a first reference frame for forward inter prediction of the current frame; selecting a second reference frame for backward inter prediction of the current frame; generating an optical flow reference frame portion for inter prediction of the current block by performing an optical flow estimation using the first reference frame and the second reference frame, where the optical flow estimation produces a respective motion field for pixels of the current block; and performing a prediction process for the current block using the optical flow reference frame portion by: using a motion vector used to encode the current block to identify a reference block; adjusting boundaries of the reference block using a subpixel interpolation filter length; and identifying blocks encompassing pixels within the adjusted boundaries of the reference block.
    Type: Application
    Filed: May 5, 2020
    Publication date: August 20, 2020
    Inventors: Yaowu Xu, Bohan Li, Jingning Han
  • Patent number: 10750171
    Abstract: Systems and methods are disclosed for encoding and decoding video. For example, methods may include: accessing an encoded bitstream; reconstructing an image including multiple color planes based on data from the encoded bitstream; decoding a first filter level from the encoded bitstream, wherein the first filter level specifies one or more thresholds that are used to select a length for a deblocking filter; decoding a second filter level from the encoded bitstream, wherein the second filter level specifies one or more thresholds that are used to select a length for a deblocking filter; after reconstruction of the image, applying a deblocking filter to a first color plane of the image using the first filter level; and, after reconstruction of the image, applying a deblocking filter to a second color plane of the image using the second filter level.
    Type: Grant
    Filed: June 25, 2018
    Date of Patent: August 18, 2020
    Assignee: GOOGLE LLC
    Inventors: Cheng Chen, Jingning Han, Yaowu Xu
  • Patent number: 10735767
    Abstract: Encoding a transform block includes de-composing transform coefficients of the transform block into binary level maps arranged in a tier and a residual transform map, the binary level maps formed by breaking down a value of a respective transform coefficient into a series of binary decisions; and encoding, using a context model, a to-be-encoded binary decision that is at a scan location in a scan order, the to-be-encoded binary decision being a value of a binary level map at a level k. The context model is selected using first neighboring binary decisions of the binary level map at a level k that precede the to-be-encoded binary decision; and second neighboring binary decisions of a binary level map at a level (k?1), the second neighboring binary decisions including values that precede and values that follow, in the scan order, a co-located binary decision of the to-be-encoded binary decision.
    Type: Grant
    Filed: March 12, 2019
    Date of Patent: August 4, 2020
    Assignee: GOOGLE LLC
    Inventors: Yaowu Xu, Jingning Han, Ching-Han Chiang
  • Publication number: 20200236350
    Abstract: A sign value of a DC coefficient of a current block is determined using neighbor blocks of the current block. First and second sign values are identified as respectively corresponding to an above neighbor block of the current block and a left neighbor block of the current block. A context value is calculated by applying a first weighting value to the first sign value and a second weighting value to the second sign value. The first weighting value is based on a boundary between the above neighbor block and the current block, and the second weighting value is based on a boundary between the left neighbor block and the current block. A probability value is selected based on the context value. The sign value of the DC coefficient of the current block is then determined using the probability model.
    Type: Application
    Filed: April 2, 2020
    Publication date: July 23, 2020
    Inventors: Yaowu Xu, Jingning Han, Ching-Han Chiang
  • Publication number: 20200236375
    Abstract: An apparatus for video coding using dual deblocking filter thresholds may include a processor generating a reconstructed frame by decoding an encoded bitstream and outputting the reconstructed frame. Decoding may include generating a decoded block by decoding a portion of the encoded bitstream, identifying a first deblocking threshold index from the encoded bitstream, identifying a second deblocking threshold index from the encoded bitstream, generating a reconstructed block based on the decoded block, and including the reconstructed block in the reconstructed frame. Generating the reconstructed block may include deblocking based on the first deblocking threshold index and the second deblocking threshold index.
    Type: Application
    Filed: April 2, 2020
    Publication date: July 23, 2020
    Inventors: Jingning Han, Yaowu Xu
  • Patent number: 10701398
    Abstract: Video coding using a context adaptive scan order for entropy coding may include an apparatus decoding a current block by identifying a context adaptive scan order for entropy decoding a transform block, which may include identifying non-zero-coefficient probabilities for the transform block such that each location in the transform block corresponds to a respective non-zero-coefficient probability from the non-zero-coefficient probabilities, assigning a respective context adaptive scan order position to each location in the transform block in descending magnitude order of the respective corresponding non-zero-coefficient probabilities such that the context adaptive scan order position for each location exceeds the context adaptive scan order position assigned to entropy coding context locations for the respective location, entropy decoding transform coefficients from the encoded video stream based on the context adaptive scan order, and reconstructing the decoded block based on the transform block.
    Type: Grant
    Filed: June 25, 2019
    Date of Patent: June 30, 2020
    Assignee: GOOGLE LLC
    Inventors: Jingning Han, Ching-Han Chiang
  • Patent number: 10694180
    Abstract: Entropy coding of transform partitioning information may include decoding a current block by determining a probability for entropy decoding a transform partitioning flag based on a transform block size of an adjacent block that is available for decoding, entropy decoding the transform partitioning flag for the current block using the probability, generating a reconstructed block based on the current block by, on a condition that the transform partitioning flag indicates that transform partitioning is omitted for the current block, inverse transforming the current block using a current block size inverse transform, and, on a condition that the transform partitioning flag indicates transform partitioning for the current block, identifying sub-blocks from the current block, wherein each sub-block from the sub-blocks has a sub-block size smaller than the current block size, inverse transforming the sub-blocks using sub-block size inverse transforms, and generating the reconstructed block based on the residual blo
    Type: Grant
    Filed: March 29, 2018
    Date of Patent: June 23, 2020
    Assignee: GOOGLE LLC
    Inventors: Jingning Han, Yaowu Xu
  • Patent number: 10659788
    Abstract: An optical flow reference frame portion (e.g., a block or an entire frame) is generated that can be used for inter prediction of blocks of a current frame in a video sequence. A forward reference frame and a backward reference frame are used in an optical flow estimation that produces a respective motion field for pixels of a current frame. The motion fields are used to warp some or all pixels of the reference frames to the pixels of the current frame. The warped reference frame pixels are blended to form the optical flow reference frame portion. The inter prediction may be performed as part of encoding or decoding portions of the current frame.
    Type: Grant
    Filed: November 20, 2017
    Date of Patent: May 19, 2020
    Assignee: GOOGLE LLC
    Inventors: Yaowu Xu, Bohan Li, Jingning Han
  • Patent number: 10652535
    Abstract: A memory-efficient filtering approach is used to code images and video. A buffer having a fixed size based on a size of processing units to use for filtering a video frame is allocated. For each of the processing units, pre-filtered pixel values are copied from a respective region of the video frame to the buffer based on a writing point for the video frame and an offset applied to the writing point, filtering is performed against the pre-filtered pixel values from the buffer to produce filtered pixel values, and the filtered pixel values are written to the video frame based on the writing point and the offset. The filtering may be performed using a loop restoration tool, such as where the pre-filtered pixel values are output from a constrained directional enhancement filter (CDEF) tool. Alternatively, the filtering may be performed using the CDEF tool or another coding tool.
    Type: Grant
    Filed: February 27, 2019
    Date of Patent: May 12, 2020
    Assignee: GOOGLE LLC
    Inventors: Cheng Chen, Jingning Han
  • Patent number: 10645408
    Abstract: Video coding using dual deblocking filter thresholds may include generating a reconstructed frame by decoding an encoded bitstream and outputting the reconstructed frame. Decoding may include generating a decoded block by decoding a portion of the encoded bitstream, identifying a first deblocking threshold index from the encoded bitstream, identifying a second deblocking threshold index from the encoded bitstream, generating a reconstructed block based on the decoded block, and including the reconstructed block in the reconstructed frame. Generating the reconstructed block may include deblocking based on the first deblocking threshold index and the second deblocking threshold index.
    Type: Grant
    Filed: September 17, 2017
    Date of Patent: May 5, 2020
    Assignee: GOOGLE LLC
    Inventors: Jingning Han, Yaowu Xu
  • Patent number: 10623731
    Abstract: A sign value of a DC coefficient of a current block is determined using neighbor blocks of the current block. First and second sign values are identified as respectively corresponding to an above neighbor block of the current block and a left neighbor block of the current block. A context value is calculated by applying a first weighting value to the first sign value and a second weighting value to the second sign value. The first weighting value is based on a boundary between the above neighbor block and the current block, and the second weighting value is based on a boundary between the left neighbor block and the current block. A probability value is selected based on the context value. The sign value of the DC coefficient of the current block is then determined using the probability model.
    Type: Grant
    Filed: January 10, 2019
    Date of Patent: April 14, 2020
    Assignee: GOOGLE LLC
    Inventors: Yaowu Xu, Jingning Han, Ching-Han Chiang
  • Publication number: 20200099956
    Abstract: A memory of an apparatus includes instructions executable by a processor to select a backward scan order for traversing, along scan-order anti-diagonal lines, a block of values related to the transform coefficients; select a template for entropy-coding the values, the template indicating, for a to-be-coded value, scan positions of already coded values, the scan positions arranged in at least two template anti-diagonal lines; maintain, for coding first values, where the first values are arranged along a current scan-order anti-diagonal line, a first line buffer and a second line buffer; for a current value of the current scan-order anti-diagonal line, determine a context using the first line buffer and the second line buffer, and entropy encode, in a compressed bitstream, the current value using the context; and replace one of the second line buffer or the first line buffer with the current scan-order anti-diagonal line.
    Type: Application
    Filed: November 25, 2019
    Publication date: March 26, 2020
    Inventors: Jingning Han, James Zern, Linfeng Zhang, Ching-Han Chiang, Yaowu Xu
  • Patent number: 10602176
    Abstract: Decoding a video stream may include decoding a first block of a current frame by decoding a first motion vector from the encoded video stream, decoding an identifier of a first interpolation filter from the encoded video stream, and reconstructing the first block using the first motion vector and the first interpolation filter. Decoding a second block of the current frame may include identifying the first motion vector from the first block as a selected motion vector for predicting the second block in response to decoding an inter-prediction mode identifier for decoding the second block, identifying the first interpolation filter as a selected interpolation filter for predicting the second block in response to identifying the first motion vector from the first block as the selected motion vector for predicting the second block, and reconstructing the second block using the first motion vector and the first interpolation filter.
    Type: Grant
    Filed: April 15, 2016
    Date of Patent: March 24, 2020
    Assignee: GOOGLE LLC
    Inventors: Yaowu Xu, Jingning Han
  • Publication number: 20200092576
    Abstract: Video coding may include generating, by a processor executing instructions stored on a non-transitory computer-readable medium, an encoded frame by encoding a current frame from an input bitstream, by generating a reference coframe spatiotemporally corresponding to the current frame, wherein the current frame is a frame from a sequence of input frames, wherein each frame from the sequence of input frames has a respective sequential location in the sequence of input frames, and wherein the current frame has a current sequential location in the sequence of input frames, and encoding the current frame using the reference coframe. Video coding may include including the encoded frame in an output bitstream and outputting the output bitstream.
    Type: Application
    Filed: September 14, 2018
    Publication date: March 19, 2020
    Inventors: Bohan Li, Yaowu Xu, Jingning Han
  • Publication number: 20200077083
    Abstract: A method for using an alternate frame reference (ARF) includes selecting an anchor frame and video frames, where the anchor frame includes an anchor block, and the anchor block includes an anchor pixel; identifying, for the anchor block of the anchor frame, respective reference blocks in the video frames; determining, for the anchor pixel and using an anchor patch, respective distances between the anchor pixel and respective co-located reference pixels of the respective reference blocks, where the anchor patch includes anchor patch pixels, and a respective distance, of the respective distances, between the anchor pixel and a respective co-located reference pixel is determined using the anchor patch pixels and co-located reference pixels; determining, using the respective distances, respective weights; determining, using the respective weights, an ARF pixel that is co-located with the anchor pixel; and encoding, in a compressed bitstream, the ARF.
    Type: Application
    Filed: August 30, 2018
    Publication date: March 5, 2020
    Inventors: Jingning Han, Paul Wilkins, Yaowu Xu
  • Patent number: 10567772
    Abstract: Methods and apparatuses for coding a current block are disclosed. An apparatus for decoding a current block of size 8×8 including a memory and a processor is also disclosed. The processor configured to execute instructions stored in the memory to generate, for the current block, a luma block and a chroma block, divide the luma block into luma sub-blocks, and determine a respective prediction mode for each luma sub-block. The chroma block is of size 4×4, each luma sub-block is of size 4×4, and the respective prediction mode is an intra-mode or an inter-mode. The instructions also include instructions to, on condition that each of the respective prediction modes is the inter-mode, divide the chroma block into 2×2 chroma sub-blocks and determine a respective motion vector for each chroma sub-block.
    Type: Grant
    Filed: July 11, 2017
    Date of Patent: February 18, 2020
    Assignee: GOOGLE LLC
    Inventors: Yaowu Xu, Jingning Han
  • Publication number: 20200029098
    Abstract: Asymmetric probability model updating and entropy coding includes using different numbers of bits for storing probabilities of a probability model and for entropy coding symbols using that probability model. The probabilities of a probability model are updated according to values of syntax elements decoded from a bitstream. The probabilities are associated with possible values of the syntax elements and are stored using a first bit precision. Based on the updated probabilities, a second bit precision to use to entropy decode the syntax elements is determined. The second bit precision is less than the first bit precision. The syntax elements are then entropy decoded using the second bit precision, such as to produce quantized transform coefficients, which may be further processed and output to an output video stream. Using the first bit precision to entropy decode the syntax elements results in a lower compression throughput than using the second bit precision.
    Type: Application
    Filed: July 23, 2018
    Publication date: January 23, 2020
    Inventors: Jingning Han, Aki Kuusela, Stanislav Vitvitskyy, Yaowu Xu, Ching-Han Chiang, Dake He
  • Patent number: 10531124
    Abstract: Multi-stage coding block partition search is disclosed. A method includes selecting a partition-none partition type and a partition-split partition type for predicting the block, determining a first cost of predicting the block using the partition-none partition type, and determining a second cost of predicting the block using the partition-split partition type. The partition-none partition type and the partition-split partition type are selected from a set of partition types that includes the partition-none partition type, the partition-split partition type, and third partition types. The method also includes, on condition that the result meets a criterion, determining a respective encoding cost corresponding to at least some of the third partition types; selecting a selected partition type corresponding to a minimal cost amongst the partition-none partition type and the at least some of the third partition types; and encoding, in a compressed bitstream, the selected partition type.
    Type: Grant
    Filed: June 25, 2018
    Date of Patent: January 7, 2020
    Assignee: GOOGLE LLC
    Inventors: Jingning Han, Yaowu Xu