Patents by Inventor Jingning Han
Jingning Han has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11102477Abstract: A sign value of a DC coefficient of a current block is determined using neighbor blocks of the current block. First and second sign values are identified as respectively corresponding to an above neighbor block of the current block and a left neighbor block of the current block. A context value is calculated by applying a first weighting value to the first sign value and a second weighting value to the second sign value. The first weighting value is based on a boundary between the above neighbor block and the current block, and the second weighting value is based on a boundary between the left neighbor block and the current block. A probability value is selected based on the context value. The sign value of the DC coefficient of the current block is then determined using the probability model.Type: GrantFiled: April 2, 2020Date of Patent: August 24, 2021Assignee: Google LLCInventors: Yaowu Xu, Jingning Han, Ching-Han Chiang
-
Patent number: 11095890Abstract: A memory-efficient filtering approach is used to code images and video. A buffer having a fixed size based on a size of processing units to use for filtering a video frame is allocated. For each of the processing units, pre-filtered pixel values are copied from a respective region of the video frame to the buffer based on a writing point for the video frame and an offset applied to the writing point, filtering is performed against the pre-filtered pixel values from the buffer to produce filtered pixel values, and the filtered pixel values are written to the video frame based on the writing point and the offset. The filtering may be performed using a loop restoration tool, such as where the pre-filtered pixel values are output from a constrained directional enhancement filter (CDEF) tool. Alternatively, the filtering may be performed using the CDEF tool or another coding tool.Type: GrantFiled: April 27, 2020Date of Patent: August 17, 2021Assignee: GOOGLE LLCInventors: Cheng Chen, Jingning Han
-
Patent number: 11025950Abstract: A motion-field based reference frame is rendered that can be used to perform a prediction process for a first frame of a video sequence to be predicted. A first reference frame from the video sequence for forward inter prediction of the first frame is determined, as is a second reference frame from the video sequence for backward inter prediction of the first frame. A respective motion field determined for blocks of the first frame is used to determine parameters forming an affine homographic model. A current block of a reconstructed reference frame is rendered at a co-located position within a motion field-based reference frame by applying the affine homographic model to the current block. An affine homographic model may be determined for each reconstructed reference frame block to render the motion-field based reference frame.Type: GrantFiled: February 12, 2018Date of Patent: June 1, 2021Assignee: GOOGLE LLCInventors: Yaowu Xu, Jingning Han, Ching-Han Chiang
-
Publication number: 20210144364Abstract: A motion field estimate determined using motion vector information of two or more reference frames of a current/encoded frame is used to derive a motion vector for inter-prediction of the current/encoded frame. Motion trajectory information, including concatenated motion vectors and locations of the current/encoded frame at which those concatenated motion vectors point, is determined by concatenating motion vectors of the reference frames. A motion field estimate is determined using the motion trajectory information and, in some cases, by interpolating unavailable motion vectors using neighbors. The motion field estimate is used to determine a co-located reference frame for the current/encoded frame, and an inter-prediction process is performed for the current/encoded frame using a motion vector derived using the co-located reference frame.Type: ApplicationFiled: November 5, 2020Publication date: May 13, 2021Inventors: Bohan Li, Yaowu Xu, Jingning Han
-
Publication number: 20210112270Abstract: Dynamic motion vector referencing is used to predict motion within video blocks. A motion trajectory is determined for a current frame including a video block to encode or decode based on a reference motion vector used for encoding or decoding one or more reference frames of the current frame. One or more temporal motion vector candidates are then determined for predicting motion within the video block based on the motion trajectory. A motion vector is selected from a motion vector candidate list including the one or more temporal motion vector candidates and used to generate a prediction block. The prediction block is then used to encode or decode the video block. The motion trajectory is based on an order of video frames indicated by frame offset values encoded to a bitstream. The motion vector candidate list may include one or more spatial motion vector candidates.Type: ApplicationFiled: December 23, 2020Publication date: April 15, 2021Inventors: Jingning Han, James Bankoski, Yaowu Xu
-
Publication number: 20210084336Abstract: A processor is configured to maintain, for encoding current values related to the transform coefficients a first line buffer and a second line buffer. The current values are arranged along a current scan-order anti-diagonal line. The first line buffer includes first values of a first scan-order anti-diagonal line. The second line buffer includes second values of a second scan-order anti-diagonal line. The processor is further configured to interleave the first values and the second values in a destination buffer; select, using the destination buffer, a probability distribution for coding a current value of the current values; entropy encode, in a compressed bitstream, the current value using the probability distribution; and replace, for coding values of an immediately subsequent scan-order anti-diagonal line to the current scan-order anti-diagonal line, one of the second line buffer or the first line buffer with the current scan-order anti-diagonal line.Type: ApplicationFiled: November 30, 2020Publication date: March 18, 2021Inventors: Jingning Han, James Zern, Linfeng Zhang, Ching-Han Chiang, Yaowu Xu
-
Patent number: 10951894Abstract: A scan order for encoding or decoding coefficients of a transform block is selected on a transform block-level. A set of candidate scan orders is processed by identifying end of block positions within the transform block for each of the candidate scan orders. Cost values are determined for each of the candidate scan orders to reflect a number of the coefficients of the transform block that are located before the respective end of block positions. In particular, a cost value for a candidate scan order reflects the number of zero-value coefficients located before the end of block position for that candidate scan order. One of the candidate scan orders is then selected based on those cost values. The selected scan order is used to scan the coefficients in the transform block, such as for encoding those coefficients to a bitstream or for decoding those coefficients to an output video stream.Type: GrantFiled: August 8, 2019Date of Patent: March 16, 2021Assignee: GOOGLE LLCInventors: Jingning Han, Dake He
-
Patent number: 10951885Abstract: A method for using an alternate frame reference (ARF) includes selecting an anchor frame and video frames, where the anchor frame includes an anchor block, and the anchor block includes an anchor pixel; identifying, for the anchor block of the anchor frame, respective reference blocks in the video frames; determining, for the anchor pixel and using an anchor patch, respective distances between the anchor pixel and respective co-located reference pixels of the respective reference blocks, where the anchor patch includes anchor patch pixels, and a respective distance, of the respective distances, between the anchor pixel and a respective co-located reference pixel is determined using the anchor patch pixels and co-located reference pixels; determining, using the respective distances, respective weights; determining, using the respective weights, an ARF pixel that is co-located with the anchor pixel; and encoding, in a compressed bitstream, the ARF.Type: GrantFiled: August 30, 2018Date of Patent: March 16, 2021Assignee: GOOGLE LLCInventors: Jingning Han, Paul Wilkins, Yaowu Xu
-
Patent number: 10880573Abstract: Dynamic motion vector referencing is used to predict motion within video blocks. A motion trajectory is determined for a current frame including a video block to encode or decode based on a reference motion vector used for encoding or decoding one or more reference frames of the current frame. One or more temporal motion vector candidates are then determined for predicting motion within the video block based on the motion trajectory. A motion vector is selected from a motion vector candidate list including the one or more temporal motion vector candidates and used to generate a prediction block. The prediction block is then used to encode or decode the video block. The motion trajectory is based on an order of video frames indicated by frame offset values encoded to a bitstream. The motion vector candidate list may include one or more spatial motion vector candidates.Type: GrantFiled: August 15, 2017Date of Patent: December 29, 2020Assignee: GOOGLE LLCInventors: Jingning Han, James Bankoski, Yaowu Xu
-
Patent number: 10869060Abstract: A memory of an apparatus includes instructions executable by a processor to select a backward scan order for traversing, along scan-order anti-diagonal lines, a block of values related to the transform coefficients; select a template for entropy-coding the values, the template indicating, for a to-be-coded value, scan positions of already coded values, the scan positions arranged in at least two template anti-diagonal lines; maintain, for coding first values, where the first values are arranged along a current scan-order anti-diagonal line, a first line buffer and a second line buffer; for a current value of the current scan-order anti-diagonal line, determine a context using the first line buffer and the second line buffer, and entropy encode, in a compressed bitstream, the current value using the context; and replace one of the second line buffer or the first line buffer with the current scan-order anti-diagonal line.Type: GrantFiled: November 25, 2019Date of Patent: December 15, 2020Assignee: GOOGLE LLCInventors: Jingning Han, James Zern, Linfeng Zhang, Ching-Han Chiang, Yaowu Xu
-
Publication number: 20200382776Abstract: Techniques for encoding and decoding image data are described. An image is reconstructed and deblocked. A respective deblocking filter is identified for different color planes of the image. The deblocking filters may include those having different lengths for a luma plane as compared to one or more chroma planes of the image. One or more of the color planes, such as the luma plane, may have different filters for filtering reconstructed pixels vertically as compared to filtering the reconstructed pixels horizontally.Type: ApplicationFiled: August 17, 2020Publication date: December 3, 2020Inventors: Yaowu Xu, Jingning Han, Cheng Chen
-
Publication number: 20200366921Abstract: Decoding a current frame includes identifying a first reference frame and a second reference frame for decoding the current frame; storing reference motion vectors of reference blocks of the first reference frame, where other reference frames are used to decode the first reference frame; identifying motion trajectories that pass through the current frame by projecting the reference motion vectors of the reference blocks of the first reference frame onto the current frame using at least a third reference frame of the other reference frames, where the projecting identifies, for a first current block of the current frame a corresponding first reference block in the first reference frame, and a corresponding reference motion vector of the reference motion vectors is associated with the corresponding first reference block; and projecting the corresponding reference motion vector onto the second reference frame to obtain a second reference block in the second reference frame.Type: ApplicationFiled: August 3, 2020Publication date: November 19, 2020Inventors: Jingning Han, Yaowu Xu, James Bankoski, Jia Feng
-
Patent number: 10820014Abstract: A prediction scheme is selected for encoding or decoding a video block. A first compound motion block can be determined by weighting distances from a first reference frame to the video frame and from a second reference frame to the video frame using one or more quantized weighting coefficients. A second compound motion block can be determined based on an average of pixel values a video block of the first reference frame and pixel values from a video block of the second reference frame. One of the first compound motion block or the second compound motion block is selected and used to generate a prediction block. Alternatively, data encoded to a bitstream including the video frame can be used to determine which compound motion block to use to generate the prediction block. The current block of the video frame is then encoded or decoded using the prediction block.Type: GrantFiled: September 5, 2019Date of Patent: October 27, 2020Assignee: GOOGLE LLCInventors: Jingning Han, Yaowu Xu
-
Patent number: 10771803Abstract: A method for decoding an encoded block of an encoded frame includes selecting motion vectors corresponding to blocks of an extended collocated processing unit in a first reference frame; identifying a block of the blocks, such that the encoded block is a projection, using a motion vector of the block that refers to a third reference frame, onto the encoded frame; on condition that a block is identified, determining a temporal motion vector candidate for the encoded block in a second reference frame using a motion vector of the identified block and respective display orders of the encoded frame, the first reference frame, the second reference frame, and the third reference frame, and adding the temporal motion vector candidate to a motion vector candidate list; selecting a motion vector from the motion vector candidate list; generating a prediction block using the selected motion vector for decoding the encoded block.Type: GrantFiled: April 9, 2019Date of Patent: September 8, 2020Assignee: GOOGLE LLCInventors: Jingning Han, Yaowu Xu, James Bankoski, Jia Feng
-
Publication number: 20200275098Abstract: A memory-efficient filtering approach is used to code images and video. A buffer having a fixed size based on a size of processing units to use for filtering a video frame is allocated. For each of the processing units, pre-filtered pixel values are copied from a respective region of the video frame to the buffer based on a writing point for the video frame and an offset applied to the writing point, filtering is performed against the pre-filtered pixel values from the buffer to produce filtered pixel values, and the filtered pixel values are written to the video frame based on the writing point and the offset. The filtering may be performed using a loop restoration tool, such as where the pre-filtered pixel values are output from a constrained directional enhancement filter (CDEF) tool. Alternatively, the filtering may be performed using the CDEF tool or another coding tool.Type: ApplicationFiled: April 27, 2020Publication date: August 27, 2020Inventors: Cheng Chen, Jingning Han
-
Publication number: 20200267391Abstract: Decoding a current block of a current frame includes selecting a first reference frame for forward inter prediction of the current frame; selecting a second reference frame for backward inter prediction of the current frame; generating an optical flow reference frame portion for inter prediction of the current block by performing an optical flow estimation using the first reference frame and the second reference frame, where the optical flow estimation produces a respective motion field for pixels of the current block; and performing a prediction process for the current block using the optical flow reference frame portion by: using a motion vector used to encode the current block to identify a reference block; adjusting boundaries of the reference block using a subpixel interpolation filter length; and identifying blocks encompassing pixels within the adjusted boundaries of the reference block.Type: ApplicationFiled: May 5, 2020Publication date: August 20, 2020Inventors: Yaowu Xu, Bohan Li, Jingning Han
-
Patent number: 10750171Abstract: Systems and methods are disclosed for encoding and decoding video. For example, methods may include: accessing an encoded bitstream; reconstructing an image including multiple color planes based on data from the encoded bitstream; decoding a first filter level from the encoded bitstream, wherein the first filter level specifies one or more thresholds that are used to select a length for a deblocking filter; decoding a second filter level from the encoded bitstream, wherein the second filter level specifies one or more thresholds that are used to select a length for a deblocking filter; after reconstruction of the image, applying a deblocking filter to a first color plane of the image using the first filter level; and, after reconstruction of the image, applying a deblocking filter to a second color plane of the image using the second filter level.Type: GrantFiled: June 25, 2018Date of Patent: August 18, 2020Assignee: GOOGLE LLCInventors: Cheng Chen, Jingning Han, Yaowu Xu
-
Patent number: 10735767Abstract: Encoding a transform block includes de-composing transform coefficients of the transform block into binary level maps arranged in a tier and a residual transform map, the binary level maps formed by breaking down a value of a respective transform coefficient into a series of binary decisions; and encoding, using a context model, a to-be-encoded binary decision that is at a scan location in a scan order, the to-be-encoded binary decision being a value of a binary level map at a level k. The context model is selected using first neighboring binary decisions of the binary level map at a level k that precede the to-be-encoded binary decision; and second neighboring binary decisions of a binary level map at a level (k?1), the second neighboring binary decisions including values that precede and values that follow, in the scan order, a co-located binary decision of the to-be-encoded binary decision.Type: GrantFiled: March 12, 2019Date of Patent: August 4, 2020Assignee: GOOGLE LLCInventors: Yaowu Xu, Jingning Han, Ching-Han Chiang
-
Publication number: 20200236375Abstract: An apparatus for video coding using dual deblocking filter thresholds may include a processor generating a reconstructed frame by decoding an encoded bitstream and outputting the reconstructed frame. Decoding may include generating a decoded block by decoding a portion of the encoded bitstream, identifying a first deblocking threshold index from the encoded bitstream, identifying a second deblocking threshold index from the encoded bitstream, generating a reconstructed block based on the decoded block, and including the reconstructed block in the reconstructed frame. Generating the reconstructed block may include deblocking based on the first deblocking threshold index and the second deblocking threshold index.Type: ApplicationFiled: April 2, 2020Publication date: July 23, 2020Inventors: Jingning Han, Yaowu Xu
-
Publication number: 20200236350Abstract: A sign value of a DC coefficient of a current block is determined using neighbor blocks of the current block. First and second sign values are identified as respectively corresponding to an above neighbor block of the current block and a left neighbor block of the current block. A context value is calculated by applying a first weighting value to the first sign value and a second weighting value to the second sign value. The first weighting value is based on a boundary between the above neighbor block and the current block, and the second weighting value is based on a boundary between the left neighbor block and the current block. A probability value is selected based on the context value. The sign value of the DC coefficient of the current block is then determined using the probability model.Type: ApplicationFiled: April 2, 2020Publication date: July 23, 2020Inventors: Yaowu Xu, Jingning Han, Ching-Han Chiang