Patents by Inventor James Bankoski

James Bankoski has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20190124363
    Abstract: Video coding using a constructed reference frame may include include generating, by a processor in response to instructions stored on a non-transitory computer readable medium, an encoded video and outputting an output bitstream. Generating the encoded video includes receiving an input video stream, generating a constructed reference frame, generating an encoded constructed reference frame by encoding the constructed reference frame, including the encoded constructed reference frame in an output bitstream such that the constructed reference fame is a non-showable frame, generating an encoded frame by encoding a current frame from the input video stream using the constructed reference frame as a reference frame, and including the encoded frame in the output bitstream.
    Type: Application
    Filed: December 17, 2018
    Publication date: April 25, 2019
    Inventors: James Bankoski, Paul Wilkins, Yaowu Xu
  • Publication number: 20190098329
    Abstract: Motion field estimation is used to predict motion within video blocks. A current block of a current frame is identified as a projection of a first reference block of a first reference frame onto the current frame. The projection uses a first motion vector of the reference block with respect to a second reference frame. A temporal motion vector candidate is determined for the current block by projecting the current block onto a second reference frame. The temporal motion vector candidate identifies a second reference block. A motion vector selected from a motion vector candidate list is used to generate a prediction block using the selected motion vector. The current block is coded using the prediction block.
    Type: Application
    Filed: September 28, 2017
    Publication date: March 28, 2019
    Inventors: Jingning Han, Yaowu Xu, James Bankoski, Jia Feng
  • Patent number: 10230986
    Abstract: An apparatus for decoding frames of a compressed video data stream having at least one frame divided into partitions, includes a memory and a processor configured to execute instructions stored in the memory to read partition data information indicative of a partition location for at least one of the partitions, decode a first partition of the partitions that includes a first sequence of blocks, decode a second partition of the partitions that includes a second sequence of blocks identified from the partition data information using decoded information of the first partition.
    Type: Grant
    Filed: May 26, 2016
    Date of Patent: March 12, 2019
    Assignee: GOOGLE LLC
    Inventors: Yaowu Xu, Paul Wilkins, James Bankoski
  • Patent number: 10225549
    Abstract: An apparatus for decoding compressed video information may include a memory and a processor configured to execute instructions stored in the memory to receive a compressed video stream including compressed video information, read, from the compressed video information, a segment map associating at least some blocks in a first frame with at least one segment identifier, read, from the compressed video information, a current encoded block, read, from the segment map, a segment identifier that is associated with the current encoded block, read, from the compressed video information, at least one segment parameter associated with the segment identifier that is associated with the current encoded block, generate a current decoded block by decoding the current encoded block using the at least one segment parameter associated with the segment identifier that is associated with the current encoded block, and output the current decoded block for presentation or storage.
    Type: Grant
    Filed: February 13, 2018
    Date of Patent: March 5, 2019
    Assignee: GOOGLE LLC
    Inventors: Paul Wilkins, James Bankoski, Yaowu Xu
  • Patent number: 10225573
    Abstract: A current block of a video frame can be encoded or decoded using parameterized motion models. First and second parameterized motion models are identified. The first parameterized motion model corresponds to a first motion model type, and the second parameterized motion model corresponds to a second motion model type. The first and second parameterized motion models are associated with one or more reference frames. One of the first or second parameterized motion models is selected along with an associated reference frame, such as based on a lowest prediction error. A motion vector is generated between the current block and the selected reference frame by warping pixels of the current block to a warped patch of the selected reference frame according to the selected parameterized motion model. A prediction block is generated using the motion vector, and the current block is encoded or decoded using the prediction block.
    Type: Grant
    Filed: January 31, 2017
    Date of Patent: March 5, 2019
    Assignee: GOOGLE LLC
    Inventors: Debargha Mukherjee, James Bankoski, Yue Chen, Sarah Parker
  • Publication number: 20190058879
    Abstract: An apparatus for decoding compressed video information may include a memory and a processor configured to execute instructions stored in the memory to receive a compressed video stream including compressed video information, read, from the compressed video information, a segment map associating at least some blocks in a first frame with at least one segment identifier, read, from the compressed video information, a current encoded block, read, from the segment map, a segment identifier that is associated with the current encoded block, read, from the compressed video information, at least one segment parameter associated with the segment identifier that is associated with the current encoded block, generate a current decoded block by decoding the current encoded block using the at least one segment parameter associated with the segment identifier that is associated with the current encoded block, and output the current decoded block for presentation or storage.
    Type: Application
    Filed: February 13, 2018
    Publication date: February 21, 2019
    Inventors: Paul Wilkins, James Bankoski, Yaowu Xu
  • Publication number: 20190058897
    Abstract: Dynamic motion vector referencing is used to predict motion within video blocks. A motion trajectory is determined for a current frame including a video block to encode or decode based on a reference motion vector used for encoding or decoding one or more reference frames of the current frame. One or more temporal motion vector candidates are then determined for predicting motion within the video block based on the motion trajectory. A motion vector is selected from a motion vector candidate list including the one or more temporal motion vector candidates and used to generate a prediction block. The prediction block is then used to encode or decode the video block. The motion trajectory is based on an order of video frames indicated by frame offset values encoded to a bitstream. The motion vector candidate list may include one or more spatial motion vector candidates.
    Type: Application
    Filed: August 15, 2017
    Publication date: February 21, 2019
    Inventors: Jingning Han, James Bankoski, Yaowu Xu
  • Patent number: 10165306
    Abstract: Decoding an encoded video stream may include generating, by a processor in response to instructions stored on a non-transitory computer readable medium, a decoded video for presentation to a user, and outputting the decoded video. Generating the decoded video may include receiving an encoded video stream, generating a decoded constructed reference frame by decoding an encoded constructed reference frame from the encoded video stream, generating a decoded current frame by decoding an encoded current frame from the encoded video stream using the decoded constructed reference frame as a reference frame, and including the decoded current frame in the decoded video such that the decoded constructed reference frame is omitted from the decoded video.
    Type: Grant
    Filed: June 20, 2016
    Date of Patent: December 25, 2018
    Assignee: GOOGLE LLC
    Inventors: James Bankoski, Yaowu Xu, Paul Wilkins
  • Publication number: 20180270502
    Abstract: Coding a current block using multi-level compound predictor is disclosed. A method includes generating a first compound predictor by combining at least two predictor blocks for the current block, generating a second predictor for the current block, and combining the first compound predictor and the second predictor to obtain a prediction block for encoding or decoding the current block. An indicator may be sent from the encoder to the decoder to indicate when multi-level compound prediction is used to encode a block.
    Type: Application
    Filed: March 14, 2017
    Publication date: September 20, 2018
    Inventors: Debargha Mukherjee, Yaowu Xu, James Bankoski, Yuxin Liu
  • Publication number: 20180220152
    Abstract: Multi-reference compound prediction using masking for encoding and decoding blocks of video frames is described. A mask for segmenting a current block of a current frame of the video sequence into at least a first segment and a second segment is determined using a first set of reference frames. A first predictor using the first set of reference frames is generated for the first segment of the current block. A second predictor is generated for the second segment of the current block using a second set of reference frames, wherein the first set of reference frames is a subset of the second set of reference frames.
    Type: Application
    Filed: January 31, 2017
    Publication date: August 2, 2018
    Inventors: Debargha Mukherjee, Yaowu Xu, James Bankoski, Yuxin Liu
  • Publication number: 20180205964
    Abstract: Generating a compound predictor block of a current block includes generating, for the current block, predictor blocks including a first predictor block formed of first predictor pixels. Using at least a subset of the first predictor pixels, a first modulation value for modulating a first weight to be applied to a first predictor pixel of the first predictor pixels is determined.
    Type: Application
    Filed: November 9, 2017
    Publication date: July 19, 2018
    Inventors: Debargha Mukherjee, James Bankoski, Yue Chen, Yuxin Liu, Sarah Parker
  • Publication number: 20180184086
    Abstract: Encoding or decoding blocks of video frames using multiple reference frames with adaptive temporal filtering can include generating one or more candidate reference frames by applying temporal filtering to one or more frames of a video sequence according to relationships between respective ones of the one or more frames and a current frame of the video sequence. A reference frame to use for predicting the current frame can be selected from the one or more candidate reference frames, and a prediction block can be generated using the selected reference frame. During an encoding operation, the prediction block can be used to encode a block of a current frame of the video sequence. During a decoding operation, the prediction block can be used to decode a block of a current frame of the video sequence.
    Type: Application
    Filed: December 22, 2016
    Publication date: June 28, 2018
    Inventors: Debargha Mukherjee, Yaowu Xu, James Bankoski, Paul Wilkins, Jingning Han, Yuxin Liu, Wei-Ting Lin
  • Patent number: 9924161
    Abstract: A method for decoding compressed video information is disclosed. The video information can include a plurality of frames each having a plurality of blocks. The method reading, from the video information, a segment map associating at least some of the plurality of blocks in a first frame of the plurality of frames with at least one segment identifier, reading, from the video information, a current block to be decoded by a decoder, reading, from the segment map, a segment identifier that is associated with the current block, reading, from the video information, at least one segment parameter associated with the segment identifier that is associated with the current block and decoding the current block using the at least one segment parameter associated with the segment identifier that is associated with the current block.
    Type: Grant
    Filed: November 19, 2012
    Date of Patent: March 20, 2018
    Assignee: GOOGLE LLC
    Inventors: Paul Wilkins, James Bankoski, Yaowu Xu
  • Patent number: 9838690
    Abstract: Disclosed herein are methods and apparatuses for selective prediction signal filtering. One aspect of the disclosed implementations is a method for encoding a frame of a video stream including determining a first performance measurement for a first set of prediction samples identified for a group of pixels using a first prediction mode, generating a filtered set of prediction samples by applying a filter to a second set of prediction samples, wherein at least one of the filtered set of prediction samples or the second set of prediction samples are identified using a second prediction mode, determining a second performance measurement for the filtered set of prediction samples, generating, using a processor, a residual based on the filtered set of prediction samples and the group of pixels if the second performance measurement exceeds the first performance measurement, and encoding the frame using the residual.
    Type: Grant
    Filed: May 9, 2016
    Date of Patent: December 5, 2017
    Assignee: GOOGLE INC.
    Inventors: Adrian Grange, James Bankoski, Paul G. Wilkins, Yaowu Xu
  • Publication number: 20170302965
    Abstract: Adaptive directional loop filtering can reduce the number of blocking artifacts produced by coding a non-perpendicular picture edge in a frame of a video sequence. A directional filter is selected from a set of directional filters based on one of an orientation of the non-perpendicular picture edge or filter data included as part of an encoded video sequence in association with the frame. The selection can include selecting a directional filter based on a directional intra prediction mode used for encoding the block, a filter angle most closely matching an angle explicitly signaled as part of the video sequence, the incremental reduction of the number of blocking artifacts, a threshold value for blocking artifacts, or a frequency of filter use. Each directional filter of the set of directional filters can have a filter angle between 0 and 180 degrees, exclusive.
    Type: Application
    Filed: April 15, 2016
    Publication date: October 19, 2017
    Applicant: GOOGLE INC.
    Inventors: Yaowu Xu, Paul Wilkins, James Bankoski
  • Publication number: 20170280143
    Abstract: Video data streams can be encoded and decoded using inter or intra prediction. The blocks of a frame can be processed based on depth, from the lowest level sub-blocks to the highest level large blocks, and divided into groups of blocks to be inter predicted, blocks having sub-blocks that are to be inter predicted and sub-blocks that are to be intra predicted, and blocks to be intra predicted, and the blocks to be inter predicted are encoded first, the blocks having sub-blocks to be inter predicted and intra predicted encoded second, and the blocks to be intra predicted encoded last. The availability of data from the inter predicted blocks can improve the performance of intra prediction over processing the blocks in the scan order since more pixel data is available for intra prediction of some blocks.
    Type: Application
    Filed: March 25, 2016
    Publication date: September 28, 2017
    Inventors: Yaowu Xu, James Bankoski, Paul Wilkins, Hui Su
  • Publication number: 20170223357
    Abstract: A mask is generated from a residual that is a difference between pixel values of at least two frames other than a current frame in a video sequence. The mask is then used to encode or decode a current block by inter-prediction. The mask forms an edge of an object within its bounds and can be used to combine different portions of a reference frame into a single block. This can increase coding efficiency.
    Type: Application
    Filed: January 29, 2016
    Publication date: August 3, 2017
    Inventors: James Bankoski, Debargha Mukherjee, Yaowu Xu
  • Publication number: 20170223377
    Abstract: Encoding or decoding a video signal includes determining a first partitioning of at least a portion of a frame before a current frame in a video sequence. The first partitioning is modified to a second partitioning using a motion vector that predicts at least one prediction sub-region of a first region from the first partitioning. The second partitioning is used to encode or decode a current region of the current frame. The current region may be collocated with the first region. The technique eliminates the need to re-partition an entire frame each time, and can be used with both rectangular-shaped block partitioning and partitioning using masks and other partitioning techniques.
    Type: Application
    Filed: January 29, 2016
    Publication date: August 3, 2017
    Inventors: James Bankoski, Debargha Mukherjee, Yaowu Xu
  • Patent number: 9648394
    Abstract: Techniques for delivering content, such as videos, over a network are described. A core server and an edge server are provided. The core server has local storage. The edge server has local storage. A set of video frames is partitioned into a first group and a second group. Video frames in the first group are encoded into a scalable encoded stream. The scalable encoded stream is sent to the local storage at the edge server. The second group of video frames is encoded into a set of supplemental encoded streams using the scalable encoded stream as a reference. The supplemental encoded streams are encoded such that the bit rate and/or resolution of any two supplemental encoded streams is different. The set of supplemental encoded streams is stored in the storage of the core server.
    Type: Grant
    Filed: March 21, 2014
    Date of Patent: May 9, 2017
    Assignee: Google Inc.
    Inventors: Yaowu Xu, James Bankoski, Paul G. Wilkins
  • Patent number: 9503746
    Abstract: A method for encoding a video stream having a plurality of blocks including a current block. The method includes identifying, from a plurality of previously coded blocks, a plurality of candidate motion vectors; identifying, for the current block, a set of reconstructed pixel values corresponding to a set of previously coded pixel values; determining, for at least one of the plurality of candidate motion vectors, an error value based on the set of reconstructed pixel values and a set of predicted values for the set of previously coded pixel values; and selecting, from the plurality of candidate motion vectors, a reference motion vector for the current block based on the error value.
    Type: Grant
    Filed: October 8, 2012
    Date of Patent: November 22, 2016
    Assignee: GOOGLE INC.
    Inventors: Yaowu Xu, Paul Gordon Wilkins, Adrian Grange, James Bankoski