Patents by Inventor Gary J. Sullivan
Gary J. Sullivan has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11770554Abstract: Innovations in signaling of reference picture list (“RPL”) modification information. For example, a video encoder evaluates a condition that depends at least in part on a variable indicating a number of total reference pictures. Depending on the results of the evaluation, the encoder signals in a bitstream a flag that indicates whether an RPL is modified according to syntax elements explicitly signaled in the bitstream. A video decoder evaluates the condition and, depending on results of the evaluation, parses from a bitstream a flag that indicates whether an RPL is modified according to syntax elements explicitly signaled in the bitstream. The condition can be evaluated as part of processing for an RPL modification structure that includes the flag, or as part of processing for a slice header. The encoder and decoder can also evaluate other conditions that affect syntax elements for list entries of the RPL modification information.Type: GrantFiled: June 21, 2022Date of Patent: September 26, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Gary J. Sullivan, Sandeep Kanumuri
-
Publication number: 20230269382Abstract: Innovations in encoding or decoding when switching color spaces are presented. For example, some of the innovations relate to signaling of control information for adaptive color space transformation (“ACT”). Other innovations relate to ACT operations. These innovations can improve coding efficiency when switching between color spaces during encoding and decoding.Type: ApplicationFiled: May 1, 2023Publication date: August 24, 2023Applicant: Microsoft Technology Licensing, LLCInventors: Bin Li, Jizheng Xu, Gary J. Sullivan
-
Publication number: 20230262249Abstract: Innovations in syntax and semantics of coded picture buffer removal delay (“CPBRD”) values potentially simplify splicing operations. For example, a video encoder sets a CPBRD value for a current picture that indicates an increment value relative to a nominal coded picture buffer removal time of a preceding picture in decoding order, regardless of whether the preceding picture has a buffering period SEI message. The encoder can signal the CPBRD value according to a single-value approach in which a flag indicates how to interpret the CPBRD value, according to a two-value approach in which another CPBRD value (having a different interpretation) is also signaled, or according to a two-value approach that uses a flag and a delta value. A corresponding video decoder receives and parses the CPBRD value for the current picture. A splicing tool can perform simple concatenation operations to splice bitstreams using the CPBRD value for the current picture.Type: ApplicationFiled: April 18, 2023Publication date: August 17, 2023Applicant: Microsoft Technology Licensing, LLCInventors: Gary J. Sullivan, Lihua Zhu
-
Publication number: 20230254499Abstract: Innovations in syntax and semantics of coded picture buffer removal delay (“CPBRD”) values potentially simplify splicing operations. For example, a video encoder sets a CPBRD value for a current picture that indicates an increment value relative to a nominal coded picture buffer removal time of a preceding picture in decoding order, regardless of whether the preceding picture has a buffering period SEI message. The encoder can signal the CPBRD value according to a single-value approach in which a flag indicates how to interpret the CPBRD value, according to a two-value approach in which another CPBRD value (having a different interpretation) is also signaled, or according to a two-value approach that uses a flag and a delta value. A corresponding video decoder receives and parses the CPBRD value for the current picture. A splicing tool can perform simple concatenation operations to splice bitstreams using the CPBRD value for the current picture.Type: ApplicationFiled: April 18, 2023Publication date: August 10, 2023Applicant: Microsoft Technology Licensing, LLCInventors: Gary J. Sullivan, Lihua Zhu
-
Patent number: 11716478Abstract: Innovations in encoding or decoding when switching color spaces are presented. For example, some of the innovations relate to signaling of control information for adaptive color space transformation (“ACT”). Other innovations relate to ACT operations. These innovations can improve coding efficiency when switching between color spaces during encoding and decoding.Type: GrantFiled: July 20, 2021Date of Patent: August 1, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Bin Li, Jizheng Xu, Gary J. Sullivan
-
Publication number: 20230239509Abstract: Techniques and tools for reducing latency in video encoding and decoding by constraining latency due to reordering of video frames, and by indicating the constraint on frame reordering latency with one or more syntax elements that accompany encoded data for the video frames. For example, a real-time communication tool with a video encoder sets a syntax element that indicates a constraint on frame reordering latency, which is consistent with inter-frame dependencies between multiple frames of a video sequence, then outputs the syntax element. A corresponding real-time communication tool with a video decoder receives the syntax element that indicates the constraint on frame reordering latency, determines the constraint on frame reordering latency based on the syntax element, and uses the constraint on frame reordering latency to determine when a reconstructed frame is ready for output (in terms of output order).Type: ApplicationFiled: March 21, 2023Publication date: July 27, 2023Applicant: Microsoft Technology Licensing, LLCInventor: Gary J. Sullivan
-
Publication number: 20230239495Abstract: Innovations in encoder-side search ranges having horizontal bias or vertical bias are described herein. For example, a video encoder determines a block vector (“BV”) for a current block of a picture, performs intra prediction for the current block using the BV, and encodes the BV. The BV indicates a displacement to a region within the picture. When determining the BV, the encoder checks a constraint that the region is within a BV search range having a horizontal bias or vertical bias. The encoder can select the BV search range from among multiple available BV search ranges, e.g., depending at least in part on BV values of one or more previous blocks, which can be tracked in a histogram data structure.Type: ApplicationFiled: January 23, 2023Publication date: July 27, 2023Applicant: Microsoft Technology Licensing, LLCInventors: Lihua Zhu, Yongjun Wu, Gary J. Sullivan
-
Publication number: 20230209070Abstract: Approaches to selection of motion vector (“MV”) precision during video encoding are presented. These approaches can facilitate compression that is effective in terms of rate-distortion performance and/or computational efficiency. For example, a video encoder determines an MV precision for a unit of video from among multiple MV precisions, which include one or more fractional-sample MV precisions and integer-sample MV precision. The video encoder can identify a set of MV values having a fractional-sample MV precision, then select the MV precision for the unit based at least in part on prevalence of MV values (within the set) having a fractional part of zero. Or, the video encoder can perform rate-distortion analysis, where the rate-distortion analysis is biased towards the integer-sample MV precision. Or, the video encoder can collect information about the video and select the MV precision for the unit based at least in part on the collected information.Type: ApplicationFiled: March 6, 2023Publication date: June 29, 2023Applicant: Microsoft Technology Licensing, LLCInventors: Gary J. Sullivan, You Zhou, Chih-Lung Lin
-
Patent number: 11665362Abstract: Innovations in syntax and semantics of coded picture buffer removal delay (“CPBRD”) values potentially simplify splicing operations. For example, a video encoder sets a CPBRD value for a current picture that indicates an increment value relative to a nominal coded picture buffer removal time of a preceding picture in decoding order, regardless of whether the preceding picture has a buffering period SEI message. The encoder can signal the CPBRD value according to a single-value approach in which a flag indicates how to interpret the CPBRD value, according to a two-value approach in which another CPBRD value (having a different interpretation) is also signaled, or according to a two-value approach that uses a flag and a delta value. A corresponding video decoder receives and parses the CPBRD value for the current picture. A splicing tool can perform simple concatenation operations to splice bitstreams using the CPBRD value for the current picture.Type: GrantFiled: September 16, 2022Date of Patent: May 30, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Gary J. Sullivan, Lihua Zhu
-
Patent number: 11665361Abstract: Innovations in syntax and semantics of coded picture buffer removal delay (“CPBRD”) values potentially simplify splicing operations. For example, a video encoder sets a CPBRD value for a current picture that indicates an increment value relative to a nominal coded picture buffer removal time of a preceding picture in decoding order, regardless of whether the preceding picture has a buffering period SEI message. The encoder can signal the CPBRD value according to a single-value approach in which a flag indicates how to interpret the CPBRD value, according to a two-value approach in which another CPBRD value (having a different interpretation) is also signaled, or according to a two-value approach that uses a flag and a delta value. A corresponding video decoder receives and parses the CPBRD value for the current picture. A splicing tool can perform simple concatenation operations to splice bitstreams using the CPBRD value for the current picture.Type: GrantFiled: August 18, 2022Date of Patent: May 30, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Gary J. Sullivan, Lihua Zhu
-
Publication number: 20230156194Abstract: Innovations in use of chroma quantization parameter (“QP”) offsets when determining a control parameter for deblock filtering. For example, as part of encoding, an encoder sets a picture-level chroma QP offset and slice-level chroma QP offset for encoding of a slice of a picture. The encoder also performs deblock filtering of at least part of the slice, where derivation of a control parameter considers only the picture-level chroma QP offset. The encoder outputs at least part of a bitstream including the encoded content. As part of decoding, a corresponding decoder sets a picture-level chroma QP offset and a slice-level chroma QP offset for decoding of a slice of a picture, but derivation of a control parameter for deblock filtering considers only the picture-level chroma QP offset.Type: ApplicationFiled: January 20, 2023Publication date: May 18, 2023Applicant: Microsoft Technology Licensing, LLCInventor: Gary J. Sullivan
-
Patent number: 11641486Abstract: Techniques and tools for reducing latency in video encoding and decoding by constraining latency due to reordering of video frames, and by indicating the constraint on frame reordering latency with one or more syntax elements that accompany encoded data for the video frames. For example, a real-time communication tool with a video encoder sets a syntax element that indicates a constraint on frame reordering latency, which is consistent with inter-frame dependencies between multiple frames of a video sequence, then outputs the syntax element. A corresponding real-time communication tool with a video decoder receives the syntax element that indicates the constraint on frame reordering latency, determines the constraint on frame reordering latency based on the syntax element, and uses the constraint on frame reordering latency to determine when a reconstructed frame is ready for output (in terms of output order).Type: GrantFiled: September 20, 2022Date of Patent: May 2, 2023Assignee: Microsoft Technology Licensing, LLCInventor: Gary J. Sullivan
-
Patent number: 11641487Abstract: Techniques and tools for reducing latency in video encoding and decoding by constraining latency due to reordering of video frames, and by indicating the constraint on frame reordering latency with one or more syntax elements that accompany encoded data for the video frames. For example, a real-time communication tool with a video encoder sets a syntax element that indicates a constraint on frame reordering latency, which is consistent with inter-frame dependencies between multiple frames of a video sequence, then outputs the syntax element. A corresponding real-time communication tool with a video decoder receives the syntax element that indicates the constraint on frame reordering latency, determines the constraint on frame reordering latency based on the syntax element, and uses the constraint on frame reordering latency to determine when a reconstructed frame is ready for output (in terms of output order).Type: GrantFiled: September 20, 2022Date of Patent: May 2, 2023Assignee: Microsoft Technology Licensing, LLCInventor: Gary J. Sullivan
-
Patent number: 11641485Abstract: Techniques and tools for reducing latency in video encoding and decoding by constraining latency due to reordering of video frames, and by indicating the constraint on frame reordering latency with one or more syntax elements that accompany encoded data for the video frames. For example, a real-time communication tool with a video encoder sets a syntax element that indicates a constraint on frame reordering latency, which is consistent with inter-frame dependencies between multiple frames of a video sequence, then outputs the syntax element. A corresponding real-time communication tool with a video decoder receives the syntax element that indicates the constraint on frame reordering latency, determines the constraint on frame reordering latency based on the syntax element, and uses the constraint on frame reordering latency to determine when a reconstructed frame is ready for output (in terms of output order).Type: GrantFiled: September 20, 2022Date of Patent: May 2, 2023Assignee: Microsoft Technology Licensing, LLCInventor: Gary J. Sullivan
-
Patent number: 11638016Abstract: Approaches to selection of motion vector (“MV”) precision during video encoding are presented. These approaches can facilitate compression that is effective in terms of rate-distortion performance and/or computational efficiency. For example, a video encoder determines an MV precision for a unit of video from among multiple MV precisions, which include one or more fractional-sample MV precisions and integer-sample MV precision. The video encoder can identify a set of MV values having a fractional-sample MV precision, then select the MV precision for the unit based at least in part on prevalence of MV values (within the set) having a fractional part of zero. Or, the video encoder can perform rate-distortion analysis, where the rate-distortion analysis is biased towards the integer-sample MV precision. Or, the video encoder can collect information about the video and select the MV precision for the unit based at least in part on the collected information.Type: GrantFiled: July 2, 2021Date of Patent: April 25, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Gary J. Sullivan, You Zhou, Chih-Lung Lin
-
Publication number: 20230086944Abstract: A format for use in encoding moving image data, comprising: a sequence of frames including plurality of the frames in which at least a region is encoded using motion estimation; a respective set of motion vector values representing motion vectors of the motion estimation for each respective one of these frames or each respective one of one or more regions within each of such frames; and at least one respective indicator associated with each of the respective frames or regions, indicating whether the respective motion vector values of the respective frame or region are encoded at a first resolution or a second resolution.Type: ApplicationFiled: November 28, 2022Publication date: March 23, 2023Applicant: Microsoft Technology Licensing, LLCInventors: You Zhou, Sergey Silkin, Sergey Sablin, Chih-Lung Lin, Ming-Chieh Lee, Gary J. Sullivan
-
Patent number: 11601684Abstract: Techniques and tools for reducing latency in video encoding and decoding by constraining latency due to reordering of video frames, and by indicating the constraint on frame reordering latency with one or more syntax elements that accompany encoded data for the video frames. For example, a real-time communication tool with a video encoder sets a syntax element that indicates a constraint on frame reordering latency, which is consistent with inter-frame dependencies between multiple frames of a video sequence, then outputs the syntax element. A corresponding real-time communication tool with a video decoder receives the syntax element that indicates the constraint on frame reordering latency, determines the constraint on frame reordering latency based on the syntax element, and uses the constraint on frame reordering latency to determine when a reconstructed frame is ready for output (in terms of output order).Type: GrantFiled: September 20, 2022Date of Patent: March 7, 2023Assignee: Microsoft Technology Licensing, LLCInventor: Gary J. Sullivan
-
Patent number: 11601683Abstract: Techniques and tools for reducing latency in video encoding and decoding by constraining latency due to reordering of video frames, and by indicating the constraint on frame reordering latency with one or more syntax elements that accompany encoded data for the video frames. For example, a real-time communication tool with a video encoder sets a syntax element that indicates a constraint on frame reordering latency, which is consistent with inter-frame dependencies between multiple frames of a video sequence, then outputs the syntax element. A corresponding real-time communication tool with a video decoder receives the syntax element that indicates the constraint on frame reordering latency, determines the constraint on frame reordering latency based on the syntax element, and uses the constraint on frame reordering latency to determine when a reconstructed frame is ready for output (in terms of output order).Type: GrantFiled: September 20, 2022Date of Patent: March 7, 2023Assignee: Microsoft Technology Licensing, LLCInventor: Gary J. Sullivan
-
Patent number: 11601682Abstract: Techniques and tools for reducing latency in video encoding and decoding by constraining latency due to reordering of video frames, and by indicating the constraint on frame reordering latency with one or more syntax elements that accompany encoded data for the video frames. For example, a real-time communication tool with a video encoder sets a syntax element that indicates a constraint on frame reordering latency, which is consistent with inter-frame dependencies between multiple frames of a video sequence, then outputs the syntax element. A corresponding real-time communication tool with a video decoder receives the syntax element that indicates the constraint on frame reordering latency, determines the constraint on frame reordering latency based on the syntax element, and uses the constraint on frame reordering latency to determine when a reconstructed frame is ready for output (in terms of output order).Type: GrantFiled: September 20, 2022Date of Patent: March 7, 2023Assignee: Microsoft Technology Licensing, LLCInventor: Gary J. Sullivan
-
Patent number: 11601681Abstract: Techniques and tools for reducing latency in video encoding and decoding by constraining latency due to reordering of video frames, and by indicating the constraint on frame reordering latency with one or more syntax elements that accompany encoded data for the video frames. For example, a real-time communication tool with a video encoder sets a syntax element that indicates a constraint on frame reordering latency, which is consistent with inter-frame dependencies between multiple frames of a video sequence, then outputs the syntax element. A corresponding real-time communication tool with a video decoder receives the syntax element that indicates the constraint on frame reordering latency, determines the constraint on frame reordering latency based on the syntax element, and uses the constraint on frame reordering latency to determine when a reconstructed frame is ready for output (in terms of output order).Type: GrantFiled: August 18, 2022Date of Patent: March 7, 2023Assignee: Microsoft Technology Licensing, LLCInventor: Gary J. Sullivan