Patents by Inventor Jonathan PFAFF

Jonathan PFAFF has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20250150593
    Abstract: A video decoder includes one or more processors that are configured to: determine that a picture is in a 4:4:4 color sampling format, determine, based at least in part on an index signaled in a data stream, a matrix-based intra prediction (MIP) mode, decode a luma block of the picture using the MIP mode, determine, for a chroma block of the picture, whether a coding tree of the chroma block is a single tree, select an intra prediction mode for decoding the chroma block, and decode the chroma block using the selected intra prediction mode. The selected intra prediction mode is the MIP mode in response to the coding tree of the chroma block being the single tree, and the selected intra prediction mode is a planar intra prediction mode in response to the coding tree of the chroma block not being the single tree.
    Type: Application
    Filed: January 9, 2025
    Publication date: May 8, 2025
    Inventors: Jonathan PFAFF, Tobias HINZ, Philipp HELLE, Philipp MERKLE, Björn STALLENBERGER, Michael SCHÄFER, Benjamin BROSS, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20250150609
    Abstract: Video decoders are described, which use block-based predictive decoding, transform-based residual decoding and a prediction loop, with an in-loop filter being connected in the prediction loop. The decoder performs a mode switching between different modes of the in-loop filter, which differ in computational complexity.
    Type: Application
    Filed: January 13, 2025
    Publication date: May 8, 2025
    Inventors: Wang-Q LIM, Jonathan PFAFF, Bjoern STALLENBERGER, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20250150595
    Abstract: A coding concept for encoding and decoding a picture is described, according to which a machine learning predictor is used to derive a set of features representing a motion estimation for the picture with respect to a previous picture. The set of features, as well as a residual picture derived using the motion estimation, are encoded into a data stream.
    Type: Application
    Filed: January 10, 2025
    Publication date: May 8, 2025
    Inventors: Michael SCHÄFER, Sophie PIENTKA, Jonathan PFAFF, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20250150589
    Abstract: An apparatus for decoding a data structure from a data stream is configured for sequentially reconstructing a set of features, wherein the apparatus is configured for reconstructing a feature of the set of features by deriving a quantization index from the data stream using arithmetic decoding; mapping the quantization index to a reconstruction value in dependence on a quantization state, and assigning the reconstruction value to the feature; updating the quantization state in dependence on the quantization index and the quantization state; and deriving the data structure based on the set of features. The apparatus is configured for deriving probabilities for the arithmetic decoding of the quantization index in dependence on the quantization state.
    Type: Application
    Filed: January 13, 2025
    Publication date: May 8, 2025
    Applicant: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Karsten SÜHRING, Michael SCHÄFER, Jonathan PFAFF, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Patent number: 12244864
    Abstract: A method for decoding a predetermined block of a picture using intra-prediction by reading, for each of predetermined intra-prediction blocks, from a data stream, a mode index identifying a matrix-based intra-prediction mode is disclosed. Samples of each predetermined intra-prediction block are predicted by computing a matrix-vector product between an input vector derived from reference samples and a prediction matrix associated with the identified matrix-based intra-prediction mode (k)—and associating components of an output vector obtained by the matrix-vector product onto sample positions of the predetermined block. Further predetermined intra-predicted blocks of the picture are predicted to obtain a prediction signal. A transformation flag is decoded from the data stream using context adaptive binary arithmetic coding, and a prediction residual is decoded and re-transformed using a reverse transformation based on the transformation flag to obtain a prediction residual signal.
    Type: Grant
    Filed: May 12, 2024
    Date of Patent: March 4, 2025
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Jonathan Pfaff, Björn Stallenberger, Michael Schafer, Philipp Merkle, Tobias Hinz, Philipp Helle, Heiko Schwarz, Detlev Marpe, Thomas Wiegand, Benjamin Bross, Martin Winken, Mischa Siekmann
  • Publication number: 20250071292
    Abstract: A video encoder for encoding a video into a data stream using motion compensated prediction for inter predicted blocks, comprising a hypothesis number control, configured to control a number of prediction hypotheses of the inter predicted blocks within a predetermined portion of the video to meet a predetermined criterion.
    Type: Application
    Filed: November 8, 2024
    Publication date: February 27, 2025
    Inventors: Martin WINKEN, Christian BARTNIK, Philipp HELLE, Mischa SIEKMANN, Jonathan PFAFF, Detlev MARPE, Heiko SCHWARZ, Thomas WIEGAND
  • Publication number: 20250071291
    Abstract: A video encoder for encoding a video into a data stream using motion compensated prediction for inter predicted blocks, comprising a hypothesis number control, configured to control a number of prediction hypotheses of the inter predicted blocks within a predetermined portion of the video to meet a predetermined criterion.
    Type: Application
    Filed: November 8, 2024
    Publication date: February 27, 2025
    Inventors: Martin WINKEN, Christian BARTNIK, Philipp HELLE, Mischa SIEKMANN, Jonathan PFAFF, Detlev MARPE, Heiko SCHWARZ, Thomas WIEGAND
  • Publication number: 20250056040
    Abstract: An apparatus for block-wise decoding a picture from a data stream and/or encoding a picture into a data stream, the apparatus supporting at least one intra-prediction mode according to which the intra-prediction signal for a block of a predetermined size of the picture is determined by applying a first template of samples which neighbours the current block onto a neural network. The apparatus may be configured, for a current block differing from the predetermined size, to: resample a second template of samples neighboring the current block, so as to conform with the first template so as to obtain a resampled template; apply the resampled template of samples onto the neural network so as to obtain a preliminary intra-prediction signal; and resample the preliminary intra-prediction signal so as to conform with the current block so as to obtain the intra-prediction signal for the current block.
    Type: Application
    Filed: October 23, 2024
    Publication date: February 13, 2025
    Inventors: Jonathan PFAFF, Philipp HELLE, Philipp MERKLE, Björn STALLENBERGER, Mischa SIEKMANN, Martin WINKEN, Adam WIECKOWSKI, Wojciech SAMEK, Stephan KALTENSTADLER, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Patent number: 12225198
    Abstract: A block-based decoder is configured to partition a picture of more than one color component and of a color sampling format, according to which each color component is equally sampled, into blocks using a partitioning scheme in which the picture is equally partitioned with respect to each color component. A first color component is decoded in units of the blocks with selecting, for each of intra-predicted first color component blocks, one out of a first set of intra-prediction modes. The first set comprises matrix-based intra prediction modes according to each of which a block inner is predicted by deriving a sample value vector out of neighboring reference samples, computing a matrix-vector product between the sample value vector and a prediction matrix associated with the respective matrix-based intra prediction mode to obtain a prediction vector, and predicting samples in the block inner on the basis of the prediction vector.
    Type: Grant
    Filed: April 1, 2021
    Date of Patent: February 11, 2025
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Jonathan Pfaff, Tobias Hinz, Philipp Helle, Philipp Merkle, Björn Stallenberger, Michael Schäfer, Benjamin Bross, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Publication number: 20250030847
    Abstract: Techniques for block wise encoding and decoding may be applied to encoders, decoders, and methods for encoding or decoding and include assigning, based on a first signalization in the data stream, the predetermined block to the first set or the second set, sorting the assigned set of intra-prediction modes according to intra-prediction modes used for neighboring blocks, neighboring the predetermined block, to obtain a list of intra prediction modes, deriving, for the predetermined block, from the data stream, an index into the list of intra prediction modes, predicting the predetermined block using an intra prediction mode onto which the index points. The decoder or encoder, if the assigned set is the first set of intra-prediction modes, in sorting the assigned set, uses a second mapping which maps each intra-prediction mode of the second set of prediction modes onto a representative one in the first set of intra-prediction modes.
    Type: Application
    Filed: October 2, 2024
    Publication date: January 23, 2025
    Inventors: Jonathan PFAFF, Heiko SCHWARZ, Philipp HELLE, Michael SCHÄFER, Roman RISCHKE, Tobias HINZ, Philipp MERKLE, Björn STALLENBERGER, Martin WINKEN, Mischa SIEKMANN, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20250024028
    Abstract: Concepts for applying intra-prediction in multi-component or color picture or video coding in a manner achieving a more efficient coding are described. A decoder for decoding a picture is configured to derive a color sampling format and a partitioning scheme from one more first syntax elements in a data stream. The decoder is configured to decode a first color component and a second color component. The decoder is configured to determine a second-color-component intra-prediction mode index from the data stream.
    Type: Application
    Filed: July 9, 2024
    Publication date: January 16, 2025
    Inventors: Benjamin BROSS, Detlev MARPE, Philipp MERKLE, Jonathan PFAFF, Heiko SCHWARZ, Björn STALLENBERGER, Adam WIECKOWSKI, Thomas WIEGAND
  • Publication number: 20250008096
    Abstract: An apparatus for decoding a picture from a binary representation of the picture is configured for reconstructing, based on the binary representation, samples of the picture; and for classifying the reconstructed samples for an adaptive in-loop filtering, ALF, using a soft classification such as a convolutional neural network, CNN.
    Type: Application
    Filed: August 9, 2024
    Publication date: January 2, 2025
    Inventors: Jonathan PFAFF, Wang-Q LIM, Björn STALLENBERGER, Johannes ERFURT, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20240430452
    Abstract: An apparatus for block-based predictive decoding of a picture comprises a combiner configured to combine a residual signal a predetermined block of the picture and a reference signal for the predetermined block so as to obtain a first spectrum, the residual signal correcting a prediction error of a prediction of the predetermined block of the picture; a reducer configured to perform thresholding on the first spectrum to obtain a second spectrum so that coefficients below a threshold value are set to a predefined value; an extractor configured to obtain from the second spectrum a modified version of the residual signal; and a reconstructor block configured to decode the predetermined block of the picture from the data stream on the basis of the modified version of the residual signal.
    Type: Application
    Filed: September 10, 2024
    Publication date: December 26, 2024
    Inventors: Phan Hoang Tung NGUYEN, Gerhard TECH, Jonathan PFAFF, Michael SCHÄFER, Jennifer RASCH, Tobias HINZ, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20240422357
    Abstract: A previously encoded or reconstructed version of a neighborhood of a predetermined block to be predicted is exploited so as to result into a more efficient predictive coding of the prediction block. In particular, a spectral decomposition of a region composed of this neighborhood and a first version of a predicted filling of the predetermined block results into a first spectrum which is subject to noise reduction and the thus resulting second spectrum may be subject to a spectral composition, thereby resulting in a modified version of this region including a second version of the predicted filling of the predetermined block. Owing to the exploitation of the already processed, i.e. encoded/reconstructed, neighborhood of the predetermined block, the second version of the predicted filling of the predetermined block tends to improve the coding efficiency.
    Type: Application
    Filed: August 27, 2024
    Publication date: December 19, 2024
    Inventors: Jonathan PFAFF, Martin WINKEN, Christian RUDAT, Heiko SCHWARZ, Thomas WIEGAND, Detlev MARPE
  • Patent number: 12160606
    Abstract: An apparatus for block-wise decoding a picture from a data stream and/or encoding a picture into a data stream, the apparatus supporting at least one intra-prediction mode according to which the intra-prediction signal for a block of a predetermined size of the picture is determined by applying a first template of samples which neighbours the current block onto a neural network. The apparatus may be configured, for a current block differing from the predetermined size, to: resample a second template of samples neighboring the current block, so as to conform with the first template so as to obtain a resampled template; apply the resampled template of samples onto the neural network so as to obtain a preliminary intra-prediction signal; and resample the preliminary intra-prediction signal so as to conform with the current block so as to obtain the intra-prediction signal for the current block.
    Type: Grant
    Filed: January 9, 2023
    Date of Patent: December 3, 2024
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung, e.V.
    Inventors: Jonathan Pfaff, Philipp Helle, Philipp Merkle, Björn Stallenberger, Mischa Siekmann, Martin Winken, Adam Wieckowski, Wojciech Samek, Stephan Kaltenstadler, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Patent number: 12160571
    Abstract: Techniques for block wise encoding and decoding may be applied to encoders, decoders, and methods for encoding or decoding. In one example, there is provided a technique to decode or encode a predetermined block of the picture by assigning, based on a first signalization in the data stream, the predetermined block to a first set or a second set; sorting the assigned set of intra-prediction modes according to intra-prediction modes used for neighboring blocks, neighboring the predetermined block, to obtain a list of intra prediction modes; deriving, for the predetermined block, from the data stream, an index into the list of intra prediction modes; and predicting the predetermined block using an intra prediction mode onto which the index points.
    Type: Grant
    Filed: May 4, 2023
    Date of Patent: December 3, 2024
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Jonathan Pfaff, Heiko Schwarz, Philipp Helle, Michael Schäfer, Roman Rischke, Tobias Hinz, Philipp Merkle, Björn Stallenberger, Martin Winken, Mischa Siekmann, Detlev Marpe, Thomas Wiegand
  • Patent number: 12143591
    Abstract: A video encoder for encoding a video into a data stream using motion compensated prediction for inter predicted blocks, comprising a hypothesis number control, configured to control a number of prediction hypotheses of the inter predicted blocks within a predetermined portion of the video to meet a predetermined criterion.
    Type: Grant
    Filed: July 26, 2022
    Date of Patent: November 12, 2024
    Assignee: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
    Inventors: Martin Winken, Christian Bartnik, Philipp Helle, Mischa Siekmann, Jonathan Pfaff, Detlev Marpe, Heiko Schwarz, Thomas Wiegand
  • Publication number: 20240373064
    Abstract: Techniques for encoding/decoding of video signals are described, e.g. implemented in decoders, encoder, methods and non-transitory storage units storing instructions to perform the methods. A decoder or encoder may be configured to predict a predetermined block of the picture using a plurality of neighbouring samples by reducing the plurality of neighbouring samples to obtain a reduced set of samples values lower, in number of samples, than compared to the plurality of neighbouring samples, subjecting the reduced set of sample values to a linear or affine linear transformation to obtain predicted values for predetermined samples of the predetermined block.
    Type: Application
    Filed: July 17, 2024
    Publication date: November 7, 2024
    Inventors: Jonathan Pfaff, Philipp Helle, Michael Schaefer, Roman Rischke, Tobias Hinz, Philipp Merkle, Bjoern Stallenberger, Martin Winken, Mischa Siekmann, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Publication number: 20240364362
    Abstract: Some embodiments relate to a method, a decoder and/or an encoder for entropy coding of parameters of neural networks and their incremental updates, and in particular to reduced value set coding and history depended significance coding.
    Type: Application
    Filed: July 9, 2024
    Publication date: October 31, 2024
    Applicant: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Gerhard TECH, Paul HAASE, Daniel BECKING, Heiner KIRCHHOFFER, Jonathan PFAFF, Karsten MÜLLER, Wojciech SAMEK, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Patent number: 12120321
    Abstract: The refinement of block-based predictive coding and decoding of pictures as applicable in, e.g., hybrid video codecs, using additional reconstructed samples or residuals is described. An apparatus for block-based predictive decoding of a picture is described. The apparatus includes a combiner configured to combine a residual signal a predetermined block of the picture and a reference signal for the predetermined block so as to obtain a first spectrum, the residual signal correcting a prediction error of a prediction of the predetermined block of the picture; a reducer configured to perform thresholding on the first spectrum to obtain a second spectrum so that coefficients below a threshold value are set to a predefined value; an extractor configured to obtain from the second spectrum a modified version of the residual signal; and a reconstructor block configured to decode the predetermined block of the picture from the data stream on the basis of the modified version of the residual signal.
    Type: Grant
    Filed: September 29, 2020
    Date of Patent: October 15, 2024
    Assignee: FRAUNHOFER-GESELLSCHAFT ZUR FÖRDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
    Inventors: Phan Hoang Tung Nguyen, Gerhard Tech, Jonathan Pfaff, Michael Schäfer, Jennifer Rasch, Tobias Hinz, Heiko Schwarz, Detlev Marpe, Thomas Wiegand