Patents by Inventor Heiko Schwarz

Heiko Schwarz has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11895290
    Abstract: A video coding method and apparatus for codes a video in a data stream using block-based predictive coding by deriving a collection of one or more primitive predictions for a block using a set of primitive prediction modes adapted for predicting blocks of a picture of the video. Each primitive prediction in the collection of primitive predictions is derived using a respective primitive prediction mode from the set of primitive prediction modes. A composite prediction signal for the block is composed by combining primitive predictions from the collection of one or more primitive predictions, and the block is predicted using the composite prediction signal.
    Type: Grant
    Filed: March 21, 2022
    Date of Patent: February 6, 2024
    Assignee: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
    Inventors: Thomas Wiegand, Detlev Marpe, Heiko Schwarz, Martin Winken, Christian Bartnik, Jonathan Pfaff, Philipp Helle, Mischa Siekmann
  • Patent number: 11889066
    Abstract: In accordance with a first aspect, an improved compression efficiency is achieved by letting a block-wise picture codec support a set of intra-prediction modes according to which the intra-prediction signal for a current block of a picture is determined by applying a set of neighboring samples of the current block onto a neural network. A second aspect of the present application is that, additionally or alternatively to the spending of neural network-based intra-prediction modes, the mode selection may be rendered more effective by the usage of a neural network dedicated to determine a rank or a probability value for each of the set of intra-prediction modes by applying a set of neighboring samples thereonto with the rank or probability value being used for the selection of one intra-prediction mode out of the plurality of intra-prediction modes including or coinciding with the set of intra-prediction modes.
    Type: Grant
    Filed: June 13, 2022
    Date of Patent: January 30, 2024
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Jonathan Pfaff, Philipp Helle, Dominique Maniry, Thomas Wiegard, Wojciech Samek, Stephan Kaltenstadler, Heiko Schwarz, Detlev Marpe, Mischa Siekmann, Martin Winken
  • Publication number: 20240031585
    Abstract: A higher coding efficiency for coding a significance map indicating positions of significant transform coefficients within a transform coefficient block is achieved by the scan order by which the sequentially extracted syntax elements indicating, for associated positions within the transform coefficient block, as to whether at the respective position a significant or insignificant transform coefficient is situated, are sequentially associated to the positions of the transform coefficient block, among the positions of the transform coefficient block depends on the positions of the significant transform coefficients indicated by previously associated syntax elements. Alternatively, the first-type elements may be context-adaptively entropy decoded using contexts which are individually selected for each of the syntax elements dependent on a number of significant transform coefficients in a neighborhood of the respective syntax element, indicated as being significant by any of the preceding syntax elements.
    Type: Application
    Filed: August 3, 2023
    Publication date: January 25, 2024
    Inventors: Heiner KIRCHHOFFER, Heiko SCHWARZ, Tung NGUYEN, Detlev MARPE, Thomas WIEGAND
  • Patent number: 11882284
    Abstract: In transform coefficient coding, predetermined sets of transforms may be used. Embodiments of the present invention provide an encoder and a decoder for encoding a picture into/from a data stream using block-based prediction and block-based residual coding, the encoder and decoder supporting a set of transforms for the block-based residual coding. The encoder and decoder are configured to determine a transform candidate list of transforms for a current residual block out of the set of transforms based on a height and a width of the current residual block, and to encode/decode a prediction residual within the residual block using one selected transform out of the selectable list of transforms and selectively signal/derive the selected transform in/from the data stream.
    Type: Grant
    Filed: September 28, 2020
    Date of Patent: January 23, 2024
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Mischa Siekmann, Bjoern Stallenberger, Christian Bartnik, Sebastian Bosse, Jonathan Pfaff, Philipp Helle, Jackie Ma, Martin Winken, Detlev Marpe, Heiko Schwarz, Thomas Wiegand
  • Publication number: 20240022768
    Abstract: Coding schemes for coding a spatially sampled information signal using sub-division and coding schemes for coding a sub-division or a multitree structure are described, wherein representative embodiments relate to picture and/or video coding applications.
    Type: Application
    Filed: July 26, 2023
    Publication date: January 18, 2024
    Inventors: Heiner KIRCHHOFFER, Martin WINKEN, Philipp HELLE, Detlev MARPE, Heiko SCHWARZ, Thomas WIEGAND
  • Patent number: 11876964
    Abstract: Block-based coding of a picture is done by providing an intra-prediction coding concept according to which a certain block of the picture is intra-prediction coded using a certain intra-coding mode by partitioning the predetermined block into partitions which are, for reconstruction purposes, sequentially subject to spatial prediction using the intra-prediction coding mode signaled for the certain block followed by correcting the thus obtained predictor using a prediction residual so that for preceding partitions a reconstruction of the samples is available to the decoder at the time of processing the next, then current, partition.
    Type: Grant
    Filed: October 3, 2022
    Date of Patent: January 16, 2024
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Santiago De Luxán Hernández, Philipp Merkle, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Patent number: 11856219
    Abstract: A proposed intermediate way of handling the renderable portion of the first view results in more efficient coding. Instead of omitting the coding of the renderable portion completely, even more efficient coding of multi-view signals entails merely suppressing the coding of the residual signal within the renderable portion, whereas the prediction parameter coding still takes place from the non-renderable portion of the multi-view signal across the renderable portion so that prediction parameters for the renderable portion may be exploited for predicting parameters for the non-renderable portion. The additional coding rate for transmitting the prediction parameters for the renderable portion may be kept low as this merely aims at forming a continuation of the parameter history across the renderable portion to serve as a basis for prediction parameters of other portions of the multi-view signal.
    Type: Grant
    Filed: July 12, 2022
    Date of Patent: December 26, 2023
    Assignee: GE Video Compression, LLC
    Inventors: Sebastian Bosse, Heiko Schwarz, Thomas Wiegand, Tobias Hinz
  • Patent number: 11856240
    Abstract: Coding schemes for coding a spatially sampled information signal using sub-division and coding schemes for coding a sub-division or a multitree structure are described, wherein representative embodiments relate to a picture and/or video coding applications.
    Type: Grant
    Filed: November 4, 2022
    Date of Patent: December 26, 2023
    Assignee: GE VIDEO COMPRESSION, LLC
    Inventors: Heiner Kirchhoffer, Martin Winken, Philipp Helle, Detlev Marpe, Heiko Schwarz, Thomas Wiegand
  • Publication number: 20230412803
    Abstract: A video decoder for decoding an encoded video signal including encoded picture data and indication data of a picture of a video to reconstruct the picture of the video is provided. The video decoder includes an interface configured for receiving the encoded video signal, and a data decoder configured for reconstructing the picture of the video by decoding the encoded picture data using the indication data. The picture is partitioned into a plurality of coding areas. One or more coding areas of the plurality of coding areas include two or more coding tree units of the plurality of coding tree units, wherein each coding area of the one or more coding areas which includes two or more coding tree units exhibits a coding order for the two or more coding tree units of the coding area.
    Type: Application
    Filed: August 14, 2023
    Publication date: December 21, 2023
    Inventors: Valeri GEORGE, Tobias HINZ, Jackie MA, Yago SÁNCHEZ DE LA FUENTE, Robert SKUPIN, Thomas SCHIERL, Jens BRANDENBURG, Christian LEHMANN, Adam WIECKOWSKI, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Patent number: 11843774
    Abstract: The present invention concerns an apparatus configured to partition a picture into leaf blocks using recursive multi-tree partitioning, block-based encode the picture into a data stream using the partitioning of the picture into the leaf blocks, wherein the apparatus is configured to, in partitioning the picture into the leaf blocks, for a predetermined block which extends beyond a boundary of the picture, reduce an available set of split modes depending on a position at which the boundary of the picture crosses the predetermined block in order to obtain a reduced set of one or more split modes, wherein the apparatus is configured to signal a selected split mode in the data stream.
    Type: Grant
    Filed: September 20, 2021
    Date of Patent: December 12, 2023
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Adam Wieckowski, Valeri George, Tobias Hinz, Heiko Schwarz, Detlev Marpe, Thomas Wiegand, Jackie Ma, Jens Brandenburg
  • Patent number: 11843757
    Abstract: Embodiments are described which exploit a finding, according to which a higher compression rate or better rate/distortion ratio may be achieved by adopting or predicting second coding parameters used for encoding a second view of the multi-view signal from first coding parameters used in encoding a first view of the multi-view signal. In other words, the inventors found out that the redundancies between views of a multi-view signal are not restricted to the views themselves, such as the video information thereof, but that the coding parameters in parallelly encoding these views show similarities which may be exploited in order to further improve the coding rate.
    Type: Grant
    Filed: April 7, 2022
    Date of Patent: December 12, 2023
    Assignee: GE Video Compression, LLC
    Inventors: Thomas Wiegand, Detlev Marpe, Karsten Mueller, Philipp Merkle, Gerhard Tech, Hunn Rhee, Heiko Schwarz
  • Publication number: 20230388518
    Abstract: A coding concept for encoding a picture uses a multi-layered convolutional neural network for determining a feature representation of the picture, the feature representation comprising first to third partial representations which have mutually different resolutions. Further, an encoder for encoding a picture determines a quantization of the picture using a polynomial function which provides an estimated distortion associated with the quantization.
    Type: Application
    Filed: August 11, 2023
    Publication date: November 30, 2023
    Inventors: Jonathan PFAFF, Michael SCHÄFER, Sophie PIENTKA, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20230379471
    Abstract: Concepts are presented which achieve a more efficient coding of coefficients of a transform block by use of dependent quantization and context adaptive entropy coding or achieve a coding of coefficients of a transform block in a manner which allows a more efficient coding even if a usage of dependent quantization is combined with the usage of context adaptive entropy coding.
    Type: Application
    Filed: April 13, 2023
    Publication date: November 23, 2023
    Inventors: Heiko SCHWARZ, Thomas WIEGAND, Phan Hoang Tung NGUYEN, Detlev MARPE
  • Patent number: 11825091
    Abstract: A video encoder is configured to entropy encode an absolute value of a quantization level of a current transform coefficient at position (xC, yC), wherein the absolute value is encoded using context adaptive binary arithmetic encoding of bins of a first binarization. The video coder is further configured, for the current transform coefficient, to encode a bin of the first binarization by using a context which is determined based on a sum of minimum absolute values of transform coefficient quantization levels, based on encoded bins of the first binarization, at one or more transform coefficient positions among (xC+1, yC), (xC+2, yC), (xC+1, yC+1), (xC, yC+1), and (xC, yC+2).
    Type: Grant
    Filed: February 1, 2023
    Date of Patent: November 21, 2023
    Assignee: FRAUNHOFER-GESELLSCHAFT ZUR FÖRDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
    Inventors: Paul Haase, Christian Rudat, Heiner Kirchhoffer, Phan Hoang Tung Nguyen, Mischa Siekmann, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Publication number: 20230353788
    Abstract: A previously encoded or reconstructed version of a neighborhood of a predetermined block to be predicted is exploited so as to result into a more efficient predictive coding of the prediction block. In particular, a spectral decomposition of a region composed of this neighborhood and a first version of a predicted filling of the predetermined block results into a first spectrum which is subject to noise reduction and the thus resulting second spectrum may be subject to a spectral composition, thereby resulting in a modified version of this region including a second version of the predicted filling of the predetermined block. Owing to the exploitation of the already processed, i.e. encoded/reconstructed, neighborhood of the predetermined block, the second version of the predicted filling of the predetermined block tends to improve the coding efficiency.
    Type: Application
    Filed: June 20, 2023
    Publication date: November 2, 2023
    Inventors: Jonathan PFAFF, Martin WINKEN, Christian RUDAT, Heiko SCHWARZ, Thomas WIEGAND, Detlev MARPE
  • Patent number: 11805241
    Abstract: Concepts are described which enable improved suitability of a respective codec for parallel processing at encoder and/or decoder.
    Type: Grant
    Filed: September 28, 2020
    Date of Patent: October 31, 2023
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Valeri George, Tobias Hinz, Jackie Ma, Jens Brandenburg, Christian Lehmann, Adam Wieckowski, Heiko Schwarz, Detlev Marpe, Thomas Wiegand, Thomas Schierl
  • Publication number: 20230345012
    Abstract: A better rate distortion ratio is achieved by making interrelationships between coding parameters of different planes available for exploitation for the aim of redundancy reduction despite the additional overhead resulting from the need to signal the inter-plane prediction information to the decoder. In particular, the decision to use inter plane prediction or not may be performed for a plurality of planes individually. Additionally or alternatively, the decision may be done on a block basis considering one secondary plane.
    Type: Application
    Filed: June 30, 2023
    Publication date: October 26, 2023
    Inventors: Martin WINKEN, Heiner KIRCHHOFFER, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20230328271
    Abstract: Rendered portions of a renderable portion of a first view of a multi-view signal are introduced into a prediction loop of a multi-view encoder to form a reference signal for the block-based prediction of the encoder's view predictor may improve the prediction and thereby increasing the coding efficiency. The introduction may be performed by completely inserting the renderable portion into the prediction loop to form new reference pictures with their own reference picture indices in addition to reference pictures obtained by the reconstructed version of the multi-view signal of the block-based prediction. Alternatively, the rendered portion may be introduced into the prediction loop completely, but with replacing the normal prediction reference signal, i.e., the normally reconstructed signal as obtained by block-based prediction. Alternatively, it may be signaled within the data stream which portions of the renderable portion are used to replace respective portions in the normally reconstructed signal.
    Type: Application
    Filed: June 8, 2023
    Publication date: October 12, 2023
    Inventors: Sebastian BOSSE, Heiko SCHWARZ, Thomas WIEGAND, Heribert BRUST
  • Publication number: 20230328287
    Abstract: Apparatus (54) for decoding a predetermined block (18) of a picture using intra-prediction, configured to read, from a data stream (12), a mode index (200) using a binarization code (202). The mode index (200) points to one out of a list (204) of matrix-based intra-prediction modes. The apparatus is configured to predict samples (108) of the predetermined block (18) by computing a matrix-vector product (206) between an input vector (102) derived from reference samples (17) in a neighbourhood of the predetermined block (18) and a prediction matrix (19) associated with the matrix-based intra-prediction mode (k) pointed to by the mode index (200) and associating components (210) of an output vector (208) obtained by the matrix-vector product (206) onto sample positions (104) of the predetermined block. The list (204) of matrix-based intra-prediction modes consists of an even number of matrix-based intra-prediction modes.
    Type: Application
    Filed: September 24, 2020
    Publication date: October 12, 2023
    Inventors: Jonathan PFAFF, Björn STALLENBERGER, Michael SCHAEFER, Philipp MERKLE, Tobias HINZ, Philipp HELLE, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND, Benjamin BROSS, Martin WINKEN, Mischa SIEKMANN
  • Publication number: 20230328242
    Abstract: A picture in a data stream is subdivided into different size transform bocks. For a transform block, a transformation is selected by checking whether the block's size exceeds a threshold. If it does, a default technique is used to select the transformation. If it does not, a transformation is used from a list of transformations as identified by an index, the list including an identity transformation and non-identity transformations. A coefficients block in the data stream is decoded, and if the selected transformation is the identity transformation, then the coefficients block is used as a residual sample array, and if the selected transformation is not the identity transformation, the coefficients block is subjected to an inverse transformation corresponding to the selected transformation to obtain a residual sample array used to correct a prediction for the transform block.
    Type: Application
    Filed: June 13, 2023
    Publication date: October 12, 2023
    Inventors: Benjamin BROSS, Phan Hoang Tung NGUYEN, Paul KEYDEL, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND, Christian HELMRICH