Patents by Inventor Heiko Schwarz

Heiko Schwarz has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20240155126
    Abstract: Block-based coding of a picture is rendered more effective by providing an intra-prediction coding concept according to which a certain block of the picture is intra-prediction coded using a certain intra-coding mode by partitioning the predetermined block into partitions along a certain dimension with the number of partitions being greater than two and/or the partitions being one sample wide along the certain dimension with the partitions being, for reconstruction purposes, sequentially subject to spatial prediction using the intra-prediction coding mode signaled for the certain block followed by correcting the thus obtained predictor using a prediction residual so that for preceding partitions a reconstruction of the samples is available to the decoder at the time of processing the next, then current, partition. Signaling overhead with respect to the partitioning may be left off or may be kept low.
    Type: Application
    Filed: November 30, 2023
    Publication date: May 9, 2024
    Inventors: Santiago DE LUXÁN HERNÁNDEZ, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Patent number: 11979584
    Abstract: Arithmetic encoder for encoding a plurality of symbols is configured to derive an interval size information for an arithmetic encoding of one or more symbol values to be encoded on the basis of one or more state variable value, in a first mode of operation, and to update the one or more state variable values and/or the interval size information for an encoding of one or more subsequent symbol values to be encoded in the first mode; in a second mode, the arithmetic encoder is configured to disable an update of the one or more state variable values and/or of the interval size information. Additionally, further embodiments of arithmetic encoders and decoders and video encoders and decoders are described.
    Type: Grant
    Filed: March 24, 2022
    Date of Patent: May 7, 2024
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Heiner Kirchhoffer, Detlev Marpe, Benjamin Bross, Phan Hoang Tung Nguyen, Christian Rudat, Heiko Schwarz, Thomas Wiegand
  • Publication number: 20240146954
    Abstract: A coding efficiency increase is achieved by using a common signalization within the bitstream with regard to activation of merging and activation of the skip mode. One possible state of one or more syntax elements within the bitstream may signalize for a current sample set of a picture that the sample set is to be merged and has no prediction residual encoded and inserted into the bitstream. A common flag may signalize whether the coding parameters associated with a current sample set are to be set according to a merge candidate or to be retrieved from the bitstream, and whether the current sample set of the picture is to be reconstructed based on a prediction signal depending on the coding parameters associated with the current sample set, without any residual data, or to be reconstructed by refining the prediction signal depending on the coding parameters associated with the current sample set by means of residual data within the bitstream.
    Type: Application
    Filed: August 1, 2023
    Publication date: May 2, 2024
    Inventors: Heiko SCHWARZ, Heiner KIRCHHOFFER, Philipp HELLE, Simon OUDIN, Jan STEGEMANN, Benjamin BROSS, Detlev MARPE, Thomas WIEGAND
  • Patent number: 11973961
    Abstract: A decoder for block-based decoding of picture data of a picture is configured to decode a current block of the picture by determining first and second sets of luma in a neighborhood of the current block, the first set of luma samples having smaller luma values than the second set of luma samples, fitting a straight-line having a linear function through first and second pairs of luma-chroma samples, wherein the first and second pairs of luma-chroma samples are determined based on the first and second sets of luma samples, respectively, and on first and second corresponding chroma samples, respectively, in said neighborhood of the current block, and predicting chroma samples of the current block using luma samples of the current block and said linear function. The present proposal further concerns a respective encoder and corresponding methods for encoding and decoding.
    Type: Grant
    Filed: June 3, 2021
    Date of Patent: April 30, 2024
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e. V.
    Inventors: Christian Helmrich, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Patent number: 11973978
    Abstract: Decoder for block-based decoding of a picture from a data stream, configured to decode an intra-coding mode for a predetermined block of the picture from the data stream, configured to decode a partition dimension flag for the predetermined block of the picture from the data stream and set a partition dimension depending on the partition dimension flag to be horizontal or vertical and the decoder is configured to partition, along the predetermined dimension, the predetermined block into transform partitions which are as wide as the predetermined block perpendicular to predetermined dimension, furthermore configured to decode, for each transform partition, a transform of a prediction residual from the data stream; intra-predicting the predetermined block depending on one or more already reconstructed samples neighboring the predetermined block in a manner depending on the intra-coding mode to obtain a predictor for the predetermined block; and reconstructing the predetermined block by correcting the predictor
    Type: Grant
    Filed: September 10, 2021
    Date of Patent: April 30, 2024
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Santiago De Luxán Hernández, Benjamin Bross, Phan Hoang Tung Nguyen, Valeri George, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Patent number: 11973988
    Abstract: Decoder for decoding a picture from a data stream, configured to check whether a plurality of coding parameters, which are contained in the data stream, relate to a predetermined portion of the picture and control a prediction residual transform mode and a quantization accuracy with respect to the predetermined portion, are indicative of a coding parameter setting corresponding to a lossless prediction residual coding. Responsive to the plurality of coding parameters being indicative of the coding parameter setting corresponding to the lossless prediction residual coding, the decoder is configured to set one or more predetermined coding options relating to one or more tools of the decoder for processing a prediction residual corrected predictive reconstruction with respect to the predetermined portion so that the one or more tools are disabled with respect to the predetermined portion.
    Type: Grant
    Filed: January 4, 2022
    Date of Patent: April 30, 2024
    Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
    Inventors: Christian Helmrich, Benjamin Bross, Tung Hoang Nguyen, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Publication number: 20240137500
    Abstract: In accordance with a first aspect, an improved compression efficiency is achieved by letting a block-wise picture codec support a set of intra-prediction modes according to which the intra-prediction signal for a current block of a picture is determined by applying a set of neighboring samples of the current block onto a neural network. A second aspect of the present application is that, additionally or alternatively to the spending of neural network-based intra-prediction modes, the mode selection may be rendered more effective by the usage of a neural network dedicated to determine a rank or a probability value for each of the set of intra-prediction modes by applying a set of neighboring samples thereonto with the rank or probability value being used for the selection of one intra-prediction mode out of the plurality of intra-prediction modes including or coinciding with the set of intra-prediction modes.
    Type: Application
    Filed: December 19, 2023
    Publication date: April 25, 2024
    Inventors: Jonathan PFAFF, Philipp HELLE, Dominique MANIRY, Thomas WIEGARD, Wojciech SAMEK, Stephan KALTENSTADLER, Heiko SCHWARZ, Detlev MARPE, Mischa SIEKMANN, Martin WINKEN
  • Patent number: 11968348
    Abstract: The usual coding order according to which the reference view is coded prior to the dependent view, and within each view, a depth map is coded subsequent to the respective picture, may be maintained and does lead to a sacrifice of efficiency in performing inter-view redundancy removal by, for example, predicting motion data of the current picture of the dependent view from motion data of the current picture of the reference view. Rather, a depth map estimate of the current picture of the dependent view is obtained by warping the depth map of the current picture of the reference view into the dependent view, thereby enabling various methods of inter-view redundancy reduction more efficiently by bridging the gap between the views.
    Type: Grant
    Filed: January 14, 2022
    Date of Patent: April 23, 2024
    Assignee: GE Video Compression, LLC
    Inventors: Heiko Schwarz, Thomas Wiegand
  • Patent number: 11956429
    Abstract: Block-based coding of a picture is rendered more effective by providing an intra-prediction coding concept according to which a certain block of the picture is intra-prediction coded using a certain intra-coding mode by partitioning the predetermined block into partitions along a certain dimension with the number of partitions being greater than two and/or the partitions being one sample wide along the certain dimension with the partitions being, for reconstruction purposes, sequentially subject to spatial prediction using the intra-prediction coding mode signaled for the certain block followed by correcting the thus obtained predictor using a prediction residual so that for preceding partitions a reconstruction of the samples is available to the decoder at the time of processing the next, then current, partition. Signaling overhead with respect to the partitioning may be left off or may be kept low.
    Type: Grant
    Filed: August 6, 2020
    Date of Patent: April 9, 2024
    Assignee: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
    Inventors: Santiago De Luxán Hernández, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Publication number: 20240114135
    Abstract: Block-based coding of a picture is done by providing an intra-prediction coding concept according to which a certain block of the picture is intra-prediction coded using a certain intra-coding mode by partitioning the predetermined block into partitions which are, for reconstruction purposes, sequentially subject to spatial prediction using the intra-prediction coding mode signaled for the certain block followed by correcting the thus obtained predictor using a prediction residual so that for preceding partitions a reconstruction of the samples is available to the decoder at the time of processing the next, then current, partition.
    Type: Application
    Filed: December 6, 2023
    Publication date: April 4, 2024
    Inventors: Santiago DE LUXÁN HERNÁNDEZ, Philipp MERKLE, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20240114138
    Abstract: In transform coefficient coding, predetermined sets of transforms may be used. Embodiments of the present invention provide an encoder and a decoder for encoding a picture into/from a data stream using block-based prediction and block-based residual coding, the encoder and decoder supporting a set of transforms for the block-based residual coding. The encoder and decoder are configured to determine a transform candidate list of transforms for a current residual block out of the set of transforms based on a height and a width of the current residual block, and to encode/decode a prediction residual within the residual block using one selected transform out of the selectable list of transforms and selectively signal/derive the selected transform in/from the data stream.
    Type: Application
    Filed: December 6, 2023
    Publication date: April 4, 2024
    Inventors: Mischa SIEKMANN, Bjoern STALLENBERGER, Christian BARTNIK, Sebastian BOSSE, Jonathan PFAFF, Philipp HELLE, Jackie MA, Martin WINKEN, Detlev MARPE, Heiko SCHWARZ, Thomas WIEGAND
  • Publication number: 20240106997
    Abstract: Embodiments are described which exploit a finding, according to which a higher compression rate or better rate/distortion ratio may be achieved by adopting or predicting second coding parameters used for encoding a second view of the multi-view signal from first coding parameters used in encoding a first view of the multi-view signal. In other words, the inventors found out that the redundancies between views of a multi-view signal are not restricted to the views themselves, such as the video information thereof, but that the coding parameters in parallelly encoding these views show similarities which may be exploited in order to further improve the coding rate.
    Type: Application
    Filed: December 11, 2023
    Publication date: March 28, 2024
    Inventors: Thomas WIEGAND, Detlev MARPE, Karsten MUELLER, Philipp MERKLE, Gerhard TECH, Hunn RHEE, Heiko SCHWARZ
  • Publication number: 20240089468
    Abstract: A further coding efficiency increase may be achieved if for a current block of a picture, for which the bit stream signals one of supported partitioning patterns, a reversal of the partitioning by block merging is avoided. In particular, if the signaled one of the supported partitioning patterns specifies a subdivision of the block into two or more further blocks, a removal of certain coding parameter candidates for all further blocks, except a first further block of the further blocks in a coding order, is performed. Particularly, those coding parameter candidates are removed from the set of coding parameter candidates for the respective further block, the coding parameters of which are the same as coding parameters associated with any of the further blocks which, when being merged with the respective further block, would result in one of the supported partitioning pattern. This avoids redundancy between partitioning coding and merging coding.
    Type: Application
    Filed: September 1, 2023
    Publication date: March 14, 2024
    Inventors: Philipp HELLE, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND, Simon OUDIN, Benjamin BROSS
  • Publication number: 20240064300
    Abstract: The present invention concerns an apparatus configured to partition a picture into leaf blocks using recursive multi-tree partitioning, block-based encode the picture into a data stream using the partitioning of the picture into the leaf blocks, wherein the apparatus is configured to, in partitioning the picture into the leaf blocks, for a predetermined block which extends beyond a boundary of the picture, reduce an available set of split modes depending on a position at which the boundary of the picture crosses the predetermined block in order to obtain a reduced set of one or more split modes, wherein the apparatus is configured to signal a selected split mode in the data stream.
    Type: Application
    Filed: November 1, 2023
    Publication date: February 22, 2024
    Inventors: Adam WIECKOWSKI, Valeri GEORGE, Tobias HINZ, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND, Jackie MA, Jens BRANDENBURG
  • Patent number: 11910029
    Abstract: Coding schemes for coding a spatially sampled information signal using sub-division and coding schemes for coding a sub-division or a multitree structure are described, wherein representative embodiments relate to picture and/or video coding applications.
    Type: Grant
    Filed: July 26, 2023
    Date of Patent: February 20, 2024
    Assignee: GE Video Compression, LLC
    Inventors: Heiner Kirchhoffer, Martin Winken, Philipp Helle, Detlev Marpe, Heiko Schwarz, Thomas Wiegand
  • Publication number: 20240046100
    Abstract: Embodiments according to the invention comprise an apparatus for decoding neural network parameters, which define a neural network. The apparatus may, optionally, be configured to obtain, e.g. to decode, parameters of a base model, e.g. NB, of the neural network which define one or more layers, e.g. base layers, of the neural network. Furthermore, the apparatus is configured to decode an update model, e.g. NU1 to NUK, which defines a modification of one or more layers, e.g. base layers, of the neural network, and the apparatus is configured modify parameters of a base model of the neural network using the update model, in order to obtain an updated model, e.g. designated as “new model” comprising new model layers LNkj. Moreover, the apparatus is configured to evaluate a skip information, e.g. a skip_row_flag and/or a skip_column_flag, indicating whether a sequence, e.g. a row, or a column or a block, of parameters of the update model is zero or not.
    Type: Application
    Filed: October 16, 2023
    Publication date: February 8, 2024
    Inventors: Paul HAASE, Heiner KIRCHHOFFER, Daniel BECKING, Gerhard TECH, Karsten MUELLER, Wojciech SAMEK, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20240048688
    Abstract: A video encoder is disclosed that is configured to encode a slice of a current picture into a data stream by partitioning the slice into rows of Coding Tree Units (CTUs), select a CTU dependency offset for the slice out of a plurality of CTU dependency offsets, the CTU dependency offset indicating a minimum CTU offset between consecutive CTU rows to be obeyed by a decoder when decoding CTU rows in parallel, and signal the CTU dependency offset in the data stream. The CTU dependency offset is selected based on a slice type of the slice.
    Type: Application
    Filed: September 28, 2023
    Publication date: February 8, 2024
    Inventors: Valeri GEORGE, Tobia HINZ, Jackie MA, Jens BRANDENBURG, Christian LEHMANN, Adam WIECKOWSKI, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND, Thomas SCHIERL
  • Publication number: 20240046093
    Abstract: Disclosed is a decoder for decoding parameters of a neural network, configured to obtain a plurality of neural network parameters of the neural network on the basis of an encoded bitstream, to obtain, e.g. to receive; e.g. to extract from an encoded bitstream, a node information describing a node of a parameter update tree, wherein the node information has a parent node identifier, which is, for example, a unique parent node identifier, for example an integer number, a string, and/or a cryptographic hash, and wherein the node information has a parameter update information, e.g. one or more update instructions, for example a difference signal between initial neural network parameters and a newer version thereof, e.g. corresponding to a child node of the update tree, and to derive one or more neural network parameters using parameter information of a parent node identified by the parent node identifier and using the parameter update information.
    Type: Application
    Filed: October 16, 2023
    Publication date: February 8, 2024
    Inventors: Heiner KIRCHHOFFER, Karsten MÜLLER, Paul HAASE, Daniel BECKING, Gerhard TECH, Wojciech SAMEK, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20240048719
    Abstract: Apparatus for encoding a video having a sequence of frames using rate control, configured to determine a global quantization parameter for the sequence of frames based on a target bit-rate, and to perform a coding pass, coding the sequence of frames, using the global quantization parameter by determining a frame quantization parameter per frame of the sequence of frames on the basis of the global quantization parameter, and subjecting the sequence of frames to R/D optimizing encoding by using, for each frame, the frame quantization parameter determined for the respective frame so as to obtain an encoded version of an associated coding size for the respective frame.
    Type: Application
    Filed: October 13, 2023
    Publication date: February 8, 2024
    Inventors: Christian Helmrich, Ivan Zupancic, Benjamin Bross, Adam Wieckowski, Jonathan Pfaff, Jens Brandenburg, Heiko Schwarz, Detlev Marpe
  • Publication number: 20240048705
    Abstract: A video encoder is configured to entropy encode an absolute value of a quantization level of a current transform coefficient at position (xC, yC), wherein the absolute value is encoded using context adaptive binary arithmetic encoding of bins of a first binarization. The video coder is further configured, for the current transform coefficient, to encode a bin of the first binarization by using a context which is determined based on a sum of minimum absolute values of transform coefficient quantization levels, based on encoded bins of the first binarization, at one or more transform coefficient positions among (xC+1, yC), (xC+2, yC), (xC+1, yC+1), (xC, yC+1), and (xC, yC+2).
    Type: Application
    Filed: October 17, 2023
    Publication date: February 8, 2024
    Inventors: Paul HAASE, Christian RUDAT, Heiner KIRCHHOFFER, Phan Hoang Tung NGUYEN, Mischa SIEKMANN, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND