Patents by Inventor Heiko Schwarz

Heiko Schwarz has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20230025784
    Abstract: Block-based coding of a picture is done by providing an intra-prediction coding concept according to which a certain block of the picture is intra-prediction coded using a certain intra-coding mode by partitioning the predetermined block into partitions which are, for reconstruction purposes, sequentially subject to spatial prediction using the intra-prediction coding mode signaled for the certain block followed by correcting the thus obtained predictor using a prediction residual so that for preceding partitions a reconstruction of the samples is available to the decoder at the time of processing the next, then current, partition.
    Type: Application
    Filed: October 3, 2022
    Publication date: January 26, 2023
    Inventors: Santiago De Luxán Hernández, Philipp Merkle, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Publication number: 20230027719
    Abstract: A proposed intermediate way of handling the renderable portion of the first view results in more efficient coding. Instead of omitting the coding of the renderable portion completely, even more efficient coding of multi-view signals entails merely suppressing the coding of the residual signal within the renderable portion, whereas the prediction parameter coding still takes place from the non-renderable portion of the multi-view signal across the renderable portion so that prediction parameters for the renderable portion may be exploited for predicting parameters for the non-renderable portion. The additional coding rate for transmitting the prediction parameters for the renderable portion may be kept low as this merely aims at forming a continuation of the parameter history across the renderable portion to serve as a basis for prediction parameters of other portions of the multi-view signal.
    Type: Application
    Filed: July 12, 2022
    Publication date: January 26, 2023
    Inventors: Sebastian BOSSE, Heiko SCHWARZ, Thomas WIEGAND, Tobias HINZ
  • Patent number: 11553207
    Abstract: A video decoder and method for supporting a prediction mode for predicting blocks of a video is configured to predict each of the blocks by extrapolating a neighborhood of the respective block into the block along a direction which varies across the respective block.
    Type: Grant
    Filed: December 18, 2020
    Date of Patent: January 10, 2023
    Assignee: FRAUNHOFER-GESELLSCHAFT ZUR FÖRDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
    Inventors: Philipp Merkle, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Publication number: 20230007252
    Abstract: Decoder for decoding a transformed representation of a sample block from a data stream. If a first coded coefficient is located inside a predetermined subarea of the transform coefficient block and if the underlying transform is within a first set of available transforms, the decoder decodes coefficients along a first coefficient scan order. If the transform is within a second set of In available transforms, the decoder decodes coefficients located within the predetermined subarea along a second coefficient scan order, and infers that coefficients located outside the predetermined subarea are zero. The first coefficient scan order is so that coefficients outside the predetermined subarea are scanned between two transform coefficients located inside the predetermined subarea. The second coefficient scan order does not scan any coefficient outside the predetermined subarea between scanning the coefficients within the predetermined subarea.
    Type: Application
    Filed: November 25, 2020
    Publication date: January 5, 2023
    Inventors: Benjamin Bross, Santiago De Luxán Hernández, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Patent number: 11546642
    Abstract: Coding schemes for coding a spatially sampled information signal using sub-division and coding schemes for coding a sub-division or a multitree structure are described, wherein representative embodiments relate to picture and/or video coding applications.
    Type: Grant
    Filed: August 7, 2020
    Date of Patent: January 3, 2023
    Assignee: GE Video Compression, LLC
    Inventors: Heiner Kirchhoffer, Martin Winken, Philipp Helle, Detlev Marpe, Heiko Schwarz, Thomas Wiegand
  • Publication number: 20220416810
    Abstract: Decomposing a value range of the respective syntax elements into a sequence of n partitions with coding the components of z laying within the respective partitions separately with at least one by VLC coding and with at least one by PIPE or entropy coding is used to greatly increase the compression efficiency at a moderate coding overhead since the coding scheme used may be better adapted to the syntax element statistics. Accordingly, syntax elements are decomposed into a respective number n of source symbols si with i=1 . . . n, the respective number n of source symbols depending on as to which of a sequence of n partitions into which a value range of the respective syntax elements is sub-divided, a value z of the respective syntax elements falls into, so that a sum of values of the respective number of source symbols si yields z, and, if n>1, for all i=1 . . . n?1, the value of si corresponds to a range of the ith partition.
    Type: Application
    Filed: June 24, 2022
    Publication date: December 29, 2022
    Inventors: Detlev Marpe, Tung Nguyen, Heiko Schwarz, Thomas Wiegand
  • Publication number: 20220417538
    Abstract: A video decoder is configured to decode a plurality of video frames which are subdivided into a set of one or more slices. The decoder evaluates slice type information indicating whether a slice is encoded using an independent coding mode having no prediction of video content of a current frame on the basis of a video content of a previous frame, or using a single-predictive mode having a prediction of a block of pixels on the basis of one block of pixels of a previous frame, or using a bi-predictive mode having a prediction of a block of pixels on the basis of two or more blocks of pixels of one or more previous frames, to select a mode of operation for decoding a slice. The video decoder includes an arithmetic decoder for providing a decoded binary sequence on the basis of an encoded representation of the binary sequence.
    Type: Application
    Filed: August 31, 2022
    Publication date: December 29, 2022
    Inventors: Jan STEGEMANN, Heiner KIRCHHOFFER, Detlev MARPE, Heiko SCHWARZ, Thomas WIEGAND
  • Patent number: 11533486
    Abstract: The coding of a media signal is rendered more efficient by describing the media signal using a sequence of samples and sequentially encoding this sequence by selecting, for a current sample, a set of quantization levels out of a plurality of quantization level sets depending on indices encoded into the data stream for previous samples of the sequence of samples, quantizing the current sample onto one level of the set of quantization levels, and encoding a quantization index to the one level for the current sample into the data stream. In other words, scalar quantization of the individual samples of the sequence of samples is used, but it is rendered dependent on quantization indices encoded into the data stream for previous samples of the sequence of samples.
    Type: Grant
    Filed: September 25, 2020
    Date of Patent: December 20, 2022
    Assignee: Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V.
    Inventors: Heiko Schwarz, Thomas Wiegand, Phan Hoang Tung Nguyen, Detlev Marpe
  • Publication number: 20220400271
    Abstract: Scalable video coding is rendered more efficient by deriving/selecting a subblock subdivision to be used for enhancement layer prediction, among a set of possible subblock subdivisions of an enhancement layer block by evaluating the spatial variation of the base layer coding parameters over the base layer signal. By this measure, less of the signalization overhead has to be spent on signaling this subblock subdivision within the enhancement layer data stream, if any. The subblock subdivision thus selected may be used in predictively coding/decoding the enhancement layer signal.
    Type: Application
    Filed: June 23, 2022
    Publication date: December 15, 2022
    Inventors: Tobias HINZ, Haricharan LAKSHMAN, Jan STEGEMANN, Philipp HELLE, Mischa SIEKMANN, Karsten SUEHRING, Detlev MARPE, Heiko SCHWARZ, Christian BARTNIK, Ali Atef Ibrahim KHAIRAT ABDELHAMID, Heiner KIRCHHOFFER, Thomas WIEGAND
  • Publication number: 20220393986
    Abstract: Embodiments according to a first aspect of the present invention are based on the idea, that neural network parameters may be compressed more efficiently by using a non-constant quantizer, but varying same during coding the neural network parameters, namely by selecting a set of reconstruction levels depending on quantization indices decoded from, or respectively encoded, into the data stream for previous or respectively previously encoded neural network parameters. Embodiments according to a second aspect of the present invention are based on the idea that a more efficient neural network coding may be achieved when done in stages—called reconstruction layers to distinguish them from the layered composition of the neural network in neural layers—and if the parametrizations provided in these stages are then, neural network parameter-wise combined to yield a neural network parametrization improved compared to any of the stages.
    Type: Application
    Filed: June 17, 2022
    Publication date: December 8, 2022
    Inventors: Paul HAASE, Heiner KIRCHHOFFER, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Patent number: 11523098
    Abstract: This disclosure is directed to coding a multi-view signal, which includes processing a list of plurality of motion vector candidates associated with a coding block of a current picture in a dependent view of the multi-view signal. Such processing includes estimating a first motion vector based on a second motion vector associated with a reference block in a current picture of a reference view of the multi-view signal, the reference block corresponding to the coding block of the current picture in the dependent view. The first motion vector is added into the list, and an index is used that specifies at least one candidate from the list to be used for motion-compensated prediction. The coding block in the current picture is coded by performing the motion-compensated prediction based on the at least one candidate indicated by the index.
    Type: Grant
    Filed: December 21, 2020
    Date of Patent: December 6, 2022
    Assignee: GE Video Compression, LLC
    Inventors: Heiko Schwarz, Thomas Wiegand
  • Patent number: 11503339
    Abstract: Techniques for encoding/decoding of video signals are described, e.g. implemented in decoders, encoder, methods and non-transitory storage units storing instructions to perform the methods. A decoder or encoder may be configured to predict a predetermined block of the picture using a plurality of neighbouring samples by reducing the plurality of neighbouring samples to obtain a reduced set of samples values lower, in number of samples, than compared to the plurality of neighbouring samples, subjecting the reduced set of sample values to a linear or affine linear transformation to obtain predicted values for predetermined samples of the predetermined block.
    Type: Grant
    Filed: June 17, 2021
    Date of Patent: November 15, 2022
    Assignee: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
    Inventors: Jonathan Pfaff, Philipp Helle, Michael Schaefer, Roman Rischke, Tobias Hinz, Philipp Merkle, Bjoern Stallenberger, Martin Winken, Mischa Siekmann, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Publication number: 20220360793
    Abstract: A video encoder for encoding a video into a data stream using motion compensated prediction for inter predicted blocks, comprising a hypothesis number control, configured to control a number of prediction hypotheses of the inter predicted blocks within a predetermined portion of the video to meet a predetermined criterion.
    Type: Application
    Filed: July 26, 2022
    Publication date: November 10, 2022
    Inventors: Martin WINKEN, Christian BARTNIK, Philipp HELLE, Mischa SIEKMANN, Jonathan PFAFF, Detlev MARPE, Heiko SCHWARZ, Thomas WIEGAND
  • Patent number: 11477467
    Abstract: Scalable video coding is rendered more efficient by deriving/selecting a subblock subdivision to be used for enhancement layer prediction, among a set of possible subblock subdivisions of an enhancement layer block by evaluating the spatial variation of the base layer coding parameters over the base layer signal. By this measure, less of the signalization overhead has to be spent on signaling this subblock subdivision within the enhancement layer data stream, if any. The subblock subdivision thus selected may be used in predictively coding/decoding the enhancement layer signal.
    Type: Grant
    Filed: May 11, 2020
    Date of Patent: October 18, 2022
    Assignee: GE Video Compression, LLC
    Inventors: Tobias Hinz, Haricharan Lakshman, Jan Stegemann, Philipp Helle, Mischa Siekmann, Karsten Suehring, Detlev Marpe, Heiko Schwarz, Christian Bartnik, Ali Atef Ibrahim Khairat Abdelhamid, Heiner Kirchhoffer, Thomas Wiegand
  • Publication number: 20220321886
    Abstract: A decoder configured to decode residual levels, and sequentially dequantize the residual levels by selecting a quantizer out of a set of default quantizers depending on a current transition state, and updating the current transition state depending on a current residual level's characteristic obtained by an application of a binary function onto the current residual level, and depending on a quantization mode information contained in the data stream. The current transition state transitions, according to a surjective mapping which depends on the quantization mode information, from a domain of combinations of a set of one or more transition states with the current residual level's characteristic onto the set of one or more transition states, wherein a cardinality of the set of one or more transition states differs depending on the quantization mode information.
    Type: Application
    Filed: June 13, 2022
    Publication date: October 6, 2022
    Inventors: Heiko SCHWARZ, Soenke SCHMIDT, Paul HAASE, Phan Hoang Tung NGUYEN, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20220321881
    Abstract: In accordance with a first aspect, an improved compression efficiency is achieved by letting a block-wise picture codec support a set of intra-prediction modes according to which the intra-prediction signal for a current block of a picture is determined by applying a set of neighboring samples of the current block onto a neural network. A second aspect of the present application is that, additionally or alternatively to the spending of neural network-based intra-prediction modes, the mode selection may be rendered more effective by the usage of a neural network dedicated to determine a rank or a probability value for each of the set of intra-prediction modes by applying a set of neighboring samples thereonto with the rank or probability value being used for the selection of one intra-prediction mode out of the plurality of intra-prediction modes including or coinciding with the set of intra-prediction modes.
    Type: Application
    Filed: June 13, 2022
    Publication date: October 6, 2022
    Inventors: Jonathan PFAFF, Philipp HELLE, Dominique MANIRY, Thomas WIEGARD, Wojciech SAMEK, Stephan KALTENSTADLER, Heiko SCHWARZ, Detlev MARPE, Mischa SIEKMANN, Martin WINKEN
  • Patent number: 11463694
    Abstract: Block-based coding of a picture is done by providing an intra-prediction coding concept according to which a certain block of the picture is intra-prediction coded using a certain intra-coding mode by partitioning the predetermined block into partitions which are, for reconstruction purposes, sequentially subject to spatial prediction using the intra-prediction coding mode signaled for the certain block followed by correcting the thus obtained predictor using a prediction residual so that for preceding partitions a reconstruction of the samples is available to the decoder at the time of processing the next, then current, partition.
    Type: Grant
    Filed: January 13, 2021
    Date of Patent: October 4, 2022
    Assignee: Fraumhofer-Gesellschaft zur Forderung der angewandten Forschung e.V.
    Inventors: Santiago De Luxán Hernández, Philipp Merkle, Heiko Schwarz, Detlev Marpe, Thomas Wiegand
  • Patent number: 11457230
    Abstract: A video decoder is configured to decode a plurality of video frames which are subdivided into a set of one or more slices. The decoder evaluates slice type information indicating whether a slice is encoded using an independent coding mode having no prediction of video content of a current frame on the basis of a video content of a previous frame, or using a single-predictive mode having a prediction of a block of pixels on the basis of one block of pixels of a previous frame, or using a bi-predictive mode having a prediction of a block of pixels on the basis of two or more blocks of pixels of one or more previous frames, to select a mode of operation for decoding a slice. The video decoder includes an arithmetic decoder for providing a decoded binary sequence on the basis of an encoded representation of the binary sequence.
    Type: Grant
    Filed: September 29, 2020
    Date of Patent: September 27, 2022
    Assignee: Fraunhofer-Gesellschaft zur Forderung der angewandten Forschung e.V.
    Inventors: Jan Stegemann, Heiner Kirchhoffer, Detlev Marpe, Heiko Schwarz, Thomas Wiegand
  • Publication number: 20220303545
    Abstract: An apparatus for determining visual activity information for a predetermined picture block of a video sequence including a plurality of video frames, the plurality of video frames including a current video frame and one or more timely-preceding video frames, wherein the one or more timely-preceding video frames precede the current video frame in time, is provided. The apparatus is configured to receive the predetermined picture block of each of the one or more timely-preceding video frames and the predetermined picture block of the current video frame. Moreover, the apparatus is configured to determine the visual activity information depending on the predetermined picture block of the current video frame and depending on the predetermined picture block of each of the one or more timely-preceding video frames and depending on a temporal high-pass filter.
    Type: Application
    Filed: April 18, 2022
    Publication date: September 22, 2022
    Inventors: Christian HELMRICH, Sebastian BOSSE, Mischa SIEKMANN, Soeren BECKER, Johannes ERFURT, Heiko SCHWARZ, Detlev MARPE, Thomas WIEGAND
  • Publication number: 20220303519
    Abstract: Embodiments are described which exploit a finding, according to which a higher compression rate or better rate/distortion ratio may be achieved by adopting or predicting second coding parameters used for encoding a second view of the multi-view signal from first coding parameters used in encoding a first view of the multi-view signal. In other words, the inventors found out that the redundancies between views of a multi-view signal are not restricted to the views themselves, such as the video information thereof, but that the coding parameters in parallelly encoding these views show similarities which may be exploited in order to further improve the coding rate.
    Type: Application
    Filed: April 7, 2022
    Publication date: September 22, 2022
    Inventors: Thomas WIEGAND, Detlev MARPE, Karsten MUELLER, Philipp MERKLE, Gerhard TECH, Hunn RHEE, Heiko SCHWARZ