Abstract: A multi-view input image covering multiple sampled views is received. A multi-view layered image stack is generated from the multi-view input image. A target view of a viewer to an image space depicted by the multi-view input image is determined based on user pose data. The target view is used to select user pose selected sampled views from among the multiple sampled views. Layered images for the user pose selected sampled views, along with alpha maps and beta scale maps for the user pose selected sampled views are encoded into a video signal to cause a recipient device of the video signal to generate a display image for rendering on the image display.

Abstract: A video delivery system for luminance adjustment based upon a viewer adaptation state comprises a processor configured to: receive a source image including a current image frame including metadata corresponding to a mean luminance value of the current image frame, and the source image including an upcoming image frame including metadata corresponding to a mean luminance value of the upcoming image frame. The processor is configured to determine an ambient luminance value based on an ambient luminance, determine an incident luminance value based on the ambient luminance value and the mean luminance value, determine a difference between a current pupil size and a target pupil size, and generate an output image by modifying the source image based on a luminance adjustment factor, the luminance adjustment factor being a function of the difference between the current pupil size and the target pupil size.

Abstract: The present document describes a method (400) for rendering an ambisonics signal using a loudspeaker arrangement comprising S loudspeakers. The method (400) comprises converting (401) a set of N ambisonics channel signals (111) into a set of unfiltered pre-rendered signals (211), with N>1 and S>1. Furthermore, the method (400) comprises performing (402) near field compensation, referred to as NFC, filtering of M unfiltered pre-rendered signals (211) of the set of unfiltered pre-rendered signals (211) to provide a set of S filtered loudspeaker channel signals (114) for rendering using the corresponding S loudspeakers.

Abstract: Diffuse or spatially large audio objects may be identified for special processing. A decorrelation process may be performed on audio signals corresponding to the large audio objects to produce decorrelated large audio object audio signals. These decorrelated large audio object audio signals may be associated with object locations, which may be stationary or time-varying locations. For example, the decorrelated large audio object audio signals may be rendered to virtual or actual speaker locations. The output of such a rendering process may be input to a scene simplification process. The decorrelation, associating and/or scene simplification processes may be performed prior to a process of encoding the audio data.

Abstract: The present disclosure relates to a method and system for predicting a future orientation of an orientation tracker (100). The method comprising obtaining a sequence of angular velocity samples, each angular velocity sample indicating an angular velocity at a point in time and obtaining a sequence of angular acceleration samples, each angular acceleration sample indicating an acceleration or deceleration of the angular velocity at each point in time. Wherein said method further comprises determining (S5a), for each point in time where the angular velocity is accelerating, a predicted orientation of the orientation tracker (100) based on a first order prediction of an accumulated rotation of the orientation tracker (100) and determining (S5c), for each point in time where the angular velocity is decelerating, a predicted orientation of the orientation tracker (100) based on a second order prediction of the accumulated rotation of the orientation tracker (100).

Abstract: A multi-input, multi-output audio process is implemented as a linear system for use in an audio filterbank to convert a set of frequency-domain input audio signals into a set of frequency-domain output signals. A transfer function from one input to one output is defined as a frequency dependent gain function. In some implementations, the transfer function includes a direct component that is substantially defined as a frequency dependent gain, and one or more decorrelated components that have frequency-varying group phase response. The transfer function is formed from a set of sub-band functions, with each sub-band function being formed from a set of corresponding component transfer functions including direct component and one or more decorrelated components.

Abstract: Tensor-Product B-splines (TPB) have been shown to improve video quality when used to represent reshaping functions to map reshaped standard dynamic range content into high dynamic range (HDR) content; however, TPB prediction is computationally intensive and may not be supported by legacy devices. Methods and systems for backwards-compatible signaling of TPB-related metadata and a fast TPB prediction method are presented to overcome both of these limitations. Computation overhead for a TPB-based 3D look-up table is reduced by using temporary two-dimensional arrays. A remapping of the most significant bits of a legacy bit-depth parameter allows for backwards compatibility.

Abstract: A system for managing user-generated content (UGC) and professionally generated content (PGC) is disclosed. The system is programmed to receive digital audio data having two channels from a social media platform. The system is programmed to extract spatial features that capture differences in the two channels from the digital audio data. The system is programmed to also extract temporal features, spectral features, and background features from the digital audio data. The system is programmed to then use the extracted features to determine whether to process the digital audio data as UGC or PGC before playback.

Abstract: A system for detecting speech from reverberant signals is disclosed. The system is programmed to receive spectral temporal amplitude data in the modulation frequency domain. The system is programmed to then enhance the spectral temporal amplitude data by reducing reverberation and other noise as well as smoothing based on certain properties of the spectral temporal spectrogram associated with the spectral temporal amplitude data. Next, the system is programmed to compute various features related to the presence of speech based on the enhanced spectral temporal amplitude data and other data in the modulation frequency domain or in the (acoustic) frequency domain. The system is programmed to then determine an extent of speech present in the audio data corresponding to the received spectral temporal amplitude data based on the various features. The system can be programmed to transmit the extent of speech present to an output device.

Abstract: Methods and systems for the super resolution of high dynamic range (HDR) video are described. Given a sequence of video frames, a current frame and two or more neighboring frames are processed by a neural-network (NN) feature extraction module, followed by a NN upscaling module, and a NN reconstruction module. In parallel, the current frame is upscaled using traditional up-sampling to generate an intermediate up-sampled frame. The output of the reconstruction module is added to the intermediate up-sampled frame to generate an output frame. Additional traditional up-sampling may be performed on the output frame to match the desired up-scaling factor, beyond the up-scaling factor for which the neural network was trained.

Abstract: Soundfield signals such as e.g. Ambisonics carry a representation of a desired sound field. The Ambisonics format is based on spherical harmonic decomposition of the soundfield, and Higher Order Ambisonics (HOA) uses spherical harmonics of at least 2nd order. However, commonly used loudspeaker setups are irregular and lead to problems in decoder design. A method for improved decoding an audio soundfield representation for audio playback comprises calculating a panning function (W) using a geometrical method based on the positions of a plurality of loudspeakers and a plurality of source directions, calculating a mode matrix (?) from the loudspeaker positions, calculating a pseudo-inverse mode matrix (?+) and decoding the audio soundfield representation. The decoding is based on a decode matrix (D) that is obtained from the panning function (W) and the pseudo-inverse mode matrix (?+).

Abstract: Embodiments are disclosed for bitrate distribution in immersive voice and audio services. In an embodiment, a method of encoding an IVAS bitstream comprises: receiving an input audio signal; downmixing the input audio signal into one or more downmix channels and spatial metadata; reading a set of one or more bitrates for the downmix channels and a set of quantization levels for the spatial metadata from a bitrate distribution control table; determining a combination of the one or more bitrates for the downmix channels; determining a metadata quantization level from the set of metadata quantization levels using a bitrate distribution process; quantizing and coding the spatial metadata using the metadata quantization level; generating, using the combination of one or more bitrates, a downmix bitstream for the one or more downmix channels; combining the downmix bitstream, the quantized and coded spatial metadata and the set of quantization levels into the IVAS bitstream.

Abstract: In some embodiments, a method for processing an audio signal in an audio processing apparatus is disclosed. The method includes receiving an audio signal and a parameter, the parameter indicating a location of an auditory event boundary. An audio portion between consecutive auditory event boundaries constitutes an auditory event. The method further includes applying a modification to the audio signal based in part on an occurrence of the auditory event. The parameter may be generated by monitoring a characteristic of the audio signal and identifying a change in the characteristic.

Abstract: A method of audio processing includes generating a detection score based on the partial loudnesses of a reference audio signal, extracted audio objects, extracted bed channels, a rendered audio signal and a channel-based audio signal. The detection score is indicative of an audio artifact in one or more of the audio objects and the bed channels. The extracted audio objects and extracted bed channels may be modified, in accordance with the detection score, to reduce the audio artifact.

Abstract: Methods for generating an object based audio program, renderable in a personalizable manner, and including a bed of speaker channels renderable in the absence of selection of other program content (e.g., to provide a default full range audio experience). Other embodiments include steps of delivering, decoding, and/or rendering such a program. Rendering of content of the bed, or of a selected mix of other content of the program, may provide an immersive experience. The program may include multiple object channels (e.g., object channels indicative of user-selectable and user-configurable objects), the bed of speaker channels, and other speaker channels. Another aspect is an audio processing unit (e.g., encoder or decoder) configured to perform, or which includes a buffer memory which stores at least one frame (or other segment) of an object based audio program (or bitstream thereof) generated in accordance with, any embodiment of the method.

Abstract: Audio content coded for a reference speaker configuration is downmixed to downmix audio content coded for a specific speaker configuration. One or more gain adjustments are performed on individual portions of the downmix audio content coded for the specific speaker configuration. Loudness measurements are then performed on the individual portions of the downmix audio content. An audio signal that comprises the audio content coded for the reference speaker configuration and downmix loudness metadata is generated. The downmix loudness metadata is created based at least in part on the loudness measurements on the individual portions of the downmix audio content.

Abstract: Methods for generating an object based audio program which is renderable in a personalizable manner, e.g., to provide an immersive, perception of audio content of the program. Other embodiments include steps of delivering (e.g., broadcasting), decoding, and/or rendering such a program. Rendering of audio objects indicated by the program may provide an immersive experience. The audio content of the program may be indicative of multiple object channels (e.g., object channels indicative of user-selectable and user-configurable objects, and typically also a default set of objects which will be rendered in the absence of a selection by a user) and a bed of speaker channels. Another aspect is an audio processing unit (e.g., encoder or decoder) configured to perform, or which includes a buffer memory which stores at least one frame (or other segment) of an object based audio program (or bitstream thereof) generated in accordance with, any embodiment of the method.

Abstract: At a first time point, a first light capturing device at a first spatial location in a three-dimensional (3D) space captures first light rays from light sources located at designated spatial locations on a viewer device in the 3D space. At the first time point, a second light capturing device at a second spatial location in the 3D space captures second light rays from the light sources located at the designated spatial locations on the viewer device in the 3D space. Based on the first light rays captured by the first light capturing device and the second light rays captured by the second light capturing device, at least one of a spatial position and a spatial direction, at the first time point, of the viewer device is determined.

Abstract: Audio content coded for a reference speaker configuration is downmixed to downmix audio content coded for a specific speaker configuration. One or more gain adjustments are performed on individual portions of the downmix audio content coded for the specific speaker configuration. Loudness measurements are then performed on the individual portions of the downmix audio content. An audio signal that comprises the audio content coded for the reference speaker configuration and downmix loudness metadata is generated. The downmix loudness metadata is created based at least in part on the loudness measurements on the individual portions of the downmix audio content.

Abstract: Methods for generating an object based audio program, renderable in a personalizable manner, and including a bed of speaker channels renderable in the absence of selection of other program content (e.g., to provide a default full range audio experience). Other embodiments include steps of delivering, decoding, and/or rendering such a program. Rendering of content of the bed, or of a selected mix of other content of the program, may provide an immersive experience. The program may include multiple object channels (e.g., object channels indicative of user-selectable and user-configurable objects), the bed of speaker channels, and other speaker channels. Another aspect is an audio processing unit (e.g., encoder or decoder) configured to perform, or which includes a buffer memory which stores at least one frame (or other segment) of an object based audio program (or bitstream thereof) generated in accordance with, any embodiment of the method.