Abstract: A multi-view input image covering multiple sampled views is received. A multi-view layered image stack is generated from the multi-view input image. A target view of a viewer to an image space depicted by the multi-view input image is determined based on user pose data. The target view is used to select user pose selected sampled views from among the multiple sampled views. Layered images for the user pose selected sampled views, along with alpha maps and beta scale maps for the user pose selected sampled views are encoded into a video signal to cause a recipient device of the video signal to generate a display image for rendering on the image display.

Abstract: Methods, systems, and media for determining sound field rotations are provided. In some embodiments, a method for determining sound field rotations involves determining an activity situation of a user. The method may involve determining a user head orientation using at least one sensor of the one or more sensors. The method may involve determining a direction of interest based on the activity situation and the user head orientation. The method may involve determining a rotation of a sound field used to present audio objects via headphones based on the direction of interest.

Abstract: A video delivery system for luminance adjustment based upon a viewer adaptation state comprises a processor configured to: receive a source image including a current image frame including metadata corresponding to a mean luminance value of the current image frame, and the source image including an upcoming image frame including metadata corresponding to a mean luminance value of the upcoming image frame. The processor is configured to determine an ambient luminance value based on an ambient luminance, determine an incident luminance value based on the ambient luminance value and the mean luminance value, determine a difference between a current pupil size and a target pupil size, and generate an output image by modifying the source image based on a luminance adjustment factor, the luminance adjustment factor being a function of the difference between the current pupil size and the target pupil size.

Abstract: Systems and methods for an entropy coding system are described. The entropy coding systems include an encoding apparatus and a decoding apparatus. The encoding apparatus is configured to receive an original input stream comprising a plurality of symbols having a known entropy characteristic according to a probability distribution of each of the symbols appearing in the original input stream, determine an input and respective state for each symbol read from the original input stream, append the determined input to the encoded output stream, and provided the encoded output stream to the decoding apparatus. The decoding apparatus is configured to receive the encoded output stream, process the encoded output stream, and for each read input: determine an output symbol and a respective output, persist the respective output state to the encoded output stream, and append the determined output symbol to the results output stream.

Abstract: Diffuse or spatially large audio objects may be identified for special processing. A decorrelation process may be performed on audio signals corresponding to the large audio objects to produce decorrelated large audio object audio signals. These decorrelated large audio object audio signals may be associated with object locations, which may be stationary or time-varying locations. For example, the decorrelated large audio object audio signals may be rendered to virtual or actual speaker locations. The output of such a rendering process may be input to a scene simplification process. The decorrelation, associating and/or scene simplification processes may be performed prior to a process of encoding the audio data.

Abstract: The present document describes a method (400) for rendering an ambisonics signal using a loudspeaker arrangement comprising S loudspeakers. The method (400) comprises converting (401) a set of N ambisonics channel signals (111) into a set of unfiltered pre-rendered signals (211), with N>1 and S>1. Furthermore, the method (400) comprises performing (402) near field compensation, referred to as NFC, filtering of M unfiltered pre-rendered signals (211) of the set of unfiltered pre-rendered signals (211) to provide a set of S filtered loudspeaker channel signals (114) for rendering using the corresponding S loudspeakers.

Abstract: The present disclosure relates to a method and system for predicting a future orientation of an orientation tracker (100). The method comprising obtaining a sequence of angular velocity samples, each angular velocity sample indicating an angular velocity at a point in time and obtaining a sequence of angular acceleration samples, each angular acceleration sample indicating an acceleration or deceleration of the angular velocity at each point in time. Wherein said method further comprises determining (S5a), for each point in time where the angular velocity is accelerating, a predicted orientation of the orientation tracker (100) based on a first order prediction of an accumulated rotation of the orientation tracker (100) and determining (S5c), for each point in time where the angular velocity is decelerating, a predicted orientation of the orientation tracker (100) based on a second order prediction of the accumulated rotation of the orientation tracker (100).

Abstract: A multi-input, multi-output audio process is implemented as a linear system for use in an audio filterbank to convert a set of frequency-domain input audio signals into a set of frequency-domain output signals. A transfer function from one input to one output is defined as a frequency dependent gain function. In some implementations, the transfer function includes a direct component that is substantially defined as a frequency dependent gain, and one or more decorrelated components that have frequency-varying group phase response. The transfer function is formed from a set of sub-band functions, with each sub-band function being formed from a set of corresponding component transfer functions including direct component and one or more decorrelated components.

Abstract: A system for detecting speech from reverberant signals is disclosed. The system is programmed to receive spectral temporal amplitude data in the modulation frequency domain. The system is programmed to then enhance the spectral temporal amplitude data by reducing reverberation and other noise as well as smoothing based on certain properties of the spectral temporal spectrogram associated with the spectral temporal amplitude data. Next, the system is programmed to compute various features related to the presence of speech based on the enhanced spectral temporal amplitude data and other data in the modulation frequency domain or in the (acoustic) frequency domain. The system is programmed to then determine an extent of speech present in the audio data corresponding to the received spectral temporal amplitude data based on the various features. The system can be programmed to transmit the extent of speech present to an output device.

Abstract: Tensor-Product B-splines (TPB) have been shown to improve video quality when used to represent reshaping functions to map reshaped standard dynamic range content into high dynamic range (HDR) content; however, TPB prediction is computationally intensive and may not be supported by legacy devices. Methods and systems for backwards-compatible signaling of TPB-related metadata and a fast TPB prediction method are presented to overcome both of these limitations. Computation overhead for a TPB-based 3D look-up table is reduced by using temporary two-dimensional arrays. A remapping of the most significant bits of a legacy bit-depth parameter allows for backwards compatibility.

Abstract: A system for managing user-generated content (UGC) and professionally generated content (PGC) is disclosed. The system is programmed to receive digital audio data having two channels from a social media platform. The system is programmed to extract spatial features that capture differences in the two channels from the digital audio data. The system is programmed to also extract temporal features, spectral features, and background features from the digital audio data. The system is programmed to then use the extracted features to determine whether to process the digital audio data as UGC or PGC before playback.

Abstract: A sequence of base layer images for a base reference image display and a set of one or more sequences of beta scale maps corresponding to one or more non-base reference image displays are generated. A subset of one or more sequences of beta scale maps is determined in the set of one or more sequences of beta scale maps based at least in part on display capabilities of a target image display. The sequence of base layer images, along with the subset of one or more sequences of beta scale maps, is encoded into a video signal to cause a recipient device of the video signal to generate a sequence of display images from the sequence of base layer images and the subset of one or more sequences of beta scale maps for rendering on the image display.

Abstract: Soundfield signals such as e.g. Ambisonics carry a representation of a desired sound field. The Ambisonics format is based on spherical harmonic decomposition of the soundfield, and Higher Order Ambisonics (HOA) uses spherical harmonics of at least 2nd order. However, commonly used loudspeaker setups are irregular and lead to problems in decoder design. A method for improved decoding an audio soundfield representation for audio playback comprises calculating a panning function (W) using a geometrical method based on the positions of a plurality of loudspeakers and a plurality of source directions, calculating a mode matrix (?) from the loudspeaker positions, calculating a pseudo-inverse mode matrix (?+) and decoding the audio soundfield representation. The decoding is based on a decode matrix (D) that is obtained from the panning function (W) and the pseudo-inverse mode matrix (?+).

Abstract: In some embodiments, a method for processing an audio signal in an audio processing apparatus is disclosed. The method includes receiving an audio signal and a parameter, the parameter indicating a location of an auditory event boundary. An audio portion between consecutive auditory event boundaries constitutes an auditory event. The method further includes applying a modification to the audio signal based in part on an occurrence of the auditory event. The parameter may be generated by monitoring a characteristic of the audio signal and identifying a change in the characteristic.

Abstract: Methods and systems for the super resolution of high dynamic range (HDR) video are described. Given a sequence of video frames, a current frame and two or more neighboring frames are processed by a neural-network (NN) feature extraction module, followed by a NN upscaling module, and a NN reconstruction module. In parallel, the current frame is upscaled using traditional up-sampling to generate an intermediate up-sampled frame. The output of the reconstruction module is added to the intermediate up-sampled frame to generate an output frame. Additional traditional up-sampling may be performed on the output frame to match the desired up-scaling factor, beyond the up-scaling factor for which the neural network was trained.

Abstract: Embodiments are disclosed for bitrate distribution in immersive voice and audio services. In an embodiment, a method of encoding an IVAS bitstream comprises: receiving an input audio signal; downmixing the input audio signal into one or more downmix channels and spatial metadata; reading a set of one or more bitrates for the downmix channels and a set of quantization levels for the spatial metadata from a bitrate distribution control table; determining a combination of the one or more bitrates for the downmix channels; determining a metadata quantization level from the set of metadata quantization levels using a bitrate distribution process; quantizing and coding the spatial metadata using the metadata quantization level; generating, using the combination of one or more bitrates, a downmix bitstream for the one or more downmix channels; combining the downmix bitstream, the quantized and coded spatial metadata and the set of quantization levels into the IVAS bitstream.

Abstract: A metadata-aided film-grain removal method and corresponding apparatus. An example embodiment enables a video decoder to substantially fully remove the film grain from a digital video signal that has undergone lossy video compression and then video decompression. Different embodiments may rely only on spatial-domain grain-removal processing, only on temporal-domain grain-removal processing, or on a combination of spatial-domain and temporal-domain grain-removal processing. Both spatial-domain and temporal-domain grain-removal processing may use metadata provided by the corresponding video encoder, the metadata including one or more parameters corresponding to the digital film grain injected into the host video at the encoder. Different film-grain-injection formats can be accommodated by the video decoder using signal preprocessing directed at supplying, to the film-grain removal module of the video decoder, an input compatible with the film-grain removal method implemented therein.

Abstract: Audio content coded for a reference speaker configuration is downmixed to downmix audio content coded for a specific speaker configuration. One or more gain adjustments are performed on individual portions of the downmix audio content coded for the specific speaker configuration. Loudness measurements are then performed on the individual portions of the downmix audio content. An audio signal that comprises the audio content coded for the reference speaker configuration and downmix loudness metadata is generated. The downmix loudness metadata is created based at least in part on the loudness measurements on the individual portions of the downmix audio content.

Abstract: A method of audio processing includes generating a detection score based on the partial loudnesses of a reference audio signal, extracted audio objects, extracted bed channels, a rendered audio signal and a channel-based audio signal. The detection score is indicative of an audio artifact in one or more of the audio objects and the bed channels. The extracted audio objects and extracted bed channels may be modified, in accordance with the detection score, to reduce the audio artifact.