Abstract: A first motion vector and a second motion vector are determined for a first block in a current picture of a video, where the first motion vector is indicative of a first reference block in a first picture, and the second motion vector is indicative of a second reference block in a second picture. A bilateral template is generated based on a weighted combination of the first reference block and the second reference block. A refined first motion vector is determined based on the bilateral template and a first set of reference blocks in the first picture. A refined second motion vector is determined based on the bilateral template and a second set of reference blocks in the second picture. Prediction information of the first block is generated according to (i) the refined first motion vector, (ii) the refined second motion vector, and (iii) a final motion compensation interpolation filter.

Abstract: Aspects of the disclosure include methods, apparatuses, and non-transitory computer-readable storage mediums for video encoding/decoding. An apparatus includes processing circuitry that decodes a video bitstream to obtain a reduced-resolution residual block for a current block. The processing circuitry determines that a block level flag is set to a pre-defined value. The pre-defined value indicates that the current block is coded in reduced-resolution coding. Based on the block level flag, the processing circuitry generates a reduced-resolution prediction block for the current block by down-sampling a full-resolution reference block of the current block. The processing circuitry generates a reduced-resolution reconstruction block for the current block based on the reduced-resolution prediction block and the reduced-resolution residual block. The processing circuitry generates a full-resolution reconstruction block for the current block by up-sampling the reduced-resolution reconstruction block.

Abstract: An apparatus for video coding is provided. The apparatus includes processing circuitry that buffers first boundary pixel values of first reconstructed samples at a first node along a loop filter chain. The first node is associated with a non linear mapping based filter that is applied in the loop filter chain before a loop restoration filter. The first boundary pixel values are values of pixels at a frame boundary. The processing circuitry applies the loop restoration filter on to-be filtered reconstructed samples based on the buffered first boundary pixel values.

Abstract: A method of point cloud geometry encoding includes receiving a slice of a point cloud frame for encoding, and constructing an octree representing a geometry of points in a bounding box of the slice where a current node of the octree is partitioned with a quadtree (QT) partition or a binary tree (BT) partition. The constructing includes determining a value of a partitionSkip variable specifying a partition type and a partition direction of the current node of the octree.

Abstract: A processing circuitry decodes a plurality of maps in 2D from a bitstream carrying a mesh frame. The mesh frame represents a surface of an object with polygons. The plurality of maps includes a decoded geometry map and a decoded attribute map with an adaptive 2D atlas sampling applied. The processing circuitry determines at least a first sampling rate and a second sampling rate according to syntaxes signaled in the bitstream. The first sampling rate is applied to a first region of the mesh frame and the second sampling rate is applied to a second region of the mesh frame during the adaptive 2D atlas sampling. The processing circuitry reconstructs, based on the plurality of maps, at least a first vertex of the mesh frame according to the first sampling rate, and a second vertex of the mesh frame according to the second sampling rate.

Abstract: A method of decoding or encoding including receiving information regarding a video sequence for encoding or decoding, determining, for the encoding or decoding of the video sequence, whether to use a first transform core matrix that is of a first size type or a second transform core matrix that is of a second size type, and based on the determining, transmitting information that causes the video sequence to be encoded or decoded using the determined first transform core matrix or second transform core matrix.

Abstract: A method performed by at least one processor includes retrieving a dialogue history including a plurality of speech utterances, each speech utterance including one or more words. The method further includes encoding the plurality of speech utterances such that each speech utterance is associated with a sequence identifier indicating an order of each speech utterance in the dialogue history. The method further includes decoding the encoded plurality of speech utterances to generate at least one discourse relation triple corresponding to the dialogue history, the at least one discourse relation triple including a first sequence identifier of a first speech utterance from the plurality of speech utterances, a second sequence identifier of a second speech utterance from the plurality of speech utterances, and a dialogue discourse type.

Abstract: Aspects of the disclosure include methods, apparatuses, and non-transitory computer-readable storage mediums for processing media streams. An apparatus includes processing circuitry that sends a message to a media aware network element that is configured to process a plurality of audio streams of a conference call. The message indicates that the plurality of audio streams is to be down mixed by the media aware network element. The processing circuitry receives the down mixed plurality of audio streams from the media aware network element and decodes the down mixed plurality of audio streams to receive the conference call.

Abstract: Coding information of a mesh is received. The coding information includes a plurality of first coordinates and a plurality of second coordinates corresponding to a plurality of vertices and a texture map that are associated with the mesh. A respective first coordinate and a respective second coordinate associated with each of the plurality of vertices are normalized by adjusting the respective first coordinate based on a first factor and the respective second coordinate based on a second factor. The first factor and the second factor are associated with at least one of (i) a bit depth value indicating a coded range of the first coordinates and the second coordinates and (ii) a size of the texture map. The normalized respective first coordinate and the normalized respective second coordinate are expanded based on the first factor and the second factor respectively.

Abstract: This disclosure relates generally to video coding and particularly to methods and systems for determination of temporal motion vector predictor (TMVP) candidates for inter-prediction in video coding. The disclosed methods, for example, include restricting the number of TMVP candidates in a motion vector predictor (MVP) list and provide various search mechanism in order to promote MVP candidate diversity among TMVP and other types of MVP candidates and to improve coding efficiency.

Abstract: A method of training a model for query generation, the method performed by at least one processor and including receiving a training instance query corresponding to a dialogue history. The method further including generating a first static view of the model based on a number of common words between the training instance query and the dialogue history. The method further including generating a second static view of the model based on one or more tokens not covered by the dialogue history, the one or more tokens corresponding to one or more query words. The method further including generating a dynamic view of the model based on a score operation that compares a candidate query generated from the model with a target query. The method further including training the model based at least on the first static view, the second static view, and the dynamic view.

Abstract: Aspects of the disclosure provide a method and an apparatus for video encoding. The apparatus includes processing circuitry configured to generate an initial feature representation from an input image to be encoded and perform an iterative update of values of a plurality of elements in the initial feature representation. The iterative update includes generate a coded representation corresponding to a final feature representation based on the final feature representation that has been updated from the initial feature representation by a number of iterations of the iterative update. A reconstructed image corresponding to the final feature representation is generated based on the coded representation. An encoded image corresponding to the final feature representation having updated values of the plurality of elements is generated. One of (i) a rate-distortion loss corresponding to the final feature representation or (ii) the number of iterations of the iterative update satisfies a pre-determined condition.

Abstract: Aspects of the disclosure provide methods and apparatuses for video encoding/decoding. Processing circuitry decodes prediction information of a block from a coded video bitstream. The prediction information is indicative of a matrix based intra prediction for the block. The processing circuitry determines entries of a vector based on neighboring samples of the block. An entry can be determined based on one or more neighboring samples of the block. The processing circuitry converts the entries into a reduced bit form with a number of bits satisfying a requirement of using a first multiplication tool that processes fewer bits than a second multiplication tool. Then, the processing circuitry multiplies, using the first multiplication tool, the entries of the vector in the reduced bit form with entries of a matrix to calculate a subset of prediction samples of the block, and determines other prediction samples of the block based on the subset.

Abstract: A method of video decoding in a decoder is provided. A first template magnitude of a first transform coefficient in a specific frequency region of the transform block is determined. The first template magnitude is a first single value representing magnitudes of a first local template of the first transform coefficient. A first context model is identified for coding the syntax element of the first transform coefficient, the first context model being shared with at least a second transform coefficient in the specific frequency region of the transform block, a second template magnitude of the second transform coefficient having a second single value that belongs to a first subinterval. A first bin of the syntax element of the first transform coefficient and a second bin of the syntax element of the second transform coefficient is determined, from the coded bits, based on the first context model.

Abstract: There is includes a method and apparatus comprising computer code configured to cause a hardware processor or processors to perform intra prediction among a plurality of reference lines, to set a plurality of intra prediction modes for a zero reference line nearest to a current block of the intra prediction among non-zero reference lines, and to set one or more most probable modes for one of the non-zero reference lines.

Abstract: End-to-end neural image compression using deep reinforcement learning (DRL) is performed by at least one processor and includes encoding an input, generating encoded representations of the input, generating a set of quantization keys using a first neural network, based on a set of previous quantization states, wherein each quantization key in the set of quantization keys and each previous quantization state in the set of previous quantization states correspond to the encoded representations of the input, generating a set of dequantized numbers representing dequantized representations of the encoded representations of the input, based on the set of quantization keys, using a second neural network, and generating a reconstructed output, based on the set of dequantized numbers.

Abstract: A method and a device for signaling multiple audio mixing gains in a teleconference using Real-time Transport Control Protocol (RTCP) feedback. The method includes receiving an input audio stream from a 360-degree video stream, the input audio stream including mixing gains, declaring an RTCP feedback rate for receiving the mixing gains, based on an allocated bandwidth, and signaling the mixing gains using the declared RTCP feedback rate. The mixing gains may include audio gains from the input audio stream and audio gains from overlay audio streams. The RTCP feedback rate used for signaling the mixing gains may be constant or event-based feedback rate.

Abstract: A method executed by at least one processor, the method comprising: segmenting a multidimensional media stream into a plurality of segments of multidimensional media in a multidimensional space; representing each segment of the plurality of segments of multidimensional media using a respective sequence vector, the respective sequence vector comprising one or more predefined multidimensional metadata, wherein the predefined multidimensional metadata includes one of a starting vector, a length vector, and a scaling vector, and a startcode; and deriving a network based media processing (NBMP) workflow based on the respective sequence vectors of each segment of the plurality of segments.

Abstract: A system and method of split rendering for lightfield or immersive media by using an edge-cloud and peer-to-peer based architecture. The system and method include the use of a combination of cloud-based devices and edge-devices to provide distributed processing in connection with the streaming of media, and in particular lightfield or immersive media, to an end user device. The system and method further include the use of multiple cloud and edge devices to provide parallel streaming of a given media package to an end user device.

Abstract: A method of adaptive neural image compression with rate control by meta-learning includes receiving an input image and a hyperparameter; and encoding the received input image, based on the received hyperparameter, using an encoding neural network, to generate a compressed representation. The encoding includes performing a first shared encoding on the received input image, using a first shared encoding layer having first shared encoding parameters, performing a first adaptive encoding on the received input image, using a first adaptive encoding layer having first adaptive encoding parameters, combining the first shared encoded input image and the first adaptive encoded input image, to generate a first combined output, and performing a second shared encoding on the first combined output, using a second shared encoding layer having second shared encoding parameters.