Abstract: A method includes receiving a current picture, a first reference picture, and a second reference picture. The method includes obtaining a plurality of predefined weighting patterns, each weighting pattern being signaled as an index value and selecting a weighting pattern based on a predetermined condition. The method includes deriving a first weight to be applied to a first sub-block in the first reference picture and a second weight to be applied to a second sub-block in the second reference picture based on the index value corresponding to the selected weighting pattern. The method includes assigning the first weight to the first sub-block and the second weight to the second sub-block based on the selected weighting pattern. The method includes decoding the current block by a weighted bi-prediction based at least on the first sub-block weighted by the first weight and the second sub-block weighted by the second weight.

Abstract: Aspects of the disclosure provide a method and an apparatus for video coding. In some examples, the apparatus includes processing circuitry for video encoding. The processing circuitry selects a resolution from a set of resolutions that includes a 1-integer-pel resolution and a 4-integer-pel resolution. A current block is encoded with an intra block copy mode based on the selected resolution. The processing circuitry determines a block vector of the current block. The processing circuitry determines a block vector difference of the current block based on the block vector and a block vector predictor of the current block. The block vector difference is in the selected resolution. The processing circuitry encodes prediction information indicating the selected resolution and the block vector difference.

Abstract: A video bitstream including coded information of a current block in a current picture is received. The current block is divided into a plurality of subblocks based on a region division shape of a plurality of region division shapes that is associated with an intra prediction mode of the current block. A weight set is determined for each of the plurality of subblocks, where the weight set includes a first weight for an intra prediction of the respective subblock and a second weight for an inter prediction of the respective subblock. Each of the plurality of subblocks is reconstructed based on the weight set, the intra prediction, and the inter prediction that are associated with the respective subblock.

Abstract: Neural network based substitutional end-to-end (E2E) image compression (NIC) being performed by at least one processor and includes receiving an input image to an E2E NIC framework, determining a step size of the input image indicating a learning rate of a training model, determining a substitute image based on the training model, encoding the substitute image in lieu of the input image to generate a bitstream, and mapping the substitute image to the bitstream to generate a compressed representation. Further, step size may be determined by a scheduler and change throughout the training of the training model. The image may also be split into patches for which a scheduler is assigned for each patch and each patch is encoded instead of the entire input image.

Abstract: Systems and methods for encoding and decoding neural network data is provided. A method includes: obtaining an independent neural network with a topology; encoding the independent neural network with the topology such as to obtain a neural network representation (NNR) bitstream; and sending the NNR bitstream to a decoder, wherein the NNR bitstream includes a group of NNR units (GON) that represents the independent neural network with the topology, and the GON includes an NNR model parameter set unit, an NNR layer parameter set unit, an NNR topology unit, an NNR quantization unit, and an NNR compressed data unit.

Abstract: A method of video processing in a decoder includes determining, by a processor, an initial block vector for predicting a current block in a current coding tree unit (CTU) in response to the current block being predicted in an intra block copy (IBC) mode. The initial block vector is determined based on a merge index included in a coded video bitstream. The method further includes performing, by the processor, template matching based on the initial block vector indicated by the merge index in the video bitstream to determine a refined block vector that points to a reference block in a picture as the current block, and reconstructing, by the processor, the current block based on the reference block.

Abstract: Aspects of the disclosure provide methods and apparatuses for video encoding/decoding. An apparatus for video decoding includes processing circuitry. The processing circuitry checks an inferable condition for a flag of a specific prediction mode for a current block before parsing the flag of the specific prediction mode for the current block from a coded video bitstream. The specific prediction mode is one of a plurality of inter picture prediction modes. When the inferable condition indicates that the flag is inferable, the processing circuitry infers the flag without parsing the flag from the coded video bitstream. When the inferable condition indicates uncertainty for inferring the flag, the processing circuitry parses the flag from the coded video bitstream. Then, the processing circuitry reconstructs the current block according to the specific prediction mode when the flag is indicative of an application of the specific prediction mode on the current block.

Abstract: A method, computer program, and computer system is provided for an all-deep-learning based AEC system by recurrent neural networks. The model consists of two stages, echo estimation stage and echo suppression stage, respectively. Two different schemes for echo estimation are presented herein: linear echo estimation by multi-tap filtering on far-end reference signal and non-linear echo estimation by single-tap masking on microphone signal. A microphone signal waveform and a far-end reference signal waveform are received. An echo signal waveform is estimated based on the microphone signal waveform and a far-end reference signal waveform. A near-end speech signal waveform is output based on subtracting the estimated echo signal waveform from the microphone signal waveform, and echoes are suppressed within the near-end speech signal waveform.

Abstract: A method, computer program, and computer system is provided for identifying a speaker in at text based work. Labeled and unlabeled instances corresponding to one or more speakers are extracted. Pseudo-labels are inferred for the extracted unlabeled instances based on the labeled instances. One or more of the unlabeled instances are labeled based on the inferred pseudo-labels.

Abstract: A pruning method of neural network based video coding of a current block of a picture of a video sequence is performed by at least one processor and includes categorizing parameters of a neural network into groups, setting a first index to indicate that a first group of the groups is to be pruned, and a second index to indicate that a second group of the groups is not to be pruned, and transmitting, to a decoder, the set first index and the set second index. Based on the transmitted first index and the transmitted second index, the current block is processed using the parameters of which the first group of the groups is pruned.

Abstract: A method, computer program, and computer system is provided for dynamic Network-Based Media Processing (NBMP) image retrieval. A call for a function from among a function group is received. The function call corresponds to an NBMP request to a workflow manager. A determination is made as to whether an image associated with the received function call is static or dynamic. A pointer to the image is returned based on the image being determined to be dynamic.

Abstract: Aspects of the disclosure provide methods and an apparatus including processing circuitry configured to receive workflow information of a workflow. The processing circuitry generates, based on the workflow information, the workflow to process input data. The workflow includes a first processing task, a second processing task, and a first buffering task. The first processing task is caused to enter a running state where a subset of the input data is processed and output to the first buffering task as first processed subset data. The first processing task is caused to transition to a paused state based on an amount of the first processed subset data in the first buffering task being equal to a first threshold. State information of the first processing task is stored in the paused state. Subsequently, the second processing task is caused to enter a running state where the first processed subset data is processed.

Abstract: Analyzing the complexity of an object of a scene in a media steam (or media data) performed by at least one processor, is provided, including receiving immersive media data comprising a plurality of scenes from a content source; obtaining a respective object of a respective scene in the plurality of scenes, from the immersive media data; analyzing the respective scene to generate complexity information associated with the respective object of the respective scene; generating metadata associated with the respective object of the respective scene, the metadata comprising the complexity information; and determining whether to distribute the respective scene to a client for processing based on the generated metadata.

Abstract: A method of unification based coding for neural network model compression is performed by at least one processor and includes receiving a layer uniform flag indicating whether a quantized weight of an input neural network is encoded using a uniform coding method, and determining whether the quantized weight is encoded using the uniform coding method, based on the received layer uniform flag. The method further includes, based on the quantized weight being determined to be encoded using the uniform coding method, encoding the quantized weight, using the uniform coding method, and based on the quantized weight being determined to not be encoded using the uniform coding method, encoding the quantized weight, using a non-uniform coding method.

Abstract: Methods, apparatus, and computer readable storage medium for intra bi-prediction and multiple reference line intra prediction in video decoding. The method includes receiving, by a device, a coded video bitstream for a block. The method also includes determining, by the device, whether a single directional intra prediction or an intra bi-prediction applies to the block, based on mode information of the block, the mode information of the block comprising at least one of: a reference line index of the block, an intra prediction mode of the block, and a size of the block; in response to determining that the single directional intra prediction applies to the block, performing, by the device, the single directional intra prediction to the block; and in response to determining that the intra bi-prediction applies to the block, performing, by the device, the intra bi-prediction to the block.

Abstract: A method, device, and computer-readable medium for decoding an encoded video bitstream using at least one processor, including obtaining a first flag indicating that a conformance window is present in a current picture; based on the first flag indicating that the conformance window is present, obtaining a second flag indicating whether the conformance window is used for reference picture resampling; based on the second flag indicating that the conformance window is used for the reference picture resampling, determining a resampling ratio between the current picture and a reference picture based on a conformance window size of the conformance window; based on the second flag indicating that the conformance window is not used for the reference picture resampling, determining the resampling ratio based on a resampling picture size; and performing the reference picture resampling on the current picture using the resampling ratio.

Abstract: A method and apparatus comprising computer code configured to cause a processor or processors to receive a text comprising a plurality of sentences, by a machine learning model, extract a nonverbal message from one of the sentences and add an annotation to the text, the annotation indicating the nonverbal message, and output a version of the text including the annotation.

Abstract: In a method for indicating build element availability, build element availability status information to a first user is determined to be displayed based on a command to trigger the display of the build element availability status information. In response to the determination to display the build element availability status information, the build element availability status information is displayed to the first user in real time. The build element availability status information indicates which of a second set of build elements is currently available to be built in a virtual scene by a second user.

Abstract: A method and apparatus comprising computer code configured to cause a processor or processors to receive an audio signal obtained from a microphone, input the audio signal into a neural-network pipeline, the neural-network pipeline including a convolutional network that receives the audio signal and provides a first output of the convolutional network to an enhancer, the enhancer including a deep complex convolutional recurrent network that receives the first output along with a mel spectrogram of the audio signal and outputs a second output to at least one of a vocoder and a decoder, and control an output of an enhanced audio signal from the at least one of the vocoder and the decoder.

Abstract: A method performed by at least one processor of an acoustic howling suppression (AHS) system includes receiving, from an input source device, an audio signal. The method includes refining one or more parameters of a Kalman filter based on one or more neural networks. The method includes filtering the audio signal using the Kalman filter with the one or more refined parameters of the Kalman filter to reduce acoustic howling included in the audio signal.