Abstract: A method for optimizing multi-frame processing model of a neural network includes: receiving a plurality of input frames by a processing engine that is configured to execute a multi frame processing model, the multi frame processing model including a plurality of convolution layers; selecting a pre-determined number of frames from the received plurality of frames for processing by the plurality of convolution layers; determining, as a sequence of frames, at least a preceding frame and a plurality of following frames amongst the selected pre-determined number of frames; removing the preceding frame by processing the sequence of frames using a plurality of filters in the multi frame processing model; and concatenating the plurality of following frames in an order, to the plurality of input frames for subsequent receiving by the multi frame processing model.

Abstract: An example method for training AI models for in-loop filters includes generating a training dataset by passing a video through a codec pipeline, extracting one or more predefined block features from the training dataset, creating a plurality of clusters based on the extracted one or more predefined block features from the training dataset, dividing the plurality of clusters into a sub-plurality of clusters based on the extracted one or more predefined block features and an intra-cluster variation threshold, and supplying the sub-plurality of clusters separately into a plurality of AI models based on the extracted one or more predefined block features.

Abstract: An electronic device for determining global attention in a deep learning model is provided. The electronic device includes a hardware accelerator, a low-complex global attention generator, a parallel switch, and a series switch. The hardware accelerator is configured to process each tile of a full-frame image and the low complex global attention generator is configured to generate a channel attention map of the full-frame image. The parallel switch is configured to bypass a connection of the channel attention map with the hardware accelerator and a series switch, configured to gate the connection of the channel attention map with the hardware accelerator.

Abstract: A method and an electronic device for low-complexity in-loop filter inference using feature-augmented training are provided. The method includes combining spatial and spectral domain features, using spectral domain features for global feature extraction and signalling to the spatial stream during training, using a detachable spectral domain stream for differential complexity during training versus inference, and combining a unique set of losses resulting from multi-stream and multi-feature approaches to obtain an optimal output.

Abstract: A method for generating metadata pertaining to a RAW frame includes selecting an input frame from a captured RAW frame, a plurality of frames obtained by processing the captured RAW frame, and a scaled RAW frame, selecting identified salient regions in an output frame, constructed from the captured RAW frame, based on errors between regions of the input frame and a corresponding reconstruction of the region of the input frame from the identified salient regions in the output frame, obtaining a plurality of reconstructed frames, reconstructed from a plurality of blocks of each salient region, corresponding to a plurality of regions of the input frame, and generating metadata for reconstructing the captured RAW frame by encoding a plurality of errors between the plurality of reconstructed frames and corresponding plurality of regions of the input frame, and a reconstruction technique used for reconstructing the plurality of reconstructed frames.

Abstract: The present disclosure disclose method and apparatus for determining memory requirement for processing a DNN model on a device, a method includes receiving a DNN model for an input, wherein the DNN model includes a plurality of processing layers. The method includes generating a network graph of the DNN model. The method includes creating a colored network graph of the DNN model based on the identified execution order of the plurality of processing layers. The colored network graph indicates assignment of at least one memory buffer for storing at least one output of at least one processing layer. The method includes determining at least one buffer reuse overlap possibility across the plurality of processing layers. Based on the determined at least one buffer reuse overlap possibility, the method includes determining and assigning the memory required for processing the DNN model.

Abstract: The disclosure relates to method and system for on-device inference in a deep neural network (DNN). The method comprises: determining whether one or more layers of the DNN satisfy one of a first, a second and a third condition, the one or more layers including one or more convolution layers and one or more resampling layers; performing the on-device inference based on the determination, wherein performing the on-device inference comprises at least one of: optimizing the one or more convolution layers in the one or more parallel branches based on the one or more layers of the DNN satisfying the first condition, optimizing the at least one of the resampling layers based on the one or more layers of the DNN satisfying the second condition, and modifying operation of the at least one of the resampling layers based on the one or more layers of the DNN satisfying the third condition.

Abstract: A method and an electronic device for performing an AI based zoom of an image in an electronic device are provided. The method includes receiving the image; obtaining, through a pixel domain neural network (NN) block, a first set of feature maps of the image based on pixels of the image; obtaining, through a frequency domain NN block, a second set of feature maps of the image based on frequencies of the image; and obtaining, through a joint refinement NN block, a final image with a resolution higher than a resolution of the image, based on the first set of feature maps and the second set of feature maps.

Abstract: The present disclosure refers to methods and apparatuses for processing of high-resolution video content. In an embodiment, a method includes generating a first group of video frames from the video content. The first group of video frames has a first resolution lower than a resolution of the video content and a first rate-distortion score. The method further includes generating a second group of video frames from the video content. The second group of video frames has a second resolution lower than the resolution of the video content and a second rate-distortion score. The method further includes selecting an optimal group of video frames from the first and second groups of video frames based on a comparison between the first and second rate-distortion scores. The optimal group of video frames has a rate-distortion score lower than the first and the second rate-distortion scores.

Abstract: Provided are a method and an apparatus for determining an optimal rotation angle during compression of a spherical multimedia content where the processor may receive first multimedia content corresponding to the spherical multimedia content, generate a plurality of second multimedia contents based on the first multimedia content by rotating the first multimedia content to a plurality of rotation angles, perform an edge analysis on each of the plurality of the second multimedia contents, identify the number of edges aligned to one of a vertical axis or a horizontal axis based on the edge analysis, select second multimedia content with the maximum number of edges, and determine an optimal rotation angle.

Abstract: Example embodiments include a method and an electronic device for detecting and removing artifacts/degradations in media. Embodiments may detect artifacts and/or degradations in the media based on tag information indicating at least one artifact included in the media. The detection may be triggered automatically or manually. Embodiments may generate artifact/quality tag information associated with the media to indicate artifacts and/or degradations present in the media, and may store the artifact/quality tag information as metadata and/or in a database. Embodiments may identify, based on the artifact/quality tag information associated with the media, at least one artificial intelligence (AI)-based media processing model to be applied to the media to enhance the media. The at least one AI-based media processing model may be configured to enhance at least one artifact detected in the media. Embodiments may enhance the media by applying the at least one AI-based media enhancement model to the media.

Abstract: Provided are methods and apparatus for determining an adjustment parameter during encoding of a spherical multimedia content which comprises finding the region of maximum concentrated energy is concentrated in the generated energy map of the spherical multimedia content, measuring the width of the maximum concentrated energy region in the generated energy map, and deriving optimal adjustment parameter from the width of the maximum concentrated energy region in the generated energy map.

Abstract: Accordingly embodiments herein disclose a method for producing 360 degree image content on a rectangular projection. The method includes detecting whether at least one discontinuous boundary is present in the 360-degree image content, wherein the at least one discontinuous boundary is detected using the packing of one or more projection segments. Further, the method includes obtaining one or more reconstructed blocks from the 360-degree image content. The reconstructed blocks are generated from the 360 degree image content. Further, the method includes performing one of enabling at least one in-loop filtering function on the one or more reconstructed blocks of the 360 degree image content when the at least one of image content discontinuous boundary is not present and disabling at least one in-loop filtering function on the one or more reconstructed blocks of the 360 degree image content located around the at least one discontinuous boundary.

Abstract: Accordingly embodiments herein disclose a method for producing 360 degree image content on a rectangular projection. The method includes detecting whether at least one discontinuous boundary is present in the 360-degree image content, wherein the at least one discontinuous boundary is detected using the packing of one or more projection segments. Further, the method includes obtaining one or more reconstructed blocks from the 360-degree image content. The reconstructed blocks are generated from the 360 degree image content. Further, the method includes performing one of enabling at least one in-loop filtering function on the one or more reconstructed blocks of the 360 degree image content when the at least one of image content discontinuous boundary is not present and disabling at least one in-loop filtering function on the one or more reconstructed blocks of the 360 degree image content located around the at least one discontinuous boundary.

Abstract: Provided are methods and apparatus for determining an adjustment parameter during encoding of a spherical multimedia content which comprises finding the region of maximum concentrated energy is concentrated in the generated energy map of the spherical multimedia content, measuring the width of the maximum concentrated energy region in the generated energy map, and deriving optimal adjustment parameter from the width of the maximum concentrated energy region in the generated energy map.

Abstract: Provided are a method and an apparatus for determining an optimal rotation angle during compression of a spherical multimedia content where the processor may receive first multimedia content corresponding to the spherical multimedia content, generate a plurality of second multimedia contents based on the first multimedia content by rotating the first multimedia content to a plurality of rotation angles, perform an edge analysis on each of the plurality of the second multimedia contents, identify the number of edges aligned to one of a vertical axis or a horizontal axis based on the edge analysis, select second multimedia content with the maximum number of edges, and determine an optimal rotation angle.

Abstract: Embodiments herein disclose a method for producing 360 degree image content on a rectangular projection in an electronic device. The method includes obtaining a 360 degree image content represented by packing one or more projection segments arranged in a rectangular projection. The method includes detecting whether at least one discontinuous boundary is present in the 360-degree image content. The at least one discontinuous boundary is detected using the packing of one or more projection segments. The method includes applying at least one padding information on the at least one discontinuous boundary. The method includes producing another 360 degree image content on the rectangular projection in the electronic device based on the padding information.

Abstract: The present disclosure disclose method and apparatus for determining memory requirement for processing a DNN model on a device, a method includes receiving a DNN model for an input, wherein the DNN model includes a plurality of processing layers. The method includes generating a network graph of the DNN model. The method includes creating a colored network graph of the DNN model based on the identified execution order of the plurality of processing layers. The colored network graph indicates assignment of at least one memory buffer for storing at least one output of at least one processing layer. The method includes determining at least one buffer reuse overlap possibility across the plurality of processing layers. Based on the determined at least one buffer reuse overlap possibility, the method includes determining and assigning the memory required for processing the DNN model.

Abstract: A method and apparatus for processing audio data are provided. When an encoded audio bitstream sampled at a sampling frequency is received, a resampling ratio for processing the encoded audio bitstream is computed. If the the resampling ratio is within the resampling threshold range, then the encoded audio bitstream is processed in frequency domain and a desired number of audio samples per frame are outputted according to the resampling ratio. The encoded audio bitstream is processed in frequency domain using sample rate converter integrated into a filter bank of an audio decoder. If the resampling ratio is outside the resampling threshold range, then the encoded audio bitstream is processed in time domain and a desired number of audio samples per frame are outputted according to the resampling ratio.

Abstract: A method and apparatus for processing audio data are provided. When an encoded audio bitstream sampled at a sampling frequency is received, a resampling ratio for processing the encoded audio bitstream is computed. If the the resampling ratio is within the resampling threshold range, then the encoded audio bitstream is processed in frequency domain and a desired number of audio samples per frame are outputted according to the resampling ratio. The encoded audio bitstream is processed in frequency domain using sample rate converter integrated into a filter bank of an audio decoder. If the resampling ratio is outside the resampling threshold range, then the encoded audio bitstream is processed in time domain and a desired number of audio samples per frame are outputted according to the resampling ratio.