Abstract: Systems and methods for panoptic video segmentation are described. A method may include identifying a target frame and a reference frame from a video, generating target features for the target frame and reference features for the reference frame, combining the target features and the reference features to produce fused features for the target frame, generating a feature matrix comprising a correspondence between objects from the reference features and objects from the fused features; and generating panoptic segmentation information for the target frame based on the feature matrix.

Abstract: An electronic device and a method thereof are provided. The electronic device includes: a memory and a processor configured to: acquire feature data of a plurality of images in a video using a first artificial neural network of a first artificial intelligence model; acquire a plurality of key frames of the video based on the feature data of the plurality of images using a second artificial neural network of the first artificial intelligence model; acquire first feature data of remaining key frames excluding at least one of the plurality of key frames using a first artificial neural network of a second artificial intelligence model; acquire second feature data including information about relationships between the remaining key frames based on the first feature data using a second artificial neural network of the second artificial intelligence model; and acquire texts for the plurality of key frames based on the second feature data.

Abstract: A method, apparatus, non-transitory computer readable medium, and system for panoptic segmentation are described. Embodiments may generate a feature pyramid for an input image, wherein the feature pyramid comprises a plurality of feature maps at different resolution levels, apply an attention module to the feature pyramid to produce an enhanced feature map, combine the enhanced feature map with each of the plurality of feature maps to produce an enhanced feature pyramid, generate semantic information for the input image based on the enhanced feature pyramid, generate a plurality of object regions based on the enhanced feature pyramid, generate instance information for each of the plurality of object regions, and generate panoptic segmentation information for the input image based on the semantic information and the instance information for each of the plurality of object regions.

Abstract: Systems and methods for panoptic video segmentation are described. A method may include identifying a target frame and a reference frame from a video, generating target features for the target frame and reference features for the reference frame, combining the target features and the reference features to produce fused features for the target frame, generating a feature matrix comprising a correspondence between objects from the reference features and objects from the fused features; and generating panoptic segmentation information for the target frame based on the feature matrix.

Abstract: A method, apparatus, non-transitory computer readable medium, and system for panoptic segmentation are described. Embodiments may generate a feature pyramid for an input image, wherein the feature pyramid comprises a plurality of feature maps at different resolution levels, apply an attention module to the feature pyramid to produce an enhanced feature map, combine the enhanced feature map with each of the plurality of feature maps to produce an enhanced feature pyramid, generate semantic information for the input image based on the enhanced feature pyramid, generate a plurality of object regions based on the enhanced feature pyramid, generate instance information for each of the plurality of object regions, and generate panoptic segmentation information for the input image based on the semantic information and the instance information for each of the plurality of object regions.

Abstract: The present disclosure relates to an artificial intelligence (AI) system for simulating functions of the human brain, such as cognition and determination, by using a machine learning algorithm, such as deep learning, and to an application of the AI system.