Abstract: A method, apparatus, and non-transitory computer readable medium for referring image segmentation are described. Embodiments of the method, apparatus, and non-transitory computer readable medium may extract an image feature vector from an input image, extract a plurality of language feature vectors for a referral expression, wherein each of the plurality of language feature vectors comprises a different number of dimensions, combine each of the language feature vectors with the image feature vector using a fusion module to produce a plurality of self-attention vectors, combine the plurality of self-attention vectors to produce a multi-modal feature vector, and decode the multi-modal feature vector to produce an image mask indicating a portion of the input image corresponding to the referral expression.

Abstract: Systems and methods for panoptic video segmentation are described. A method may include identifying a target frame and a reference frame from a video, generating target features for the target frame and reference features for the reference frame, combining the target features and the reference features to produce fused features for the target frame, generating a feature matrix comprising a correspondence between objects from the reference features and objects from the fused features; and generating panoptic segmentation information for the target frame based on the feature matrix.

Abstract: Systems and methods for video object segmentation are described. Embodiments of systems and methods may receive a referral expression and a video comprising a plurality of image frames, generate a first image mask based on the referral expression and a first image frame of the plurality of image frames, generate a second image mask based on the referral expression, the first image frame, the first image mask, and a second image frame of the plurality of image frames, and generate annotation information for the video including the first image mask overlaid on the first image frame and the second image mask overlaid on the second image frame.

Abstract: This disclosure describes one or more implementations of a panoptic segmentation system that generates panoptic segmented digital images that classify both known and unknown instances of digital images. For example, the panoptic segmentation system builds and utilizes a panoptic segmentation neural network to discover, cluster, and segment new unknown object subclasses for previously unknown object instances. In addition, the panoptic segmentation system can determine additional unknown object instances from additional digital images. Moreover, in some implementations, the panoptic segmentation system utilizes the newly generated unknown object subclasses to refine and tune the panoptic segmentation neural network to improve the detection of unknown object instances in input digital images.

Abstract: Systems and techniques are described that provide for generalizable approach policy learning and implementation for robotic object approaching. Described techniques provide fast and accurate approaching of a specified object, or type of object, in many different environments. The described techniques enable a robot to receive an identification of an object or type of object from a user, and then navigate to the desired object, without further control from the user. Moreover, the approach of the robot to the desired object is performed efficiently, e.g., with a minimum number of movements. Further, the approach techniques may be used even when the robot is placed in a new environment, such as when the same type of object must be approached in multiple settings.

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for automatic tagging of videos. In particular, in one or more embodiments, the disclosed systems generate a set of tagged feature vectors (e.g., tagged feature vectors based on action-rich digital videos) to utilize to generate tags for an input digital video. For instance, the disclosed systems can extract a set of frames for the input digital video and generate feature vectors from the set of frames. In some embodiments, the disclosed systems generate aggregated feature vectors from the feature vectors. Furthermore, the disclosed systems can utilize the feature vectors (or aggregated feature vectors) to identify similar tagged feature vectors from the set of tagged feature vectors. Additionally, the disclosed systems can generate a set of tags for the input digital videos by aggregating one or more tags corresponding to identified similar tagged feature vectors.

Abstract: A method, apparatus, and non-transitory computer readable medium for referring image segmentation are described. Embodiments of the method, apparatus, and non-transitory computer readable medium may extract an image feature vector from an input image, extract a plurality of language feature vectors for a referral expression, wherein each of the plurality of language feature vectors comprises a different number of dimensions, combine each of the language feature vectors with the image feature vector using a fusion module to produce a plurality of self-attention vectors, combine the plurality of self-attention vectors to produce a multi-modal feature vector, and decode the multi-modal feature vector to produce an image mask indicating a portion of the input image corresponding to the referral expression.

Abstract: Certain aspects involve using a space-time memory network to locate one or more target objects in video content for segmentation or other object classification. In one example, a video editor generates a query key map and a query value map by applying a space-time memory network to features of a query frame from video content. The video editor retrieves a memory key map and a memory value map that are computed, with the space-time memory network, from a set of memory frames from the video content. The video editor computes memory weights by applying a similarity function to the memory key map and the query key map. The video editor classifies content in the query frame as depicting the target feature using a weighted summation that includes the memory weights applied to memory locations in the memory value map.

Abstract: Systems and methods for video object segmentation are described. Embodiments of systems and methods may receive a referral expression and a video comprising a plurality of image frames, generate a first image mask based on the referral expression and a first image frame of the plurality of image frames, generate a second image mask based on the referral expression, the first image frame, the first image mask, and a second image frame of the plurality of image frames, and generate annotation information for the video including the first image mask overlaid on the first image frame and the second image mask overlaid on the second image frame.

Abstract: Various embodiments describe video object segmentation using a neural network and the training of the neural network. The neural network both detects a target object in the current frame based on a reference frame and a reference mask that define the target object and propagates the segmentation mask of the target object for a previous frame to the current frame to generate a segmentation mask for the current frame. In some embodiments, the neural network is pre-trained using synthetically generated static training images and is then fine-tuned using training videos.

Abstract: A portable terminal apparatus includes a bended display including a front display area and a side display area, and a controller that displays a UI element corresponding to an application on a side display area of the bended display if the cover is closed, and in response to a touch with respect to at least one UI element from among the UI elements and an opening of the cover being detected, executes an application corresponding to a UI element with the detected touch, and controls the bended display so that an execution screen of the application is displayed on the front display area.

Abstract: Systems and methods for panoptic video segmentation are described. A method may include identifying a target frame and a reference frame from a video, generating target features for the target frame and reference features for the reference frame, combining the target features and the reference features to produce fused features for the target frame, generating a feature matrix comprising a correspondence between objects from the reference features and objects from the fused features; and generating panoptic segmentation information for the target frame based on the feature matrix.

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for automatic tagging of videos. In particular, in one or more embodiments, the disclosed systems generate a set of tagged feature vectors (e.g., tagged feature vectors based on action-rich digital videos) to utilize to generate tags for an input digital video. For instance, the disclosed systems can extract a set of frames for the input digital video and generate feature vectors from the set of frames. In some embodiments, the disclosed systems generate aggregated feature vectors from the feature vectors. Furthermore, the disclosed systems can utilize the feature vectors (or aggregated feature vectors) to identify similar tagged feature vectors from the set of tagged feature vectors. Additionally, the disclosed systems can generate a set of tags for the input digital videos by aggregating one or more tags corresponding to identified similar tagged feature vectors.

Abstract: Embodiments herein describe a framework for classifying images. In some embodiments, it is determined whether an image includes synthetic image content. If it does, characteristics of the image are analyzed to determine if the image includes characteristics particular to panoramic images (e.g., possess a threshold equivalency of pixel values among the top and/or bottom boundaries of the image, or a difference between summed pixel values of the pixels comprising the right vertical boundary of the image and summed pixel values of the pixels comprising the left vertical boundary of the image being less than or equal to a threshold value). If the image includes characteristics particular to panoramic images, the image is classified as a synthetic panoramic image. If the image is determined to not include synthetic image content, a neural network is applied to the image and the image is classified as one of non-synthetic panoramic or non-synthetic non-panoramic.

Abstract: Methods and systems are provided for performing material capture to determine properties of an imaged surface. A plurality of images can be received depicting a material surface. The plurality of images can be calibrated to align corresponding pixels of the images and determine reflectance information for at least a portion of the aligned pixels. After calibration, a set of reference materials from a material library can be selected using the calibrated images. The set of reference materials can be used to determine a material model that accurately represents properties of the material surface.

Abstract: Disclosed herein are embodiments of systems, methods, and products for a webly supervised training of a convolutional neural network (CNN) to predict emotion in images. A computer may query one or more image repositories using search keywords generated based on the tertiary emotion classes of Parrott's emotion wheel. The computer may filter images received in response to the query to generate a weakly labeled training dataset labels associated with the images that are noisy or wrong may be cleaned prior to training of the CNN. The computer may iteratively train the CNN leveraging the hierarchy of emotion classes by increasing the complexity of the labels (tags) for each iteration. Such curriculum guided training may generate a trained CNN that is more accurate than the conventionally trained neural networks.

Abstract: Provided is a broadcast signal transmitting method including: generating a first bitstream including encoded data; generating a second bitstream by adding zero bits to the first bitstream, based on a difference between a modem data rate and a codec data rate; generating a third bitstream by encoding the second bitstream by using zero padding information about the zero bits; and transmitting a carrier signal generated by modulating the third bitstream.

Abstract: Methods and systems are provided for performing material capture to determine properties of an imaged surface. A plurality of images can be received depicting a material surface. The plurality of images can be calibrated to align corresponding pixels of the images and determine reflectance information for at least a portion of the aligned pixels. After calibration, a set of reference materials from a material library can be selected using the calibrated images. The set of reference materials can be used to determine a material model that accurately represents properties of the material surface.

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for automatic tagging of videos. In particular, in one or more embodiments, the disclosed systems generate a set of tagged feature vectors (e.g., tagged feature vectors based on action-rich digital videos) to utilize to generate tags for an input digital video. For instance, the disclosed systems can extract a set of frames for the input digital video and generate feature vectors from the set of frames. In some embodiments, the disclosed systems generate aggregated feature vectors from the feature vectors. Furthermore, the disclosed systems can utilize the feature vectors (or aggregated feature vectors) to identify similar tagged feature vectors from the set of tagged feature vectors. Additionally, the disclosed systems can generate a set of tags for the input digital videos by aggregating one or more tags corresponding to identified similar tagged feature vectors.

Abstract: In implementations of segmenting objects in video sequences, user annotations designate an object in any image frame of a video sequence, without requiring user annotations for all image frames. An interaction network generates a mask for an object in an image frame annotated by a user, and is coupled both internally and externally to a propagation network that propagates the mask to other image frames of the video sequence. Feature maps are aggregated for each round of user annotations and couple the interaction network and the propagation network internally. The interaction network and the propagation network are trained jointly using synthetic annotations in a multi-round training scenario, in which weights of the interaction network and the propagation network are adjusted after multiple synthetic annotations are processed, resulting in a trained object segmentation system that can reliably generate realistic object masks.