Abstract: In implementations of systems for joint trimap estimation and alpha matte prediction, a computing device implements a matting system to estimate a trimap for a frame of a digital video using a first stage of a machine learning model. An alpha matte is predicted for the frame based on the trimap and the frame using a second stage of the machine learning model. The matting system generates a refined trimap and a refined alpha matte for the frame based on the alpha matte, the trimap, and the frame using a third stage of the machine learning model. An additional trimap is estimated for an additional frame of the digital video based on the refined trimap and the refined alpha matte using the first stage of the machine learning model.

Abstract: A method, apparatus, non-transitory computer readable medium, and system for generating synthetic videos includes obtaining an input prompt describing a video scene. The embodiments then generate a plurality of frame-wise token embeddings corresponding to a sequence of video frames, respectively, based on the input prompt. Subsequently, embodiments generate, using a video generation model, a synthesized video depicting the video scene. The synthesized includes a plurality of images corresponding to the sequence of video frames.

Abstract: In one aspect, a processor determines a first set of video frames of a video based on a target video frame. The first set of video frames includes the target video frame, one or more frames of the video preceding the target video frame, and one or more frames of the video subsequent to the target video frame. The first set of video frames includes a sequence of video frames of the video. An encoder neural network executing on the processor encodes the first set of video frames of a video to generate a respective feature vector for each video frame in the first set. A decoder neural network executing on the processor decodes the feature vectors to generate a mask for the target video frame.

Abstract: Embodiments are disclosed for a method including obtaining a region of interest of a current frame of a video sequence depicting an object. The method may further include determining, by a mask propagation model, a likelihood of each pixel of the current frame being associated with the object in the region of interest of the current frame based on the region of interest of the current frame and a fixed number of previous frames of the video sequence including the object. The method may further include replacing a previous frame of the fixed number of previous frames with the current frame. The method may further include displaying the current frame of the video sequence including a masked object in the region of interest of the current frame based on the likelihood of one or more pixels of the current frame being associated with the object.

Abstract: Embodiments are disclosed for performing universal segmentation to mask objects across multiple frames of a video. The method may include determining an image segmentation mask which masks an object of a frame of a video sequence using the frame and an image segmentation module of a segmentation system. The method further includes determining a mask propagation mask which masks the object of the frame of the video sequence using the frame, a representation of a previous frame of the video sequence, and a mask propagation module of the segmentation system. The method further includes determining a frame mask which masks the object of the frame of the video sequence based on a comparison of the image segmentation mask and the mask propagation mask. The method further includes displaying the frame mask of the video sequence.

Abstract: Proposed are an apparatus for recording a holographic interference pattern and a method of recording a holographic interference pattern using the apparatus, the apparatus and the method being capable of simplifying a process of aligning a light source and a photosensitive material in order to record a holographic interference pattern on a three-dimensional coordinate system and then realigning the light source and the photosensitive material in order to record a different holographic interference pattern.

Abstract: A method includes receiving a frame depicting an object. The frame is one frame of a plurality of frames of a video sequence. The method further includes encoding a plurality of tokens of the frame. Each token is a representation of a grid of pixels of the frame. The method further includes selecting a subset of tokens for decoding based on a likelihood of a token satisfying a confidence threshold. The token satisfies the confidence threshold based on a confidence score of the token including a past object in a past frame. The method further includes decoding the subset of tokens using a decoder.

Abstract: The present disclosure provides a display apparatus. The display apparatus includes a light source; a compensating diffraction optical element which reflects a light incident from the light source; and an off-axis reflection diffraction optical element which reflects a light reflected from the compensating diffraction optical element to the outside, wherein the compensating diffraction optical element reflects the light incident from the light source toward the off-axis reflection diffraction optical element while adjusting the reflection angle of the light incident from the light source so as to compensate for the reflection angle distortion of the light reflected from the off-axis reflection diffraction optical element.

Abstract: Embodiments are disclosed for a machine learning-based chroma keying process. The method may include receiving an input including an image depicting a chroma key scene and a color value corresponding to a background color of the image. The method may further include generating a preprocessed image by concatenating the image and the color value. The method may further include providing the preprocessed image to a trained neural network. The method may further include generating, using the trained neural network, an alpha matte representation of the image based on the preprocessed image.

Abstract: Systems and methods perform image matte generation using image bursts. In accordance with some aspects, an image burst comprising a set of images is received. Features of a reference image from the set of images is aligned with features of other images from the set of images. A matte for the reference image is generated using the aligned features.

Abstract: Systems and methods for video processing are described. Embodiments of the present disclosure identify an image that depicts an expression of a face; encode the image to obtain a latent code representing the image; edit the latent code to obtain an edited latent code that represents the face with a target attribute that is different from an original attribute of the face and with an edited expression that is different from the expression of the face; modify the edited latent code to obtain a modified latent code that represents the face with the target attribute and a modified expression, wherein a difference between the expression and the modified expression is less than a difference between the expression and the edited expression; and generate a modified image based on the modified latent code, wherein the modified image depicts the face with the target attribute and the modified expression.

Abstract: Systems and methods for easily extracting images of sections from existing video and generating sub-videos are illustrated. One embodiment includes a video generating method including obtaining a target video including a plurality of frame images, wherein the target video is transmitted through a session established between two or more terminals, obtaining first frame information about at least some frame images, from among the plurality of frame images, selected to be inference targets, inputting the at least some frame images corresponding to the first frame information into an inference model to obtain content event information for extracting at least one extracted video with respect to the target video, obtaining information about a selected section of the target video based on the content event information, and generating a first extracted video based on the target video and information about the selected section.

Abstract: The present disclosure provides a waveguide display apparatus. The waveguide waveguide display apparatus of the present disclosure is a waveguide display apparatus for correcting curved surface reflection distortion, the waveguide display apparatus including a waveguide for guiding light inputted from the outside; a first diffractive optical element disposed at the waveguide, and diffracting the light inputted from the outside to the inside of the waveguide; and a second diffractive optical element disposed at the waveguide, and diffracting the light guided by the waveguide to output a plurality of diffracted lights in a direction of a curved surface reflector located outside, wherein the second diffractive optical element has a structure of a diffraction grating corresponding to a curvature of the curved surface reflector such that the diffracted lights are reflected at different locations of the curved surface reflector in directions parallel to each other.

Abstract: A method for replicating a holographic optical element and a holographic optical element replicated thereby are provided. The holographic optical element is larger than a master. The master has a holographic grating pattern generated on the master by interference of the reflected, diffracted or transmitted beam generated by irradiating the master having a specific diffraction grating pattern formed thereon with a laser beam.

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for automatic tagging of videos. In particular, in one or more embodiments, the disclosed systems generate a set of tagged feature vectors (e.g., tagged feature vectors based on action-rich digital videos) to utilize to generate tags for an input digital video. For instance, the disclosed systems can extract a set of frames for the input digital video and generate feature vectors from the set of frames. In some embodiments, the disclosed systems generate aggregated feature vectors from the feature vectors. Furthermore, the disclosed systems can utilize the feature vectors (or aggregated feature vectors) to identify similar tagged feature vectors from the set of tagged feature vectors. Additionally, the disclosed systems can generate a set of tags for the input digital videos by aggregating one or more tags corresponding to identified similar tagged feature vectors.

Abstract: This disclosure describes one or more implementations of a panoptic segmentation system that generates panoptic segmented digital images that classify both known and unknown instances of digital images. For example, the panoptic segmentation system builds and utilizes a panoptic segmentation neural network to discover, cluster, and segment new unknown object subclasses for previously unknown object instances. In addition, the panoptic segmentation system can determine additional unknown object instances from additional digital images. Moreover, in some implementations, the panoptic segmentation system utilizes the newly generated unknown object subclasses to refine and tune the panoptic segmentation neural network to improve the detection of unknown object instances in input digital images.

Abstract: Systems and methods for video processing are described. Embodiments of the present disclosure identify an image that depicts an expression of a face; encode the image to obtain a latent code representing the image; edit the latent code to obtain an edited latent code that represents the face with a target attribute that is different from an original attribute of the face and with an edited expression that is different from the expression of the face; modify the edited latent code to obtain a modified latent code that represents the face with the target attribute and a modified expression, wherein a difference between the expression and the modified expression is less than a difference between the expression and the edited expression; and generate a modified image based on the modified latent code, wherein the modified image depicts the face with the target attribute and the modified expression.

Abstract: Embodiments are disclosed for performing per-clip object segmentation of objects in a video sequence using machine learning. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a query video sequence and memory data, the memory data including a memory video frame from the query video sequence and an annotated memory video frame including an object mask for an object in the memory video frame, segmenting the query video sequence into a plurality of query video clips and passing a first set of query video frames of a first query video clip and the memory data through a trained encoder-decoder network, predicting a modified set of query video frames, including predictions of object masks for the object, and updating the memory data to include one or more frames of the first set of query video frames and the modified set of query video frames.

Abstract: In implementations of systems for joint trimap estimation and alpha matte prediction, a computing device implements a matting system to estimate a trimap for a frame of a digital video using a first stage of a machine learning model. An alpha matte is predicted for the frame based on the trimap and the frame using a second stage of the machine learning model. The matting system generates a refined trimap and a refined alpha matte for the frame based on the alpha matte, the trimap, and the frame using a third stage of the machine learning model. An additional trimap is estimated for an additional frame of the digital video based on the refined trimap and the refined alpha matte using the first stage of the machine learning model.

Abstract: This disclosure describes one or more implementations of a panoptic segmentation system that generates panoptic segmented digital images that classify both known and unknown instances of digital images. For example, the panoptic segmentation system builds and utilizes a panoptic segmentation neural network to discover, cluster, and segment new unknown object subclasses for previously unknown object instances. In addition, the panoptic segmentation system can determine additional unknown object instances from additional digital images. Moreover, in some implementations, the panoptic segmentation system utilizes the newly generated unknown object subclasses to refine and tune the panoptic segmentation neural network to improve the detection of unknown object instances in input digital images.