Abstract: A method and system determines flows by first acquiring a video of the flows with a camera, wherein the flows are pedestrians in a scene, wherein the video includes a set of frames. Motion vectors are extracted from each frame in the set, and a data matrix is constructed from the motion vectors in the set of frames. A low rank Koopman operator is determined from the data matrix and a spectrum of the low rank Koopman operator is analyzed to determine a set of Koopman modes. Then, the frames are segmented into independent flows according to a clustering of the Koopman modes.

Abstract: A set of input images are acquired sequentially as image tensors. A low-tubal rank tensor and a sparse tensor are initialized using the image tensor, wherein the low-tubal rank tensor is a tensor product of a low-rank spanning tensor basis and corresponding tensor coefficients, and for each image, updating iteratively the image tensor, the tensor coefficients, and the sparse tensor using the image tensor and the low-rank spanning basis from a previous iteration. The spanning tensor basis is updated using the tensor coefficients, the sparse tensor, and the low rank tubal tensor, wherein the low rank tubal tensor represents a set of output images and the sparse tensor representing a set of sparse images.

Abstract: A method segments an image acquired by a sensor of a scene by first obtaining motion vectors corresponding to the image and generating a motion vanishing point image, wherein each pixel in the motion vanishing point image represents a number of intersections of pairs of motion vectors at the pixel. In the motion vanishing point image, a representation point for each motion vector is generated and distances between the motion vectors are determined based on the representation points. Then, a motion graph is constructed wherein each node represents a motion vector, and each edge represents a weight based on the distance between the nodes. Graph spectral clustering is performed on the motion graph to produce segments of the image.

Abstract: A method separates foreground from background in a sequence of images, by first acquiring the sequence of images and a depth map of a scene by a camera. Groups of pixels are determined based on the depth map. Then, the sequence of images is decomposed into a sparse foreground component, and a low rank background component, according to apparent motion in the sequence of images, and the groups.

Abstract: An input segment of an input video is encoded by first extracting and storing, for each segment of previously encoded videos, a set of reference features. The set of input features are matched with each set of the reference features to produce a set of scores. The reference segments having largest scores are selected to produce a first reduced set of reference segments. A rate-distortion cost for each reference segment in the first reduced set of reference segments is estimated. The reference segments in the first reduced set of reference segments is selected to produce a second reduced set of reference segments. Then, the input segment are encoded based on second reduced set of reference segments.

Abstract: In a decoder, a desired image is estimated by first retrieving coding modes from an encoded side information image. For each bitplane in the encoded side information image, syndrome bits or parity bits are decoded to obtain an estimated bitplane of quantized transform coefficients of the desired image. A quantization and a transform are applied to a prediction residual obtained using the coding modes, wherein the decoding uses the quantized transform coefficients of the encoded side information image, and is based on previously decoded bitplanes in a causal neighborhood. The estimated bitplanes of quantized transform coefficients of the desired image are combined to produce combined bitplanes. Then, an inverse quantization, an inverse transform and a prediction based on the coding modes are applied to the combined bitplanes to recover the estimate of the desired image.

Abstract: Videos of a scene are processed for view synthesis. The videos are acquired by corresponding cameras arranged so that a view of each camera overlaps with the view of at least one other camera. For each current block, motion or disparity vector is obtained from neighboring blocks. A depth block is based on a corresponding reference depth image and the motion or disparity vector. A prediction block is generated based on the depth block using backward warping. Then, predictive coding for the current block using the prediction block.

Abstract: Videos of a scene are processed for view synthesis. The videos are acquired by corresponding cameras arranged so that a view of each camera overlaps with the view of at least one other camera. For each current block, disparity vectors are obtained from neighboring blocks. A depth block is based on a corresponding reference depth image and the disparity vectors. A prediction block is generated based on the depth block using backward warping of a motion field. Then, predictive coding for the current block using the prediction block. Backward mapping can also be performed in the spatial domain.

Abstract: A method processes a video acquired of a scene by first aligning a group of video images using compressed domain motion information and then solving for a low rank component and a sparse component of the video. A homography map is computed from the motion information to determine image alignment parameters. The video images are then warped using the homography map to share a similar camera perspective. A Newton root step is followed to traverse separately Pareto curves of each low rank component and sparse component. The solving for the low rank component and the sparse component is repeated alternately until a termination condition is reached. Then, the low ranks component and the sparse component are outputted. The low rank component represents a background in the video, and the sparse component represents moving objects in the video.

Abstract: Blocks in pixel images are template matched to select candidate blocks and weights according to a structural similarity and a perceptual distortion of the blocks. The perceptual distortion is a function of a just-noticeable-distortion (JND). A filter outputs a prediction residual between the block and the candidate blocks. The prediction residual is transformed and quantized to produce a quantized prediction residual using the JND. The matching and quantizing is optimized jointly using the perceptual distortion. Then, the quantized prediction residual and the weights are entropy encoded into a bit-stream for later decoding.

Abstract: A method decodes a picture in a form of a bitstream, wherein the picture includes components, by first receiving the bitstream in a decoder. The decoder includes an intra boundary filtering process. A flag is decoded from the bitstream. Then, the intra boundary filtering process is applied, according to the flag.

Abstract: A method processes a signal by first constructing a graph from the signal, and then determining a graph matrix from the graph and the signal. A Krylov-based subspace is determined based on the graph matrix and the signal. A filter for the Krylov subspace is determined. The filter transforms the signal to produce a filtered signal, which is output.

Abstract: A method decodes blocks in pictures of a video in an encoded bitstream by storing previously decoded blocks in a buffer. The previously decoded blocks are displaced less than a predetermined range relative to a current block being decoded. Cached blocks are maintained in a cache. The cached blocks include a set of best matching previously decoded blocks that are displaced greater than the predetermined range relative to the current block. The bitstream is parsed to obtain a prediction indicator that determines whether the current block is predicted from the previously decoded blocks in the buffer or the cached blocks in the cache. Based on the prediction indicator, a prediction residual block is generated, and in a summation process, the prediction residual block is added to a reconstructed residual block to form a decoded block as output.

Abstract: A method for decoding a bitstream, including compressed pictures of a video, wherein each picture includes one or more slices, wherein each slice includes one or more blocks of pixels, and each pixel has a value corresponding to a color, for each slice, first obtains a reduced number of colors corresponding to the slice, wherein each color is represented as a color triplet and the reduced number of colors is less than or equal to a number of colors in the slice. Then, for each block, a prediction mode is determined, wherein an independent uniform prediction mode is included in a candidate set of prediction modes. For each block, a predictor block is generated, wherein all values of the predictor block have a uniform value according to a color index when the prediction mode is set as the independent uniform prediction mode. Lastly, the predictor block is added to a reconstructed residue block to form a decoded block as output.

Abstract: A method extracts a low-rank descriptor of a video acquired of a scene by first extracting a set of descriptors for each image in the video. The sets of descriptors for the video are aggregated to form a descriptor matrix. Iteratively, a low-rank descriptor matrix is determined from the descriptor matrix, as well as a selection matrix that associates each column in the descriptor matrix to a corresponding column in the low-rank descriptor matrix. The low-rank descriptor matrix is output upon convergence.

Abstract: A method processes a video acquired of a scene by first aligning a group of video images using compressed domain motion information and then solving for a low rank component and a sparse component of the video. A homography map is computed from the motion information to determine image alignment parameters. The video images are then warped using the homography map to share a similar camera perspective. A Newton root step is followed to traverse separately Pareto curves of each low rank component and sparse component. The solving for the low rank component and the sparse component is repeated alternately until a termination condition is reached. Then, the low ranks component and the sparse component are outputted. The low rank component represents a background in the video, and the sparse component represents moving objects in the video.

Abstract: A method decodes a picture that is encoded and represented by blocks in a bitstream, by first determining, from the bitstream, motion associated with the block. Using a model, the motion is mapped to indices indicating a subset of quantized transform coefficients to be decoded from the bitstream. Then, values are assigned and reinserted to the quantized transform coefficients not in the subset.

Abstract: A method processes a signal represented as a graph by first determining a graph spectral transform based on the graph. In a spectral domain, parameters of a graph filter are estimated using a training data set of unenhanced and corresponding enhanced signals. The graph filter is derived based on the graph spectral transform and the estimated graph filter parameters. Then, the signal is processed using the graph filter to produce an output signal. The processing can enhance signals such as images b denoising or interpolating missing samples.

Abstract: In a decoder, a desired image is estimated by first retrieving coding modes from an encoded side information image. For each bitplane in the encoded side information image, syndrome bits or parity bits are decoded to obtain an estimated bitplane of quantized transform coefficients of the desired image. A quantization and a transform are applied to a prediction residual obtained using the coding modes, wherein the decoding uses the quantized transform coefficients of the encoded side information image. The estimated bitplanes of quantized transform coefficients of the desired image are combined to produce combined bitplanes. Then, an inverse quantization, an inverse transform and a prediction based on the coding modes are applied to the combined bitplanes to recover the estimate of the desired image.

Abstract: An image for a virtual view of a scene is generated based on a set of texture images and a corresponding set of depth images acquired of the scene. A set of candidate depths associated with each pixel of a selected image is determined. For each candidate depth, a cost that estimates a synthesis quality of the virtual image is determined. The candidate depth with a least cost is selected to produce an optimal depth for the pixel. Then, the virtual image is synthesized based on the optimal depth of each pixel and the texture images. The method also applies first and second depth enhancement before, and during view synthesis to correct errors or suppress noise due to the estimation or acquisition of the dense depth images and sparse depth features.