Abstract: Techniques are disclosed for generating virtual reference frames that may be used for prediction of input video frames. The virtual reference frames may be derived from already-coded reference frames and thereby incur reduced signaling overhead. Moreover, signaling of virtual reference frames may be avoided until an encoder selects the virtual reference frame as a prediction reference for a current frame. In this manner, the techniques proposed herein contribute to improved coding efficiencies.

Abstract: A video dynamic mesh coding (v-DMC) decoding system, includes a de-multiplexer that receives and demultiplexes an encoded v-DMC bitstream into: a parameter set and mesh data, geometry, atlas data, and attribute video substreams. The decoding system also includes: a mesh data substream decoder; a video decoder that decodes the geometry data substream; an atlas data substream decoder; a mesh subdivision component that subdivides the one or more base meshes into one or more resampled base meshes based upon the decoded atlas data; a displacement decoder that outputs one or more displacements to verticies of the one or more resampled base meshes; a mesh position refinement component that applies the one or more displacements to the one or more resampled base meshes and outputs one or more resultant meshes; and a video decoder that decodes the attribute video substream into one or more texture images.

Abstract: A method of compressing a 3D textured mesh M(i), the 3D textured mesh being defined by connectivity C(i), geometry G(i), texture coordinates T(i), and texture connectivity CT(i), wherein the mesh is associated with one or more 2D image attribute maps A(i) describing attributes associated with the mesh surface, can include pre-processing 3D textured mesh M(i) and attribute maps A(i) to generate a base mesh m(i) and displacement field d(i); and processing 3D textured mesh M(i), attribute maps A(i), base mesh m(i), and the displacement field d(i) to generate a compressed bitstream b(i).

Abstract: A method of post processing a decoded 3D textured mesh to adaptively tesselate the mesh, can include: receiving from a mesh decoder one or more of metadata(i) describing various information about the mesh structure, a decoded base mesh m?(i), displacements d?(i) associated with the vertices of the decoded base mesh, and optionally one or more attribute maps A?(i) describing information associated with the mesh surface; and receiving control parameters from an application consuming the decoded 3D textured mesh; and performing one or more subdivisions of the decoded base mesh m?(i) based on the received control information.

Abstract: A re-meshing pre-processor for re-meshing a 3D textured mesh M(i) to generate a base mesh m(i) and displacement field d(i) for input into a mesh encoder, can include a Mesh Decimation module that includes processing hardware that reduces the number of vertices or faces of input mesh M(i), or a mesh derived therefrom, while substantially preserving the shape of the input mesh M(i), thereby producing a decimated mesh dm(i) and a projected mesh P(i); and a Fitting Subdivision Surface module that includes processing hardware that processes the input mesh M(i), the decimated mesh dm(i) or a mesh derived therefrom, and the projected mesh P(i) to produce a base mesh m(i) and the displacement field d(i) for input into a mesh encoder.

Abstract: A method of encoding motion data associated with an input data corresponding to set of 3D meshes M(i), the motion data including at least one of geometry and vertex attribute changes from one frame to another, can include: dividing input mesh M(i) into a set of patches P(i, j), each patch P(i, j) corresponding to a corresponding patch P(k, l) in a previously encoded reference frame; quantizing at least one of vertices and attributes of each patch P(i, j); predicting residuals based on a difference between quantized vertices or attributes of each patch P(i, j) with respect to corresponding patch P(k, l) in the previously encoded reference frame; and entropy encoding the predicted residuals.

Abstract: An encoder system may include an analyzer that analyzes a current image area in an input video to select a transform. A selectable residue transformer, controlled by the analyzer, may perform the selectable transform on a residue image generated from the current image area and a predicted current image area, to generate a transformed residue image. An encoder may encode the transformed residue image to generate output data. The analyzer controls the encoder to encode information to identify the selectable transform and to indicate that the selectable transform for the current image area is different from a transform of a previous image area of the input video. A decoder system may include components appropriate for decoding the output data from the encoder system.

Abstract: Techniques for coding and decoding video may include predicting picture regions defined by a time-varying tessellation and/or by a tessellation that varies spatially within a picture. These techniques improve decoded video quality, for example, by reducing block-based visual artifacts. Tessellation patterns may be irregular spatially to prevent alignment of some prediction region boundaries within a picture. Tessellation patterns may vary over time based on a spatial offset value, and the spatial offset value may be determined via a modulo function. Tessellation patterns may include overlapped shapes, for example when used in conjunction with overlapped block motion compensation.

Abstract: A system comprises an encoder configured to compress attribute information for a point cloud and/or a decoder configured to decompress compressed attribute information. Attribute values for at least one starting point are included in a compressed attribute information file and attribute correction values are included in the compressed attribute information file. An order for the points is determined based on a space filling curve, wherein an encoder and a decoder determine a same order for the points based on the space filling curve. Levels of detail are determined by sampling the ordered points according to different sampling parameters, and attribute values are predicted for the points in the levels of detail using the determined order. The encoder determines attribute correction values based on a comparison of the predicted values to an original value prior to compression. The decoder corrects the predicted attribute values based on received attribute correction values.

Abstract: A system comprises an encoder configured to compress attribute information and/or spatial information for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. The encoder is configured to convert a point cloud into an image based representation. The encoder packs patch images into an image frame and fills empty spaces in the image frame with a padding. Various compression strategies may be used to encode an occupancy map and related block-to-patch information indicating which portions of the image frame correspond to which packed patches. Packed image frames comprising patches and padding, along with an encoded occupancy map and related block-to-patch information are sent to a decoder. The decoder is configured to generate a decompressed point cloud based on the packed image frames, compressed occupancy map, and related block-to-patch information.

Abstract: In communication applications, aggregate source image data at a transmitter exceeds the data that is needed to display a rendering of a viewport at a receiver. Improved streaming techniques that include estimating a location of a viewport at a future time. According to such techniques, the viewport may represent a portion of an image from a multi-directional video to be displayed at the future time, and tile(s) of the image may be identified in which the viewport is estimated to be located. In these techniques, the image data of tile(s) in which the viewport is estimated to be located may be requested at a first service tier, and the other tile in which the viewport is not estimated to be located may be requested at a second service tier, lower than the first service tier.

Abstract: A system comprises an encoder configured to compress attribute information for a point cloud and/or a decoder configured to decompress compressed attribute for the point cloud. To compress the attribute information, attribute values are predicted using one of a plurality of prediction strategies, wherein a selected prediction strategy is selected based at least in part on attribute variability of points in a neighborhood of points. A decoder follows a similar prediction process. Also, attribute correction values may be determined to correct predicted attribute values and may be used by a decoder to decompress a point cloud, wherein the decoder applies the same prediction strategy applied at the encoder.

Abstract: A method of adaptive chroma downsampling is presented. The method comprises converting a source image to a converted image in an output color format, applying a plurality of downsample filters to the converted image and estimating a distortion for each filter chose the filter that produces the minimum distortion. The distortion estimation includes applying an upsample filter, and a pixel is output based on the chosen filter. Methods for closed loop conversions are also presented.

Abstract: A system comprises an encoder configured to compress attribute information and/or spatial for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. To compress the attribute and/or spatial information, the encoder is configured to convert a point cloud into an image based representation. Also, the decoder is configured to generate a decompressed point cloud based on an image based representation of a point cloud. Additionally, an encoder is configured to signal and/or a decoder is configured to receive a supplementary message comprising volumetric tiling information that maps portions of 2D image representations to objects in the point. In some embodiments, characteristics of the object may additionally be signaled using the supplementary message or additional supplementary messages.

Abstract: A system comprises an encoder configured to compress attribute information and/or spatial information for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. To compress the attribute and/or spatial information, the encoder is configured to convert a point cloud into an image based representation. Also, the decoder is configured to generate a decompressed point cloud based on an image based representation of a point cloud. In some embodiments, a bit stream structure may be used to communicate compressed point cloud data. The bit stream structure may include point cloud compression network abstraction layer (PCCNAL) units that enable use of groups of frames (GOFs), frame, and sub-frame signaling of patch information. Such a bit stream structure may permit low delay streaming and random access reconstruction of point clouds amongst other applications.

Abstract: A system comprises an encoder configured to entropy encode a bitstream comprising both compressible and non-compressible symbols. The encoder parses the bitstream into a compressible symbol sub-stream and a non-compressible sub-stream. The non-compressible symbol sub-stream bypass an entropy encoding component of the encoder while the compressible symbol sub-stream is entropy encoded. When a quantity of bytes of entropy encoded symbols and bypass symbols is accumulated a chunk of fixed or known size is formed using the accumulated entropy encoded symbol bytes and the bypass bytes without waiting on the full bitstream to be processed by the encoder. In a complementary manner, a decoder reconstructs the bitstream from the packets or chunks.

Abstract: An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bitstream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.

Abstract: An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bitstream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.

Abstract: A search space for performing nearest neighbor searches for encoding point cloud data may be trimmed. Ranges of a space filling curve may be used to identify search space to exclude or reuse, instead of generating nearest neighbor search results for at least some of the points of a point cloud located within some of the ranges of the space filling curve. Additionally, neighboring voxels may be searched to identify any neighboring points missed during the trimmed search based on the ranges of the space filling curve.

Abstract: Sampled data is packaged in checkerboard format for encoding and decoding. The sampled data may be quincunx sampled multi-image video data (e.g., 3D video or a multi-program stream), and the data may also be divided into sub-images of each image which are then multiplexed, or interleaved, in frames of a video stream to be encoded and then decoded using a standardized video encoder. A system for viewing may utilize a standard video decoder and a formatting device that de-interleaves the decoded sub-images of each frame reformats the images for a display device. A 3D video may be encoded using a most advantageous interleaving format such that a preferred quality and compression ratio is reached. In one embodiment, the invention includes a display device that accepts data in multiple formats.