Abstract: Systems, methods, and instrumentalities are disclosed for performing motion information candidates re-ordering. A device may be configured to obtain motion information, for example, comprising motion information candidates. The device may obtain a video block including one or more subblocks. The device may determine a motion model (e.g., global motion model and/or local motion model) associated with a subblock. The device may compute a score associated with a motion information candidate, for example, based on the motion model. The score may be determined based on a determined consistency between the motion model and the motion information candidate. The device may perform refinement on the motion information candidate. The score may be computed based on a refined motion information candidate. The device may determine an order associated with the motion information candidates, for example, based on the scores. The order may be organized from a low score to a high score.

Abstract: There are several types of plenoptic devices having their proprietary file format. At present there is no standard supporting the acquisition and transmission of multi-dimensional information for an exhaustive over-view of the different parameters upon which a light-field depends. As such acquired light-field data for different cameras have a diversity of formats. The notion of pixel beam, which represents a volume occupied by a set of rays of light in an object space of an optical system of a camera is thus introduced. The method according to the invention enables to provide data representative of a collection of pixel beams describing a first optical system that is agnostic since these data are obtained by imaging the collection of pixel beams through a second optical system. Such data representative of a collection of pixel beams enable the generation of parameterized output images from which post-processing.

Abstract: Methods and apparatus to fill in a motion information buffer with coding units or sub-blocks of coding units that are coded in intra mode comprise selection of neighboring inter prediction parameters into the motion information buffer using an intra prediction direction. In one embodiment, rescaling of inter prediction parameters is performed. In a variant, motion information such as motion vectors can be averaged or an affine model can be applied before storing of the parameters in a motion information buffer.

Abstract: A method for decoding or encoding includes obtaining views parameters for a set of views comprising at least one reference view and a current view of a multi-views video content wherein each view comprises a texture layer and a depth layer. For at least one couple of a reference view and the current view of the set of views, an intermediate prediction image applying a forward projection method to pixels of the reference view is generated to project these pixels from a camera coordinates system of the reference view to a camera coordinates system of the current view, the prediction image comprising information allowing reconstructing image data. At least one final prediction image obtained from at least one intermediate prediction image is stored in a buffer of reconstructed images of the current view. A current image of the current view from the images stored in said buffer is reconstructed, said buffer comprising said at least one final prediction image.

Abstract: A device includes at least one first sensor and at least one second sensor for capturing first image data, the at least one first and second sensors being rectangular and arranged orthogonal to the at least one first rectangular sensor, and at least one hardware processor configured to cause the at least one first and second sensors, respectively, the first image data and the second image data at least substantially simultaneously, and at least one of display simultaneously data from the first image data and data from the second image data as a cross-shaped image or store together data from the first image data and data from the second image data as a cross-shaped image. The resulting first and second image data can be stored in a single file in memory. The at least one hardware processor can be processed to remove redundancies between the image data. The device can also extract from the first and second image data, image data corresponding to a rectangle parallel with an horizon.

Abstract: A method and an apparatus for encoding or decoding a video are provided. At least one motion compensation interpolation filter for a block of the video is determined. In an embodiment, the length of the motion compensation interpolation filter is adapted based on block conditions. A prediction block is determined based on the at least one motion compensation interpolation filter and a reference block determined for the block and the block of the video is encoded or decoded based at least on the prediction block.

Abstract: Systems and method for processing image content are described. In one embodiment, the method comprises receiving a plurality of captured contents showing same scene as captured by one or more cameras having a different focal length and depth maps and generating a consensus cube by obtaining depth map estimations from said received contents. The visibility of different objects in then analyzed to create a soft visibility cube that provides visibility information about each content. A color cube is then generated by using information from the consensus and soft visibility cube. The color cube is then used to combine different received contents and generate a single image for the plurality of contents received.

Abstract: A method and system are provided for processing image content. In one embodiment the method comprises receiving a plurality of captured contents showing same scene as captured by one or more cameras having a different focal length and depth maps and generating a consensus cube by obtaining depth map estimations from said received contents. The visibility of different objects in then analysed to create a soft visibility cube that provides visibility information about each content. A color cube is then generated by using information from the consensus and soft visibility cube. The color cube is then used to combine different received contents and generate a single image for the plurality of contents received.

Abstract: A method and system are provided for generating an image from each camera array in a camera array matrix. In one embodiment, the method comprises increasing redundancy between to be captured images from the camera matrix by rotating direction of any cameras disposed in upper and lower rows of said matrix by a 90 degree angle around the roll axis. Then any cameras disposed at the corners of the matrix are rotated in an angle that is less than 90 degrees around the roll axis. Subsequently, the location of central cameras are determined and analysed so that they can be rotated and disposed in a manner that provides both horizontal and vertical compensation for any redundancies.

Abstract: The present disclosure relates to method and apparatus for the estimation of an accurate multiplane image from a set of posed images. The accuracy of an MPI representation is measured by the performance of the views generated by it through inverse homography sampling and alpha compositing. A variable number of posed images at a possibly different resolution representative of scenes with arbitrary geometry are used as input of the system. A first level of three modules, possibly trained based on deep learning methods, is built for extracting view and scene features and color scores. Numerous levels of the same three modules may be built and organized in a structure where a level deals with down-scaled version of the posed images.

Abstract: Predicting a component of a current pixel belonging to a current sub-aperture image in a matrix of sub-aperture images captured by a sensor of a type I plenoptic camera can involve, first, determining a location on the sensor based on: a distance from an exit pupil of a main lens of the camera to a micro-lens array of the camera; a focal length of the main lens; a focal length of the micro-lenses of the micro-lens array; and a set of parameters of a model of the camera allowing for a derivation of a two-plane parameterization describing the field of rays corresponding to the pixels of the sensor; and, second, predicting the component based on one reference pixel belonging to a reference sub-aperture image in the matrix and located on the sensor in a neighborhood of the location.

Abstract: An apparatus and a method are provided for image processing. In one embodiment, the method comprises accessing a plurality of images captured by at least a reference camera, wherein the images represent a plurality of views corresponding to said same scene. A plurality of plane sweep volume (PSV) slices are then generated from said images and computing for each slice a flow map from at least the reference camera calibration parameter and this flow map a previous slice of the plane sweep volume is generated.

Abstract: Methods and apparatuses for encoding/decoding data content representative of a volumetric video are provided, wherein. the encoding/decoding comprises encoding in/decoding from a bitstream. an indicator specifying whether data content has information representative of at least one set of depth layers. the information representative of a set of depth layers specifying a number of depth layers and a depth value for each of the depth layers for a layer-based representation of the volumetric video. Methods and apparatuses for reconstructing Computer Generated Holograms from a reconstructed layered-based representation of the volumetric video are also provided.

Abstract: Methods (1400) and apparatuses for encoding/decoding a sequence of multiple plane images representative of a 3D scene are provided, wherein the sequence of multiple plane images comprises at least one multiple plane image. a multiple plane image comprising a plurality of layers. said encoding comprising encoding in a bitstream. for at least one layer of the plurality of layers of the at least one multiple plane image. an indicator indicating whether the at least one layer has changed with respect to a corresponding layer of a reference multiple plane image, and encoding the at least one multiple plane image. Methods (1100) and apparatuses for reconstructing Computer Generated Holograms from the sequence of multiple plane images are also provided.

Abstract: For multi-view video content represented in the MVD (Multi-view+Depth) format, the depth maps may be processed to improve the coherency therebetween. In one implementation, to process a target view based on an input view, pixels of the input view are first projected into the world coordinate system, then into the target view to form a projected view. The texture of the projected view and the texture of the target view are compared. If the difference at a pixel is small, then the depth of the target view at that pixel is adjusted, for example, replaced by the corresponding depth of the projected view. When the multi-view video content is encoded and decoded in a system, depth map processing may be applied in the pre-processing and post-processing modules to improve video compression efficiency and the rendering quality.

Abstract: A method and system are provided for processing image content. The method comprises receiving information about a content image captured at least by one camera. The content includes multi-view representation of an image including both distorted and undistorted areas. The camera parameters and image parameters are then obtained and used to determine to which areas are undistorted and which areas are distorted in said image. This is used to calculate depth map of the image using the determined undistorted and distorted information. A final stereoscopic image is then rendered that uses the distorted and undistorted areas and calculation of depth map.

Abstract: Methods and devices for generated multiplane images from a representation of a 3D scene are disclosed. When an MPI is generated by the system, an image of the accumulation of the alpha values is computed. A convolutional networked is trained with a penalization function that penalizes MPIs with high values of accumulated alpha according to a parameter selected or trained to reduce the redundancy of information between layers of the MPI. When generating a MPI from a subset of the views of a new captured 3D scene, a first MPI is generated with the trained CNN with the associated parameter. An error is calculated between the other views and corresponding synthesized with the first MPI. This error is used to modify the parameter. Then, a second MPI is generated with the trained CNN with the new parameter.

Abstract: In one embodiment of the disclosure, it is proposed a method for encoding a matrix of image views obtained from data acquired by a plenoptic camera. The method is remarkable in that it comprises: obtaining at least one reference image view from a combination of at least two image views from said matrix of image views; encoding image views that are different from said at least one reference image view, based on said at least one reference image view.

Abstract: A method for reducing the parameters defining an acquired light field ray which enables only the colour associated with the light field ray to be stored instead of 4 light field co-ordinates (x,y,i,j) and its associated colour.

Abstract: The invention relates to layered depth data. In multi-view images, there is a large amount of redundancy between images. Layered Depth Video format is a well-known formatting solution for formatting multi-view images which reduces the amount of redundant information between images. In LDV, a reference central image is selected and information brought by other images of the multi-view images that are mainly regions occluded in the central image are provided. However, LDV format contains a single horizontal occlusion layer, and thus fails rendering viewpoints that uncover multiple layers dis-occlusions. The invention uses light filed content which offers disparities in every directions and enables a change in viewpoint in a plurality of directions distinct from the viewing direction of the considered image enabling to render viewpoints that may uncover multiple layer dis-occlusions which may occurs with complex scenes viewed with wide inter-axial distance.