Abstract: An electronic device and method of preservation of geometry details in tracked meshes is provided. A set of three-dimensional (3D) scans and a sequence of tracked meshes of an object of interest are acquired. The sequence of tracked meshes comprises a set of tracked meshes which temporally corresponds to the set of 3D scans. Based on differences between the set of tracked meshes and the set of 3D scans, a set of displacement maps is generated. A plurality of vectors is computed, each of which includes surface tension values associated with mesh vertices of a corresponding tracked mesh. Based on the set of displacement maps and a corresponding set of vectors, a neural network is trained on a task of displacement map generation. The trained neural network model is applied on the plurality of vectors to generate displacement maps. Each tracked mesh is updated based on a corresponding displacement map.

Abstract: An electronic device and method of preservation of geometry details in tracked meshes is provided. A set of three-dimensional (3D) scans and a sequence of tracked meshes of an object of interest are acquired. The sequence of tracked meshes comprises a set of tracked meshes which temporally corresponds to the set of 3D scans. Based on differences between the set of tracked meshes and the set of 3D scans, a set of displacement maps is generated. A plurality of vectors is computed, each of which includes surface tension values associated with mesh vertices of a corresponding tracked mesh. Based on the set of displacement maps and a corresponding set of vectors, a neural network is trained on a task of displacement map generation. The trained neural network model is applied on the plurality of vectors to generate displacement maps. Each tracked mesh is updated based on a corresponding displacement map.

Abstract: There is provided an image processing apparatus and an image processing method that enable selection of viewpoints of depth images suitable for creation of a 3D model. A viewpoint determination unit determines that a candidate viewpoint be a viewpoint of a depth image of a 3D model on the basis of the 3D model projected on a screen from the candidate viewpoint. The present disclosure can be applied to, for example, an encoding device or the like that generates and encodes color images and depth images from predetermined viewpoints, from color images and depth images from a plurality of viewpoints of a 3D model of a subject.

Abstract: A calibration system and method for online calibration of 3D scan data from multiple viewpoints is provided. The calibration system receives a set of depth scans and a corresponding set of color images of a scene that includes a human-object as part of a foreground of the scene. The calibration system extracts a first three-dimensional (3D) representation of the foreground based on a first depth scan and spatially aligns the extracted first 3D representation with a second 3D representation of the foreground. The first 3D representation and the second 3D representation are associated with a first viewpoint and a second viewpoint, respectively, in a 3D environment. The calibration system updates the spatially aligned first 3D representation based on the set of color images and a set of structural features of the human-object and reconstructs a 3D mesh of the human-object based on the updated first 3D representation of the foreground.

Abstract: The present disclosure relates to an image processing apparatus and an image processing method that make it possible to suppress deterioration of picture quality of an image within a viewing range of a viewer. An image processing apparatus includes an image processing section that performs, based on priorities between a plurality of encoded streams obtained by encoding a plurality of projection images that are obtained by projecting an omnidirectional image to a plurality of faces or a plurality of viewpoint images from different viewpoints, decoding of the encoded streams and generation or selection of an image to be used for generation of a display image, and a drawing section that generates the display image based on the generated or selected image.

Abstract: The present disclosure relates to an image processing apparatus and an image processing method that make it possible to generate a texture image of high picture quality at a predetermined viewpoint using an omnidirectional image. A plurality of real cameras image a plurality of reference images at a plurality of viewpoint positions in a plurality of sight line directions. A great weight is applied to a reference image in a sight line direction close to a virtual viewpoint specified in response to a movement of a head mounted display and having a virtual viewpoint position and a sight line direction to generate a virtual viewpoint image by weighted averaging, and the virtual viewpoint image is displayed on the head mounted display. The present disclosure can be applied, for example, to a home server or the like that generates a display image of a predetermined viewpoint from an omnidirectional image.

Abstract: The present disclosure relates to an information processing apparatus and an information processing method that are configured to be capable of efficiently acquiring information for use in generating three-dimensional data from two-dimensional image data. A grouping block sorts two or more virtual cameras for acquiring two-dimensional image data into two or more groups. A global table generation block generates a global table is which group information related with each of two or more groups is registered. A group table generation block generates, for each group, a group table in which camera information for use in generating three-dimensional data from two-dimensional image data acquired by a virtual camera sorted into a group is registered. The present disclosure is applicable to an encoding apparatus and the like, for example.

Abstract: The present disclosure relates to an information processing apparatus and an information processing method that are configured to be capable of efficiently acquiring information for use in generating three-dimensional data from two-dimensional image data. A grouping block sorts two or more virtual cameras for acquiring two-dimensional image data into two or more groups. A global table generation block generates a global table in which group information related with each of two or more groups is registered. A group table generation block generates, for each group, a group table in which camera information for use in generating three-dimensional data from two-dimensional image data acquired by a virtual camera sorted into a group is registered. The present disclosure is applicable to an encoding apparatus and the like, for example.

Abstract: The present disclosure relates to an image processing apparatus and an image processing method that make it possible to suppress deterioration of picture quality of an image within a viewing range of a viewer. An image processing apparatus includes an image processing section configured to perform, based on priorities between a plurality of encoded streams obtained by encoding a plurality of projection images that are obtained by projecting an omnidirectional image to a plurality of faces or a plurality of viewpoint images from different viewpoints, decoding of the encoded streams and generation or selection of an image to be used for generation of a display image, and a drawing section configured to generate the display image based on the generated or selected image. The present disclosure can be applied to a home server and so forth that generate a display image within a viewing range of a viewer from an omnidirectional image.

Abstract: The present disclosure relates to an image processing apparatus and an image processing method that make it possible to generate a texture image of high picture quality at a predetermined viewpoint using an omnidirectional image. A plurality of real cameras image a plurality of reference images at a plurality of viewpoint positions in a plurality of sight line directions. A great weight is applied to a reference image in a sight line direction close to a virtual viewpoint specified in response to a movement of a head mounted display and having a virtual viewpoint position and a sight line direction to generate a virtual viewpoint image by weighted averaging, and the virtual viewpoint image is displayed on the head mounted display. The present disclosure can be applied, for example, to a home server or the like that generates a display image of a predetermined viewpoint from an omnidirectional image.

Abstract: There is provided an image processing apparatus and an image processing method that enable selection of viewpoints of depth images suitable for creation of a 3D model. A viewpoint determination unit determines that a candidate viewpoint be a viewpoint of a depth image of a 3D model on the basis of the 3D model projected on a screen from the candidate viewpoint. The present disclosure can be applied to, for example, an encoding device or the like that generates and encodes color images and depth images from predetermined viewpoints, from color images and depth images from a plurality of viewpoints of a 3D model of a subject.

Abstract: The present disclosure relates to an information processing apparatus and an information processing method that are configured to be capable of efficiently acquiring information for use in generating three-dimensional data from two-dimensional image data. A grouping block sorts two or more virtual cameras for acquiring two-dimensional image data into two or more groups. A global table generation block generates a global table in which group information related with each of two or more groups is registered. A group table generation block generates, for each group, a group table in which camera information for use in generating three-dimensional data from two-dimensional image data acquired by a virtual camera sorted into a group is registered. The present disclosure is applicable to an encoding apparatus and the like, for example.