Abstract: A rendering process and system that may be used to composite virtual objects in panoramic video to provide a virtual reality and augmented reality experience. The process includes receiving low dynamic range (LDR) video data, e.g. 360° video; generating radiance maps, such as diffuse and specular maps, from the LDR data; inverse tone mapping the LDR data of the maps to generate high dynamic range (HDR) data for the maps; and receiving at least one virtual object and applying image based lighting (IBL) to the virtual object using the HDR data of the maps. A perceptually based threshold is also applied to radiance maps to detect prominent pixels, and using the prominent pixels as salient lights for image based shadowing (IBS) associated the virtual object. Objects are composited 360° video in real time using IBL and IBS without precomputation allowing user interaction with the objects.

Abstract: A rendering process and system that may be used to composite virtual objects in panoramic video to provide a virtual reality and augmented reality experience. The process includes receiving low dynamic range (LDR) video data, e.g. 360° video; generating radiance maps, such as diffuse and specular maps, from the LDR data; inverse tone mapping the LDR data of the maps to generate high dynamic range (HDR) data for the maps; and receiving at least one virtual object and applying image based lighting (IBL) to the virtual object using the HDR data of the maps. A perceptually based threshold is also applied to radiance maps to detect prominent pixels, and using the prominent pixels as salient lights for image based shadowing (IBS) associated the virtual object. Objects are composited 360° video in real time using IBL and IBS without precomputation allowing user interaction with the objects.

Abstract: Image processing apparatus and method. A modeling unit may generate a three-dimensional (3D) model corresponding to an actual object from an input color image and an input depth image corresponding to the input color image. A calculator may perform photon-based rendering with respect to an input virtual object and the 3D model based on input light environment information, and may generate a difference image comprising color distortion information occurring by inserting the virtual object into the 3D model. A rendering unit may generate a result image comprising the virtual object by synthesizing the difference image and the input color image.

Abstract: Disclosed is a method and apparatus for encoding a three-dimensional (3D) mesh. The method for encoding the 3D mesh includes determining a priority of a gate configuring a 3D mesh corresponding to a 3D object, removing vertices configuring the 3D mesh using the determined priority of the gate, and simplifying the 3D mesh.

Abstract: An image processing apparatus may include a light capturing unit to generate light environment information by photographing or capturing a light environment of an object, and an object capturing unit to generate object information by photographing or capturing the object. The image processing apparatus may further include a modeling unit to generate a 3-dimensional (3D) model by reflecting or combining the light environment information and the object information, and a rendering unit to render a resultant image with respect to at least one view using the 3D model.

Abstract: An apparatus and method for encoding a 3D mesh, and an apparatus and method for decoding the 3D mesh are disclosed. The 3D mesh encoding apparatus may determine mesh information including position information of each of vertices constituting the 3D mesh, and connectivity information among the vertices, based on a level, and may progressively encode the determined mesh information based on the level, thereby reducing an error with an original 3D object when compared to an equal transmission rating.

Abstract: An image processor, a lighting processor, and a method therefore are provided. According to one aspect of the invention, the lighting processor can extract information related to defuse lighting applied to a real object using a colored image and a depth image of the real object. The lighting processor can recover the diffuse image for the real object using the extracted information related to diffuse lighting, and generate either a speculum image or a shadow image using the recovered diffuse image and the colored image.

Abstract: Provided is an apparatus and method to create a model of a target organ of a patient. The apparatus and method generate a deformation matrix between at least two shape models of a target organ based on at least two three-dimensional (3D) images of the target organ. The apparatus and method also extract constraints about the target organ from a two-dimensional (2D) image of the target organ. The apparatus and method create the model of the target organ based on the constraints and the deformation matrix.

Abstract: An image processing apparatus. The image processing apparatus may perform filtering to emphasize a surface lighting of a low frequency band from an input lighting environment image, and may also perform at least one virtual point light (VPL) sampling from the input lighting environment image. The image processing apparatus may perform precomputed radiance transfer (PRT) based rendering using the filtered lighting environment image and VPL based rendering using the sampled VPL.

Abstract: A method and apparatus for expressing a rigid area based on expression control points that express a motion change of the rigid area with respect to an input expression. The method and apparatus express the rigid area by interpolating a change of expression of an avatar using a weight applied to an expression vector related to a plurality of expression samples to express the input expression and also using a transformation matrix set corresponding to the rigid area of each of the plurality of expression samples.

Abstract: A plane detection apparatus for detecting at least one plane model from an input depth image. The plane detection apparatus may include an image divider to divide the input depth image into a plurality of patches, a plane model estimator to calculate one or more plane models with respect to the plurality of patches including a first patch and a second patch, and a patch merger to iteratively merge patches having a plane model a similarity greater than or equal to a first threshold by comparing plane models of the plurality of patches. When a patch having the plane model similarity greater than or equal to the first threshold is absent, the plane detection apparatus may determine at least one final plane model with respect to the input depth image using previously merged patches.

Abstract: Provided is a method and apparatus for realistically reproducing an eyeball that may verify and analyze a material property and a deformation property with respect to each of constituent portions of an eyeball and may render each of the constituent portions based on the analyzed priorities, thereby more realistically reproducing the eyeball.

Abstract: A marker-less augmented reality system that may extract a plurality of planes included in an image generated by a camera, based on three-dimensional (3D) information of the image, and may estimating a pose of the camera based on a correspondence among the plurality of planes extracted, and an operating method thereof.

Abstract: Provided is an apparatus and method of multi-view rendering. A method of multi-view rendering includes rendering one or more 3D objects based on a first viewpoint, transforming pixel values of pixels of the first viewpoint, which are obtained by the rendering of the 3D objects, into pixel values of pixels based on a second viewpoint that is different from the first viewpoint, detecting an occlusion region that is a remaining region other than a region represented by the pixel values obtained by the transforming of the pixel values in an image based on the second viewpoint, and rendering the detected occlusion region based on the second viewpoint.

Abstract: A method of estimating an organ deformation model includes generating at least one 3D organ shape model of an organ of a subject based on at least one non-real time medical image representing a deformation state of the organ of the subject; generating a deformation space for the organ of the subject based on the at least one 3D organ shape model and prior knowledge regarding the organ; and estimating a 3D organ deformation model of the organ of the subject based on a real-time medical image of the organ of the subject and the deformation space.

Abstract: A graphic processing unit (GPU) and method for decompressing compressed 3-dimensional (3D) compressed data. The GPU may extract segment information by analyzing a compressed data header and decompress segments included in a bit stream based on the segment information.

Abstract: A method and apparatus for realistically expressing teeth performs rendering on a shape model of the teeth using a hybrid normal map based on materials and characteristics of the teeth in order to more realistically express the teeth.

Abstract: An apparatus and method for encoding a 3D mesh, and an apparatus and method for decoding the 3D mesh are disclosed. The 3D mesh encoding apparatus may determine mesh information including position information of each of vertices constituting the 3D mesh, and connectivity information among the vertices, based on a level, and may progressively encode the determined mesh information based on the level, thereby reducing an error with an original 3D object when compared to an equal transmission rating.

Abstract: An apparatus and method for encoding a 3D mesh, and an apparatus and method for decoding the 3D mesh are disclosed. The 3D mesh encoding apparatus may determine mesh information including position information of each of vertices constituting the 3D mesh, and connectivity information among the vertices, based on a level, and may progressively encode the determined mesh information based on the level, thereby reducing an error with an original 3D object when compared to an equal transmission rating.

Abstract: Disclosed is a method and apparatus for encoding a three-dimensional (3D) mesh. The method for encoding the 3D mesh includes determining a priority of a gate configuring a 3D mesh corresponding to a 3D object, removing vertices configuring the 3D mesh using the determined priority of the gate, and simplifying the 3D mesh.