Abstract: An apparatus is described which includes two or more colour displays arranged to provide piece-wise continuous illumination of a volume, and one or more cameras arranged to image the volume. The apparatus is configured to control the colour displays and the cameras to illuminate the volume with each of two or more illumination conditions. The apparatus is also configured to obtain two or more sets of images, which include sufficient information for calculation of a reflectance map and a photometric normal map of an object or subject positioned within the volume. Each set of images is obtained during illumination of the volume with one or more corresponding illumination conditions. When viewed from the volume, the apparatus only provides direct illumination of the volume from angles within a zone of a hemisphere, which is less than a hemisphere.

Abstract: A method is described which includes receiving a number of object images of an object. Each object image corresponds to a different view direction. First and second object images correspond to first and second directions. A mesh is determined corresponding to the target region of the object surface based on a first subset of the number of object images. Diffuse and specular maps are determined corresponding to the target region of the object surface based on processing a second subset of the object images using a deep learning neural network model trained to estimate diffuse and specular albedo components based on an input image. A tangent normal map is determined corresponding to the target region of the object surface based on high-pass filtering each object image of the second subset. The method also includes storing and/or outputting the mesh, the diffuse map, the specular map and the tangent normal map.

Abstract: A method is described which included receiving (S1) a number of object images (IMn) of an object (1). Each object image (IMn) corresponds to a different view direction (n). The object images include first (IM1) and second (IM2) object images corresponding to first (n1) and second (n2) directions. The method also includes determining (S2) a mesh (20) corresponding to the target region (5) of the object (1) surface (2) based on a first subset (MESH) of the number of object images (IMn) which includes two or more object images (IMn) of the number of object images (IMn). The method also includes determining (S3) diffuse (DFUV) and specular (SPUV) maps corresponding to the target region (5) of the object (1) surface (2) based on processing a second subset (REFLECT) of the object images (IMn) using a deep learning neural network model trained to estimate diffuse (DFn) and specular (SPn) albedo components based on an input image (IMn).

Abstract: Techniques are disclosed for capturing facial appearance properties. In some examples, a facial capture system includes light source(s) that produce linearly polarized light, at least one camera that is cross-polarized with respect to the polarization of light produced by the light source(s), and at least one other camera that is not cross-polarized with respect to the polarization of the light produced by the light source(s). Images captured by the cross-polarized camera(s) are used to determine facial appearance properties other than specular intensity, such as diffuse albedo, while images captured by the camera(s) that are not cross-polarized are used to determine facial appearance properties including specular intensity. In addition, a coarse-to-fine optimization procedure is disclosed for determining appearance and detailed geometry maps based on images captured by the cross-polarized camera(s) and the camera(s) that are not cross-polarized.

Abstract: A method is described which included receiving (S1) a number of object images (IMn) of an object (1). Each object image (IMn) corresponds to a different view direction (n). The object images include first (IM1) and second (IM2) object images corresponding to first (n1) and second (n2) directions. The method also includes determining (S2) a mesh (20) corresponding to the target region (5) of the object (1) surface (2) based on a first subset (MESH) of the number of object images (IMn) which includes two or more object images (IMn) of the number of object images (IMn). The method also includes determining (S3) diffuse (DFUV) and specular (SPUV) maps corresponding to the target region (5) of the object (1) surface (2) based on processing a second subset (REFLECT) of the object images (IMn) using a deep learning neural network model trained to estimate diffuse (DFn) and specular (SPn) albedo components based on an input image (IMn).

Abstract: An apparatus (1, 5, 6) is described which includes two or more colour displays (2) arranged to provide piece-wise continuous illumination of a volume. The apparatus (1, 5, 6) also includes one or more cameras (3). Each camera (3) is arranged to image the volume. The apparatus (1, 5, 6) is configured to control the two or more colour displays (2) and the one or more cameras (3) to illuminate the volume with each of two or more illumination conditions. The apparatus (1, 5, 6) is also configured to obtain two or more sets of images. Each set of images is obtained during illumination of the volume with one or more corresponding illumination conditions. The two or more sets of images include sufficient information for calculation of a reflectance map and a photometric normal map of an object or subject (4) positioned within the volume. When viewed from the volume, the apparatus (1, 5, 6) only provides direct illumination of the volume from angles within a zone of a hemisphere.

Abstract: A method of image processing includes receiving a first image of human skin. The first image corresponds to a first, uniform broadband illumination condition. The method also includes receiving a second image which has the same field of view and contents as the first image. The second image corresponds to a second illumination condition which comprises a uniform narrowband illumination condition. The method also includes processing the first and second images to fit parameter maps for a spectral bidirectional scattering surface reflectance distribution function skin model. The parameter maps include a modelled melanin concentration, a modelled haemoglobin concentration, a modelled melanin blend-type fraction and a modelled epidermal haemoglobin fraction. At least three of the parameter maps are independent.

Abstract: A method of image processing is described. The method comprises receiving a set of at least three images of an object including at least two linearly-polarized images and at least one color image, wherein the three images have the same view of the object and are acquired under the same illumination condition in which either diffuse polarization or specular polarization dominates in surface reflectance, and wherein a set of Stokes parameters s0, s1 and s2 is determinable from the at least three images.

Abstract: A method of image processing is described. The method comprises receiving a set of at least three images of an object including at least two linearly-polarized images and at least one color image, wherein the three images have the same view of the object and are acquired under the same illumination condition in which either diffuse polarization or specular polarization dominates in surface reflectance, and wherein a set of Stokes parameters s0, s1 and s2 is determinable from the at least three images.

Abstract: Techniques are disclosed for capturing facial appearance properties. In some examples, a facial capture system includes light source(s) that produce linearly polarized light, at least one camera that is cross-polarized with respect to the polarization of light produced by the light source(s), and at least one other camera that is not cross-polarized with respect to the polarization of the light produced by the light source(s). Images captured by the cross-polarized camera(s) are used to determine facial appearance properties other than specular intensity, such as diffuse albedo, while images captured by the camera(s) that are not cross-polarized are used to determine facial appearance properties including specular intensity. In addition, a coarse-to-fine optimization procedure is disclosed for determining appearance and detailed geometry maps based on images captured by the cross-polarized camera(s) and the camera(s) that are not cross-polarized.

Abstract: Techniques are disclosed for capturing facial appearance properties. In some examples, a facial capture system includes light source(s) that produce linearly polarized light, at least one camera that is cross-polarized with respect to the polarization of light produced by the light source(s), and at least one other camera that is not cross-polarized with respect to the polarization of the light produced by the light source(s). Images captured by the cross-polarized camera(s) are used to determine facial appearance properties other than specular intensity, such as diffuse albedo, while images captured by the camera(s) that are not cross-polarized are used to determine facial appearance properties including specular intensity. In addition, a coarse-to-fine optimization procedure is disclosed for determining appearance and detailed geometry maps based on images captured by the cross-polarized camera(s) and the camera(s) that are not cross-polarized.

Abstract: A method of image processing includes receiving a first image of human skin. The first image corresponds to a first, uniform broadband illumination condition. The method also includes receiving a second image which has the same field of view and contents as the first image. The second image corresponds to a second illumination condition which comprises a uniform narrowband illumination condition. The method also includes processing the first and second images to fit parameter maps for a spectral bidirectional scattering surface reflectance distribution function skin model. The parameter maps include a modelled melanin concentration, a modelled haemoglobin concentration, a modelled melanin blend-type fraction and a modelled epidermal haemoglobin fraction. At least three of the parameter maps are independent.

Abstract: Techniques are disclosed for capturing facial appearance properties. In some examples, a facial capture system includes light source(s) that produce linearly polarized light, at least one camera that is cross-polarized with respect to the polarization of light produced by the light source(s), and at least one other camera that is not cross-polarized with respect to the polarization of the light produced by the light source(s). Images captured by the cross-polarized camera(s) are used to determine facial appearance properties other than specular intensity, such as diffuse albedo, while images captured by the camera(s) that are not cross-polarized are used to determine facial appearance properties including specular intensity. In addition, a coarse-to-fine optimization procedure is disclosed for determining appearance and detailed geometry maps based on images captured by the cross-polarized camera(s) and the camera(s) that are not cross-polarized.

Abstract: A method of image processing includes receiving a set of images of an object, the set of images including images corresponding to binary spherical gradient illumination along first, second and third mutually orthogonal axes and images corresponding to complementary binary spherical gradient illumination along the first, second and third axes. The method also includes determining a specular reflectance map of the object and a diffuse reflectance map of the object based on the set of images, and/or determining a diffuse photometric normal map and a specular photometric normal map of the object based on the set of images.

Abstract: A method of image processing includes receiving a set of images of an object, the set of images including images corresponding to binary spherical gradient illumination along first, second and third mutually orthogonal axes and images corresponding to complementary binary spherical gradient illumination along the first, second and third axes. The method also includes determining a specular reflectance map of the object and a diffuse reflectance map of the object based on the set of images, and/or determining a diffuse photometric normal map and a specular photometric normal map of the object based on the set of images.

Abstract: Embodiments of the present disclosure techniques for modeling and capturing the dynamic appearance of skin. These techniques can couple dynamic reflectance parameters for skin (albedo and specular reflectance) with dynamic geometry. The disclosed techniques allow for capture and modeling of the dynamic appearance of skin for an actor. The techniques can re-render the actor's face accurately to accurately model the appearance of skin including the albedo of skin that can change primarily due to blood flow. The techniques can also re-render the actor's face accurately under multiple different lighting conditions.

Abstract: Embodiments of the present disclosure techniques for modeling and capturing the dynamic appearance of skin. These techniques can couple dynamic reflectance parameters for skin (albedo and specular reflectance) with dynamic geometry. The disclosed techniques allow for capture and modeling of the dynamic appearance of skin for an actor. The techniques can re-render the actor's face accurately to accurately model the appearance of skin including the albedo of skin that can change primarily due to blood flow. The techniques can also re-render the actor's face accurately under multiple different lighting conditions.

Abstract: An apparatus to measure surface orientation maps of an object may include a light source that is configured to illuminate the object with a controllable field of illumination. One or more cameras may be configured to capture at least one image of the object. A processor may be configured to process the image(s) to extract the reflectance properties of the object including an albedo, a reflection vector, a roughness, and/or anisotropy parameters of a specular reflectance lobe associated with the object. The controllable field of illumination may include limited-order Spherical Harmonics (SH) and Fourier Series (FS) illumination patterns with substantially similar polarization. The SH and FS illumination patterns are used with different light sources.

Abstract: Provided is a method and apparatus for realistically reproducing an eyeball that may verify and analyze a material property and a deformation property with respect to each of constituent portions of an eyeball and may render each of the constituent portions based on the analyzed priorities, thereby more realistically reproducing the eyeball.

Abstract: A multiview face capture system may acquire detailed facial geometry with high resolution diffuse and specular photometric information from multiple viewpoints. A lighting system may illuminate a face with polarized light from multiple directions. The light may be polarized substantially parallel to a reference axis during a parallel polarization mode of operation and substantially perpendicular to the reference axis during a perpendicular polarization mode of operation. Multiple cameras may each capture an image of the face along a materially different optical axis and have a linear polarizer configured to polarize light traveling along its optical axis in a direction that is substantially parallel to the reference axis. A controller may cause each of the cameras to capture an image of the face while the lighting system is in the parallel polarization mode of operation and again while the lighting system is in the perpendicular polarization mode of operation.