Abstract: Virtual reality-based apparatus that includes a memory device, a depth sensor and a modeling circuitry, captures a plurality of depth values of a first human subject from a single viewpoint using the depth sensor. The memory device stores a reference three dimensional (3D) human body model that comprises a mean body shape and a set of body shape variations. The modeling circuitry determines a first shape of the first human subject based on the plurality of depth values and generates a first deformed 3D human body model by deformation of the mean body shape. The modeling circuitry determines a first plurality of pose parameters for a first pose based on a plurality of rigid transformation matrices. The modeling circuitry generates a second deformed 3D human body model by deformation of a plurality of vertices and controls display of the second deformed 3D human body model as a reconstructed 3D model.

Abstract: A virtual reality-based apparatus and method to generate a 3D human face model includes, storage of a 3D face model that is an existing 3D face model or a mean-shape face model in at least a neutral expression. A point cloud of the face of the first user is generated based on the plurality of color images and depth information of the face of the first user. A first 3D face model of the first user having neutral expression is generated by a shape-based model-fitment on the stored 3D face model. A shape of the first 3D face model is refined based on a difference between the first 3D face model, the shape-based model-fitment, and the generated point cloud. The display of the refined first 3D face model is controlled to exhibit a minimum deviation from the shape and appearance of the face of the first user.

Abstract: An image-processing apparatus includes a first-type of sensor, a second-type of sensor, and a control circuitry. The control circuitry receives an input color image frame of a sequence of color image frames from the first-type of sensor and a corresponding input depth image from the second-type of sensor. The control circuitry generates a first foreground mask of an object-of-interest for the input color image frame. The control circuitry then detects a first set of pixels in the generated first foreground mask as misclassified image pixels. The control circuitry updates the background color image and the background depth image of the scene. The control circuitry precisely extracts the object-of-interest from the input color image frame, based on at least the updated background color image and the background depth image of the scene.

Abstract: An object segmentation system that includes a first type of sensor, a second type of sensor and a control circuitry. The first-type of sensor captures a sequence of color image frames of a scene. The second-type of sensor captures a depth image for each corresponding color image frame of the sequence of color image frames. The control circuitry generates a point cloud for an input color image frame. The control circuitry segments a foreground human object from a background of the input color image frame. The control circuitry detects a ground plane for the scene captured in the input color image frame. The control circuitry recovers a feet region in a defined region from a level of the detected ground plane. The control circuitry extracts the foreground human object with the recovered feet region from the background of the input color image frame, based on the detection of ground plane.

Abstract: An image processing apparatus includes a first type of sensor, a second type of sensor, and a control circuitry. The control circuitry receives a first foreground mask for an object-of-interest in a previous color image frame and further estimate a second foreground mask for the object-of-interest in a current color image frame at a first image resolution. The control circuitry further upsamples the second foreground mask to a second image resolution and select at least one region-of-interest in the current color image frame. The control circuitry then upscales the upsampled second foreground mask by an expectation parameter generated based on a kernel density-based estimation for the at least one pixel in a selected region-of-interest. The upscaled second foreground mask exhibits a minimum foreground mask error that is caused by upsampling with respect to the estimated second foreground mask.

Abstract: Apparatus ad method for texturing a 3D model by UV map in-painting, generates a first UV map from the 3D model and applies color values to the first UV map based on correspondence between a first color image and the first UV map mapped to the first region of the 3D model. A first point is detected on a boundary between a textured region and a first non-textured region within the textured region of the first UV map and a region-of-interest is detected around the first point on the boundary that is utilized to determine texture information of a specific patch of the textured region. A first portion of the first non-textured region of the detected region-of-interest is texturized to obtain a textured region-of-interest in the first UV map, which is utilized to generate a second UV map and texture the at least the first region of the 3D model.

Abstract: An image processing apparatus includes a first type of sensor, a second type of sensor, and a control circuitry. The control circuitry receives a first foreground mask for an object-of-interest in a previous color image frame and further estimate a second foreground mask for the object-of-interest in a current color image frame at a first image resolution. The control circuitry further upsamples the second foreground mask to a second image resolution and select at least one region-of-interest in the current color image frame. The control circuitry then upscales the upsampled second foreground mask by an expectation parameter generated based on a kernel density-based estimation for the at least one pixel in a selected region-of-interest. The upscaled second foreground mask exhibits a minimum foreground mask error that is caused by upsampling with respect to the estimated second foreground mask.

Abstract: An image-processing apparatus includes a first-type of sensor, a second-type of sensor, and a control circuitry. The control circuitry receives an input color image frame of a sequence of color image frames from the first-type of sensor and a corresponding input depth image from the second-type of sensor. The control circuitry generates a first foreground mask of an object-of-interest for the input color image frame. The control circuitry then detects a first set of pixels in the generated first foreground mask as misclassified image pixels. The control circuitry updates the background color image and the background depth image of the scene. The control circuitry precisely extracts the object-of-interest from the input color image frame, based on at least the updated background color image and the background depth image of the scene.

Abstract: An image-processing method and system are provided for detection of planar surfaces for use in scene modeling of a captured scene, which includes determination of three dimensional (3D) coordinates that corresponds to a plurality of scene points of the captured scene. The 3D coordinates are determined based on depth information associated with the plurality of scene points. A plurality of vectors are computed for the plurality of scene points, based on the determined 3D coordinates. The computed plurality of vectors associated with the captured scene are clustered into a first set of clusters based on orientation information associated with each of the computed plurality of vectors. Further, for each cluster of the first set of clusters, a second set of clusters, are determined based on distance information associated with the computed plurality of vectors, followed by detection of a plurality of planar surfaces in the captured scene.

Abstract: Various aspects of an image-processing apparatus and method for object boundary stabilization in an image of a sequence of image frames are disclosed. The image-processing apparatus includes an image processor that receives a depth image of a scene from a first-type of sensor and a color image of the scene from the second-type of sensor. The scene may comprise at least an object-of-interest. A first object mask of the object-of-interest is generated by a depth thresholding operation on the received depth image. Dangling-pixels artifact present on a first object boundary of the first object mask, are removed. The first object boundary is smoothened using a moving-template filter on the color image. A second object mask having a second object boundary is generated based on the smoothening of the first object boundary. The object-of-interest from the color image is extracted based on the generated second object mask.

Abstract: Various aspects of a video-processing system and method for object detection in a sequence of image frames are disclosed herein. The system includes an image-processing device configured to receive a first object template for an object in a first image frame that includes one or more objects. A plurality of object candidates that corresponds to the object for a second image frame are determined by use of the shape of the received first object template. One of the determined plurality of object candidates is selected as a second object template, based on one or more parameters. The received first object template is updated to the selected second object template to enable segmentation of the object in the second image frame and/or subsequent image frames.

Abstract: An image-processing device and method for foreground mask correction for object segmentation, includes receipt of a sequence of image frames. A first FG mask is estimated using depth information associated with the input image frame and by binarizing a difference between the input image frame and a background (BG) image of the input image frame. A first set of pixels with a first mask value to be updated to a second mask value is identified in a boundary region of the estimated first FG mask. A second FG mask is determined based on the identification of the first set of pixels in the boundary region of the estimated first FG mask.

Abstract: A system and method of operation of an image processing system includes: a get original image module for receiving an original image; a viewer detection module, coupled to the get original image module, for receiving a first position and a second position of a viewer; a crop image module, coupled to the position detector, for calculating a cropping offset for the original image based on the first position and the second position, and for calculating a cropped image by cropping the original image by the cropping offset; and a display image module, coupled to the crop image module, for displaying the cropped image on a display unit.

Abstract: Segmentation is the process of partitioning an image into regions under certain rules. One implementation is to separate human objects that appear in a sequence of images (video) from the background. The goal is to find humans and segment them out in real-time, fully automatic (with no user input), and the result is produced immediately after a new image is captured. Once the segmentation process is started, the method is constantly learning (or updating) the decision rule for segmenting out human objects from the background by itself. Moving object detection by a Kalman filter-based approach roughly detects the region where moving objects are present.

Abstract: Various aspects of an image processing system and method for object boundary smoothening for image segmentation, includes receipt of a user input to enable selection of a foreground object in an input color image. A frequency of occurrence of foreground pixels with respect to background pixels is determined for a plurality of pixels within a local pixel analysis window. The local pixel analysis window is positioned in a first region of the input color image to encompass at least a first pixel to be validated for a correct mask value. A first cost value and a second cost value is selected for the first pixel based on the determined frequency of occurrence of the foreground pixels. An object boundary is generated for a portion of the foreground object based on the selected first cost value and the second cost value for the first pixel.

Abstract: An image-processing method and system are provided for detection of planar surfaces for use in scene modeling of a captured scene, which includes determination of three dimensional (3D) coordinates that corresponds to a plurality of scene points of the captured scene. The 3D coordinates are determined based on depth information associated with the plurality of scene points. A plurality of vectors are computed for the plurality of scene points, based on the determined 3D coordinates. The computed plurality of vectors associated with the captured scene are clustered into a first set of clusters based on orientation information associated with each of the computed plurality of vectors. Further, for each cluster of the first set of clusters, a second set of clusters, are determined based on distance information associated with the computed plurality of vectors, followed by detection of a plurality of planar surfaces in the captured scene.

Abstract: A virtual reality-based apparatus and method to generate a 3D human face model includes, storage of a 3D face model that is an existing 3D face model or a mean-shape face model in at least a neutral expression. A point cloud of the face of the first user is generated based on the plurality of color images and depth information of the face of the first user. A first 3D face model of the first user having neutral expression is generated by a shape-based model-fitment on the stored 3D face model. A shape of the first 3D face model is refined based on a difference between the first 3D face model, the shape-based model-fitment, and the generated point cloud. The display of the refined first 3D face model is controlled to exhibit a minimum deviation from the shape and appearance of the face of the first user.

Abstract: Apparatus ad method for texturing a 3D model by UV map in-painting, generates a first UV map from the 3D model and applies color values to the first UV map based on correspondence between a first color image and the first UV map mapped to the first region of the 3D model. A first point is detected on a boundary between a textured region and a first non-textured region within the textured region of the first UV map and a region-of-interest is detected around the first point on the boundary that is utilized to determine texture information of a specific patch of the textured region. A first portion of the first non-textured region of the detected region-of-interest is texturized to obtain a textured region-of-interest in the first UV map, which is utilized to generate a second UV map and texture the at least the first region of the 3D model.

Abstract: An apparatus and method to generate a realistic rigged three-dimensional (3D) model animation for view-point transform, includes storage of a first 3D model, which further includes a first hierarchal set of interconnected bones defined in a first set of bone orientations. Bone structure information of a second hierarchal set of interconnected bones of an object is received from a motion-sensing device. The first set of bone orientations is modified to a second set of bone orientations based on the bone structure information. A second 3D model is generated by transformation of a size of one or more bones in the first hierarchal set of interconnected bones based on the bone structure information. The second 3D model is animated on a display device in accordance with the second set of bone orientations and the transformed first hierarchal set of interconnected bones.

Abstract: A three-dimensional (3D) animation apparatus and method for roll rotation determination for rigged 3D model animation, includes display of a 3D model on a display device, which allows for selection of a first set of three feature points. The selection is executed on a first surface representation of a first facial portion of the displayed 3D model. A second surface representation of a second facial portion of an object is received from a motion-sensing device. A second set of three feature points is identified on the received second surface representation. A roll rotation of a head portion of the 3D model is determined based on the first set of three feature points and the second set of three feature points. The movement of the head portion of the 3D model rendered on the display device is controlled in accordance with the determined roll rotation.