Patents by Inventor Minye WU

Minye WU has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20240104822
    Abstract: An image rendering system comprising a preprocessing unit coupled to a feature extract unit and a color rendering unit over a data bus. The preprocessing unit generates vector representations of spatial coordinates of sample points along camera rays corresponding to pixels of an image to be rendered. The feature extract unit generates a feature map of the image based on the vector representations, color and intensity values of the sample point through a first machine learning model. The color rendering unit renders the image based on the feature map through a second machine learning model. The first machine learning model is different from the second machine learning model.
    Type: Application
    Filed: December 7, 2023
    Publication date: March 28, 2024
    Applicant: SHANGHAITECH UNIVERSITY
    Inventors: Chaolin RAO, Minye WU, Xin LOU, Pingqiang ZHOU, Jingyi YU
  • Patent number: 11880964
    Abstract: A method of processing light field images for separating a transmitted layer from a reflection layer. The method comprises capturing a plurality of views at a plurality of viewpoints with different polarization angles; obtaining an initial disparity estimation for a first view using SIFT-flow, and warping the first view to a reference view; optimizing an objective function comprising a transmitted layer and a secondary layer using an Augmented Lagrange Multiplier (ALM) with Alternating Direction Minimizing (ADM) strategy; updating the disparity estimation for the first view; repeating the steps of optimizing the objective function and updating the disparity estimation until the change in the objective function between two consecutive iterations is below a threshold; and separating the transmitted layer and the secondary layer using the disparity estimation for the first view.
    Type: Grant
    Filed: October 19, 2020
    Date of Patent: January 23, 2024
    Assignee: SHANGHAITECH UNIVERSITY
    Inventors: Minye Wu, Zhiru Shi, Jingyi Yu
  • Patent number: 11880935
    Abstract: An image-based method of modeling and rendering a three-dimensional model of an object is provided. The method comprises: obtaining a three-dimensional point cloud at each frame of a synchronized, multi-view video of an object, wherein the video comprises a plurality of frames; extracting a feature descriptor for each point in the point cloud for the plurality of frames without storing the feature descriptor for each frame; producing a two-dimensional feature map for a target camera; and using an anti-aliased convolutional neural network to decode the feature map into an image and a foreground mask.
    Type: Grant
    Filed: September 23, 2022
    Date of Patent: January 23, 2024
    Assignee: SHANGHAITECH UNIVERSITY
    Inventors: Minye Wu, Jingyi Yu
  • Publication number: 20240013479
    Abstract: A computer-implemented method includes encoding a radiance field of an object onto a machine learning model; conducting, based on a set of training images of the object, a training process on the machine learning model to obtain a trained machine learning model, wherein the training process includes a first training process using a plurality of first test sample points followed by a second training process using a plurality of second test sample points located within a threshold distance from a surface region of the object; obtaining target view parameters indicating a view direction of the object; obtaining a plurality of rays associated with a target image of the object; obtaining render sample points on the plurality of rays associated with the target image; and rendering, by inputting the render sample points to the trained machine learning model, colors associated with the pixels of the target image.
    Type: Application
    Filed: September 19, 2023
    Publication date: January 11, 2024
    Applicant: SHANGHAITECH UNIVERSITY
    Inventors: Minye WU, Chaolin RAO, Xin LOU, Pingqiang ZHOU, Jingyi YU
  • Publication number: 20230360372
    Abstract: Systems, methods, and non-transitory computer-readable media are configured to obtain a set of content items to train a neural radiance field-based (NeRF-based) machine learning model for object recognition. Depth maps of objects depicted in the set of content items can be determined. A first set of training data comprising reconstructed content items depicting only the objects can be generated based on the depth maps. A second set of training data comprising one or more optimal training paths associated with the set of content items can be generated based on the depth maps. The one or more optimal training paths are generated based at least in part on a dissimilarity matrix associated with the set of content items. The NeRF-based machine learning model can be trained based on the first set of training data and the second set of training data.
    Type: Application
    Filed: July 19, 2023
    Publication date: November 9, 2023
    Applicant: SHANGHAITECH UNIVERSITY
    Inventors: Fuqiang ZHAO, Minye WU, Lan XU, Jingyi YU
  • Publication number: 20230027234
    Abstract: An image-based method of modeling and rendering a three-dimensional model of an object is provided. The method comprises: obtaining a three-dimensional point cloud at each frame of a synchronized, multi-view video of an object, wherein the video comprises a plurality of frames; extracting a feature descriptor for each point in the point cloud for the plurality of frames without storing the feature descriptor for each frame; producing a two-dimensional feature map for a target camera; and using an anti-aliased convolutional neural network to decode the feature map into an image and a foreground mask.
    Type: Application
    Filed: September 23, 2022
    Publication date: January 26, 2023
    Inventors: Minye WU, Jingyi YU
  • Patent number: 11410459
    Abstract: A method of detecting and recognizing faces using a light field camera array is provided. The method includes capturing multi-view color images using the light field camera array; obtaining a depth map; conducting light field rendering using a weight function comprising a depth component and a sematic component, where the weight function assigns a ray in the light field with a weight; and detecting and recognizing a face.
    Type: Grant
    Filed: June 5, 2020
    Date of Patent: August 9, 2022
    Assignee: ShanghaiTech University
    Inventors: Zhiru Shi, Minye Wu, Wenguang Ma, Jingyi Yu
  • Patent number: 11398019
    Abstract: A method of image processing for occlusion removal in images and videos captured by light field camera systems. The method comprises: capturing a plurality of camera views using a plurality of cameras; capturing a plurality of depth maps using a plurality of depth sensors; generating a depth map for each camera view; calculating a target view on a focal plane corresponding to a virtual camera; set a weighting function on the pixels on the camera views based on the depth map and a virtual distance; and blending the pixels in accordance with the weighting function to generate a refocused target view.
    Type: Grant
    Filed: October 16, 2020
    Date of Patent: July 26, 2022
    Assignee: ShanghaiTech University
    Inventors: Minye Wu, Zhiru Shi
  • Patent number: 11164358
    Abstract: The present invention discloses a method for real-time rendering of giga-pixel images. Image data are subject to offline pre-processing, and then are subject to data decoding and redirection through a decoding module. A corresponding scheduling strategy is determined according to different image inputs, and rendering is executed by a renderer. The present invention may realize the real-time rendering of a giga-pixel panoramic view in a conventional display device, to greatly reduce the resource allocated for rendering of giga-pixel images. The present invention may render an image originally requiring a 40G or more video memory capacity on a common video card with a 1G-4G video memory capacity.
    Type: Grant
    Filed: September 23, 2019
    Date of Patent: November 2, 2021
    Assignee: PLEX-VR DIGITAL TECHNOLOGY (SHANGHAI) CO., LTD.
    Inventors: Wentao Lyu, Yingliang Zhang, Anpei Chen, Minye Wu
  • Publication number: 20210090316
    Abstract: The present invention discloses a method for real-time rendering of giga-pixel images. Image data are subject to offline pre-processing, and then are subject to data decoding and redirection through a decoding module. A corresponding scheduling strategy is determined according to different image inputs, and rendering is executed by a renderer. The present invention may realize the real-time rendering of a giga-pixel panoramic view in a conventional display device, to greatly reduce the resource allocated for rendering of giga-pixel images. The present invention may render an image originally requiring a 40G or more video memory capacity on a common video card with a 1G-4G video memory capacity.
    Type: Application
    Filed: September 23, 2019
    Publication date: March 25, 2021
    Inventors: Wentao LYU, Yingliang ZHANG, Anpei CHEN, Minye WU
  • Publication number: 20210082096
    Abstract: A method of processing light field images for separating a transmitted layer from a reflection layer. The method comprises capturing a plurality of views at a plurality of viewpoints with different polarization angles; obtaining an initial disparity estimation for a first view using SIFT-flow, and warping the first view to a reference view; optimizing an objective function comprising a transmitted layer and a secondary layer using an Augmented Lagrange Multiplier (ALM) with Alternating Direction Minimizing (ADM) strategy; updating the disparity estimation for the first view; repeating the steps of optimizing the objective function and updating the disparity estimation until the change in the objective function between two consecutive iterations is below a threshold; and separating the transmitted layer and the secondary layer using the disparity estimation for the first view.
    Type: Application
    Filed: October 19, 2020
    Publication date: March 18, 2021
    Inventors: Minye WU, Zhiru SHI, Jingyi YU
  • Publication number: 20210042898
    Abstract: A method of image processing for occlusion removal in images and videos captured by light field camera systems. The method comprises: capturing a plurality of camera views using a plurality of cameras; capturing a plurality of depth maps using a plurality of depth sensors; generating a depth map for each camera view; calculating a target view on a focal plane corresponding to a virtual camera; set a weighting function on the pixels on the camera views based on the depth map and a virtual distance; and blending the pixels in accordance with the weighting function to generate a refocused target view.
    Type: Application
    Filed: October 16, 2020
    Publication date: February 11, 2021
    Inventors: Minye WU, Zhiru SHI
  • Publication number: 20200302155
    Abstract: A method of detecting and recognizing faces using a light field camera array is provided. The method includes capturing multi-view color images using the light field camera array; obtaining a depth map; conducting light field rendering using a weight function comprising a depth component and a sematic component, where the weight function assigns a ray in the light field with a weight; and detecting and recognizing a face.
    Type: Application
    Filed: June 5, 2020
    Publication date: September 24, 2020
    Inventors: Zhiru SHI, Minye WU, Wengguang MA, Jingyi YU