Patents by Inventor Xiaoyong Shen

Xiaoyong Shen has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11907637
    Abstract: The present disclosure provides an image processing method and apparatus, and an electronic device, and relates to the field of artificial intelligence (AI). The method includes: obtaining an input image and extracting region features of image regions in the input image to obtain a first image feature; processing pixels in the first image feature according to a predetermined rule, and determining a second image feature according to the processed pixels; and determining, based on the second image feature and at least one word vector that is determined for the input image, word vectors corresponding to the region features of the image regions in the first image feature at different moments, predicting positions of the word vectors in a text description, and forming the text description corresponding to the input image according to the word vectors and the positions.
    Type: Grant
    Filed: November 2, 2021
    Date of Patent: February 20, 2024
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Lei Ke, Wenjie Pei, Ruiyu Li, Xiaoyong Shen, Yuwing Tai, Jiaya Jia
  • Patent number: 11810377
    Abstract: This application relates to a point cloud segmentation method, a computer-readable storage medium, and a computer device. The method includes encoding a to-be-processed point cloud to obtain a shared feature, the shared feature referring to a feature shared at a semantic level and at an instance level; decoding the shared feature to obtain a semantic feature and an instance feature respectively; adapting the semantic feature to an instance feature space and fusing the semantic feature with the instance feature, to obtain a semantic-fused instance feature of the point cloud, the semantic-fused instance feature representing an instance feature fused with the semantic feature; dividing the semantic-fused instance feature of the point cloud, to obtain a semantic-fused instance feature of each point in the point cloud; and determining an instance category to which each point belongs according to the semantic-fused instance feature of each point.
    Type: Grant
    Filed: July 22, 2021
    Date of Patent: November 7, 2023
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Shu Liu, Xinlong Wang, Xiaoyong Shen, Jiaya Jia, Yuwing Tai
  • Patent number: 11790497
    Abstract: Embodiments of this disclosure include an image enhancement method and apparatus. The image enhancement may include obtaining an original image and performing synthesis processing on features of the original image to obtain a first illumination map corresponding to the original image. A resolution of the first illumination map may be lower than a resolution of the original image. The image enhancement may further include obtaining, based on the first illumination map, a mapping relationship between an image to an illumination map and performing mapping processing on the original image based on the mapping relationship to obtain a second illumination map. A resolution of the second illumination map may be equal to the resolution of the original image. The image enhancement may further include performing image enhancement processing on the original image according to the second illumination map to obtain a target image.
    Type: Grant
    Filed: May 19, 2021
    Date of Patent: October 17, 2023
    Assignee: Tencent Technology (Shenzhen) Company Limited
    Inventors: Ruixing Wang, Xin Tao, Xiaoyong Shen, Jiaya Jia, Yuwing Tai
  • Patent number: 11749029
    Abstract: A gesture language recognition method is provided. In the method, a first video is obtained. Gesture features are extracted from frames of images in the first video. Gesture change features are extracted from the frames of the images in the first video. Gesture language word information is extracted from fused features that are determined based on the gesture features and the gesture change features. The gesture language word information is combined into a gesture language sentence according to context information corresponding to the gesture language word information.
    Type: Grant
    Filed: August 31, 2021
    Date of Patent: September 5, 2023
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Zhaoyang Yang, Xiaoyong Shen, Yuwing Tai, Jiaya Jia
  • Patent number: 11741581
    Abstract: Embodiments of this application disclose a training method using image processing model for processing blurry images. The method includes obtaining a sample pair comprising a clear image and a corresponding blurry image; the sharpness of the clear image being greater than a preset threshold, the sharpness of the blurry image being less than the preset threshold; activating the image processing model to perform sharpness restoration on the blurry image to obtain a restored image; and updating network parameters of a first network and network parameters of a second network in the image processing model according to the restored image and the clear image to obtain a trained image processing model; the network parameters of the first network and the network parameters of the second network meeting a selective sharing condition indicating whether the network parameters between the first network and the second network are shared or independent.
    Type: Grant
    Filed: May 27, 2021
    Date of Patent: August 29, 2023
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Hongyun Gao, Xin Tao, Jiaya Jia, Yuwing Tai, Xiaoyong Shen
  • Patent number: 11743551
    Abstract: A video caption generating method is provided to a computer device. The method includes encoding a target video by using an encoder of a video caption generating model, to obtain a target visual feature of the target video, decoding the target visual feature by using a basic decoder of the video caption generating model, to obtain a first selection probability corresponding to a candidate word, decoding the target visual feature by using an auxiliary decoder of the video caption generating model, to obtain a second selection probability corresponding to the candidate word, a memory structure of the auxiliary decoder including reference visual context information corresponding to the candidate word, determining a decoded word in the candidate word according to the first selection probability and the second selection probability, and generating a video caption according to decoded word.
    Type: Grant
    Filed: May 24, 2021
    Date of Patent: August 29, 2023
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Wenjie Pei, Jiyuan Zhang, Lei Ke, Yuwing Tai, Xiaoyong Shen, Jiaya Jia, Xiangrong Wang
  • Patent number: 11610321
    Abstract: This application provides a target tracking method, including: obtaining a plurality of consecutive picture frames of a target video, and setting a tracked target region of an nth picture frame; performing a feature extraction operation on the tracked target region of the nth picture frame, to obtain a feature map of the tracked target region; calculating a weighted filter corresponding to the input feature map according to a correlation filter algorithm and a mean pooling constraint condition; calculating an output response of an (n+1)th picture frame by using the weighted filter and an input feature map of the (n+1)th picture frame in the plurality of consecutive picture frames, and determining a tracked target region of the (n+1)th picture frame according to the output response of the (n+1)th picture frame, until tracked target regions of all the consecutive picture frames are obtained. This application improves precision and effectiveness of target tracking.
    Type: Grant
    Filed: July 1, 2021
    Date of Patent: March 21, 2023
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Chong Sun, Yuxuan Sun, Huchuan Lu, Xiaoyong Shen, Yuwing Tai, Jiaya Jia
  • Patent number: 11450080
    Abstract: An image processing method and apparatus, and a storage medium are provided. The method includes: detecting a target object in a current video frame of a target video stream, to obtain a current detection region for the target object; adjusting the current detection region according to a historic detection region corresponding to the target object in a historic video frame of the target video stream, to obtain a determined current detection region; performing key point positioning on the target object based on the determined current detection region, to obtain a first set of key points; and performing stabilization on locations of the key points in the first set according to locations of key points in a second set corresponding to the target object in the historic video frame, to obtain current locations of a set of key points of the target object in the current video frame.
    Type: Grant
    Filed: November 3, 2020
    Date of Patent: September 20, 2022
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Weilong Peng, Xiaoyong Shen, Yilun Chen, Yanan Sun
  • Patent number: 11302064
    Abstract: Embodiments of this application disclose a method and an apparatus for reconstructing a three-dimensional model of a human body, and a storage medium. The method includes obtaining a two-dimensional image of a human body part to be reconstructed; mapping the two-dimensional image to a voxel space, to obtain a voxelized result of the two-dimensional image; obtaining three-dimensional morphable model parameters corresponding to the two-dimensional image based on calculating the voxelized result using a preset three-dimensional parameter regression network; and reconstructing a three-dimensional model of the human body part based on the three-dimensional morphable model parameters.
    Type: Grant
    Filed: September 29, 2020
    Date of Patent: April 12, 2022
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Chen Li, Hongwei Yi, Qiong Cao, Xiaoyong Shen, Yuwing Tai, Jiaya Jia
  • Patent number: 11275931
    Abstract: A human pose prediction method is provided for an electronic device. The method includes using a basic neural network based on image-feature-based prediction to perform prediction on an inputted target image, to obtain an initial prediction map of a human key-point; inputting the initial prediction map of the human key-point and a human structure diagram into a pose graph neural network based on spatial information mining, each node in the human structure diagram corresponding to a human joint respectively, and each edge connecting adjacent human joints; using the pose graph neural network to initialize the human structure diagram by using the initial prediction map of the human key-point, to obtain an initialized human structure diagram; and using the pose graph neural network to perform iterative prediction on the initialized human structure diagram, to obtain a final prediction map, the final prediction map indicating a predicted human pose.
    Type: Grant
    Filed: June 17, 2020
    Date of Patent: March 15, 2022
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Hong Zhang, Xiaoyong Shen, Jiaya Jia
  • Publication number: 20220058332
    Abstract: The present disclosure provides an image processing method and apparatus, and an electronic device, and relates to the field of artificial intelligence (AI). The method includes: obtaining an input image and extracting region features of image regions in the input image to obtain a first image feature; processing pixels in the first image feature according to a predetermined rule, and determining a second image feature according to the processed pixels; and determining, based on the second image feature and at least one word vector that is determined for the input image, word vectors corresponding to the region features of the image regions in the first image feature at different moments, predicting positions of the word vectors in a text description, and forming the text description corresponding to the input image according to the word vectors and the positions.
    Type: Application
    Filed: November 2, 2021
    Publication date: February 24, 2022
    Applicant: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Lei KE, Wenjie PEI, Ruiyu LI, Xiaoyong SHEN, Yuwing TAI, Jiaya JIA
  • Patent number: 11257299
    Abstract: Embodiments of the present disclosure disclose a face model processing method performed at an electronic device. The method includes the following steps: obtaining a three-dimensional face model corresponding to a user picture, and selecting a sample oral cavity model in an oral cavity model library for the three-dimensional face model; registering the sample oral cavity model into the three-dimensional face model by using an oral cavity position parameter in the three-dimensional face model; performing form adjustment on an oral cavity form of the registered sample oral cavity model by using an expression parameter of the three-dimensional face model to generate a target oral cavity model; and generating, based on the three-dimensional face model and the target oral cavity model, a three-dimensional face expression model corresponding to the user picture.
    Type: Grant
    Filed: June 10, 2020
    Date of Patent: February 22, 2022
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Weilong Peng, Chen Li, Yuwing Tai, Xiaoyong Shen, Jiaya Jia
  • Publication number: 20220004794
    Abstract: A computer device extracts an image feature of an image that includes one or more characters to be recognized. The image feature includes a plurality of image feature vectors. The device uses an attention mechanism to compute and output attention weight values corresponding to the target number of characters, based on the image feature vectors, through parallel computing. Each of the attention weight values corresponds to one or more respective characters and represents an importance of the plurality of image feature vectors for the respective characters. The device obtains at least one character according to the plurality of image feature vectors and the target number of attention weight values. Therefore, in a character recognition process, with recognition based on the foregoing attention mechanism, a character in any shape can be effectively recognized by using a simple procedure, thereby avoiding a cyclic operation process and greatly improving operation efficiency.
    Type: Application
    Filed: September 15, 2021
    Publication date: January 6, 2022
    Inventors: Pengyuan LV, Zhicheng YANG, Xinhang LENG, Ruiyu LI, Xiaoyong SHEN, Yuwing TAI, Jiaya JIA
  • Publication number: 20210390289
    Abstract: A gesture language recognition method is provided. In the method, a first video is obtained. Gesture features are extracted from frames of images in the first video. Gesture change features are extracted from the frames of the images in the first video. Gesture language word information is extracted from fused features that are determined based on the gesture features and the gesture change features. The gesture language word information is combined into a gesture language sentence according to context information corresponding to the gesture language word information.
    Type: Application
    Filed: August 31, 2021
    Publication date: December 16, 2021
    Applicant: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Zhaoyang YANG, Xiaoyong SHEN, Yuwing TAI, Jiaya JIA
  • Publication number: 20210390370
    Abstract: A data processing method is provided. In the data processing method, target sequence data is obtained. The target sequence data includes N groups of data sorted in chronological order. Processing is performed, according to an ith group of data in the N groups of data, processing results of a target neural network model for the ith group of data, and a processing result of the target neural network model for a jth piece of data in an (i+1)th group of data, a (j+1)th piece of data in the (i+1)th group of data by using the target neural network model, to obtain a processing result of the target neural network model for the (j+1)th piece of data in the (i+1)th group of data, i being greater than or equal to 1 and less than N, and j being greater than or equal to 1 and less than Q.
    Type: Application
    Filed: August 27, 2021
    Publication date: December 16, 2021
    Applicant: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Canmiao FU, Qiong CAO, Wenjie PEI, Xiaoyong SHEN, Yuwing TAI, Jiaya JIA
  • Publication number: 20210350183
    Abstract: This application relates to a point cloud segmentation method, a computer-readable storage medium, and a computer device. The method includes encoding a to-be-processed point cloud to obtain a shared feature, the shared feature referring to a feature shared at a semantic level and at an instance level; decoding the shared feature to obtain a semantic feature and an instance feature respectively; adapting the semantic feature to an instance feature space and fusing the semantic feature with the instance feature, to obtain a semantic-fused instance feature of the point cloud, the semantic-fused instance feature representing an instance feature fused with the semantic feature; dividing the semantic-fused instance feature of the point cloud, to obtain a semantic-fused instance feature of each point in the point cloud; and determining an instance category to which each point belongs according to the semantic-fused instance feature of each point.
    Type: Application
    Filed: July 22, 2021
    Publication date: November 11, 2021
    Inventors: Shu LIU, Xinlong WANG, Xiaoyong SHEN, Jiaya JIA, Yuwing TAI
  • Publication number: 20210334942
    Abstract: This application discloses an image processing method, device, and apparatus, and a storage medium. The method is performed by a computing device, the method including: receiving an input image; determining a context feature of the input image; determining a first feature set and a second feature set according to the context feature and based on a size of a target image and a location of the input image in the target image; adjusting the second feature set according to a first feature statistic of the first feature set, to obtain an adjusted second feature set; and generating the target image based on the adjusted second feature set and the first feature set.
    Type: Application
    Filed: July 9, 2021
    Publication date: October 28, 2021
    Inventors: Yi WANG, Xin TAO, Jiaya JIA, Yuwing TAI, Xiaoyong SHEN
  • Publication number: 20210327076
    Abstract: This application provides a target tracking method, including: obtaining a plurality of consecutive picture frames of a target video, and setting a tracked target region of an nth picture frame; performing a feature extraction operation on the tracked target region of the nth picture frame, to obtain a feature map of the tracked target region; calculating a weighted filter corresponding to the input feature map according to a correlation filter algorithm and a mean pooling constraint condition; calculating an output response of an (n+1)th picture frame by using the weighted filter and an input feature map of the (n+1)th picture frame in the plurality of consecutive picture frames, and determining a tracked target region of the (n+1)th picture frame according to the output response of the (n+1)th picture frame, until tracked target regions of all the consecutive picture frames are obtained. This application improves precision and effectiveness of target tracking.
    Type: Application
    Filed: July 1, 2021
    Publication date: October 21, 2021
    Inventors: Chong SUN, Yuxuan SUN, Huchuan LU, Xiaoyong SHEN, Yuwing TAI, Jiaya JIA
  • Publication number: 20210287047
    Abstract: Embodiments of this application disclose a training method using image processing model for processing blurry images. The method includes obtaining a sample pair comprising a clear image and a corresponding blurry image; the sharpness of the clear image being greater than a preset threshold, the sharpness of the blurry image being less than the preset threshold; activating the image processing model to perform sharpness restoration on the blurry image to obtain a restored image; and updating network parameters of a first network and network parameters of a second network in the image processing model according to the restored image and the clear image to obtain a trained image processing model; the network parameters of the first network and the network parameters of the second network meeting a selective sharing condition indicating whether the network parameters between the first network and the second network are shared or independent.
    Type: Application
    Filed: May 27, 2021
    Publication date: September 16, 2021
    Inventors: Hongyun GAO, Xin TAO, Jiaya JIA, Yuwing TAI, Xiaoyong SHEN
  • Publication number: 20210281774
    Abstract: A video caption generating method is provided to a computer device. The method includes encoding a target video by using an encoder of a video caption generating model, to obtain a target visual feature of the target video, decoding the target visual feature by using a basic decoder of the video caption generating model, to obtain a first selection probability corresponding to a candidate word, decoding the target visual feature by using an auxiliary decoder of the video caption generating model, to obtain a second selection probability corresponding to the candidate word, a memory structure of the auxiliary decoder including reference visual context information corresponding to the candidate word, determining a decoded word in the candidate word according to the first selection probability and the second selection probability, and generating a video caption according to decoded word.
    Type: Application
    Filed: May 24, 2021
    Publication date: September 9, 2021
    Inventors: Wenjie PEI, Jiyuan ZHANG, Lei KE, Yuwing TAI, Xiaoyong SHEN, Jiaya JIA, Xiangrong WANG