Patents by Inventor Xiaoou Tang

Xiaoou Tang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11301719
    Abstract: A semantic segmentation model training method includes: performing, by a semantic segmentation model, image semantic segmentation on at least one unlabeled image to obtain a preliminary semantic segmentation result as the category of the unlabeled image; obtaining, by a convolutional neural network based on the category of the at least one unlabeled image and the category of at least one labeled image, sub-images respectively corresponding to the at least two images and features corresponding to the sub-images, where the at least two images comprise the at least one unlabeled image and the at least one labeled image, and the at least two sub-images carry the categories of the corresponding images; and training the semantic segmentation model on the basis of the categories of the at least two sub-images and feature distances between the at least two sub-images.
    Type: Grant
    Filed: December 25, 2019
    Date of Patent: April 12, 2022
    Assignee: BEIJING SENSETIME TECHNOLOGY DEVELOPMENT CO., LTD.
    Inventors: Xiaohang Zhan, Ziwei Liu, Ping Luo, Chen Change Loy, Xiaoou Tang
  • Patent number: 11222211
    Abstract: A method and an apparatus for segmenting a video object, an electronic device, a storage medium, and a program include: performing, among at least some frames of a video, inter-frame transfer of an object segmentation result of a reference frame in sequence from the reference frame, to obtain an object segmentation result of at least one other frame among the at least some frames; determining other frames having lost objects with respect to the object segmentation result of the reference frame among the at least some frames; using the determined other frames as target frames to segment the lost objects, so as to update the object segmentation results of the target frames; and transferring the updated object segmentation results of the target frames to the at least one other frame in the video in sequence. The accuracy of video object segmentation results can therefore be improved.
    Type: Grant
    Filed: December 29, 2018
    Date of Patent: January 11, 2022
    Assignee: BEIJING SENSETIME TECHNOLOGY DEVELOPMENT CO., LTD
    Inventors: Xiaoxiao Li, Yuankai Qi, Zhe Wang, Kai Chen, Ziwei Liu, Jianping Shi, Ping Luo, Chen Change Loy, Xiaoou Tang
  • Patent number: 11144800
    Abstract: An image disambiguation method includes: performing image feature extraction and semantic recognition on at least two images in an image set including similar targets to obtain N K-dimensional semantic feature probability vectors, where the image set includes N images, N and K are both positive integers, and N is greater than or equal to 2; determining a differential feature combination according to the N K-dimensional semantic feature probability vectors, the differential feature combination indicating a difference between the similar targets in the at least two images in the image set; and generating a natural language for representing or prompting the difference between the similar targets in the at least two images in the image set according to the differential feature combination and image features of the at least two images in the image set.
    Type: Grant
    Filed: October 24, 2019
    Date of Patent: October 12, 2021
    Assignee: BEIJING SENSETIME TECHNOLOGY DEVELOPMENT CO., LTD.
    Inventors: Xiaoou Tang, Yining Li, Chen Huang, Chen Change Loy
  • Publication number: 20210295473
    Abstract: A method for image restoration, an electronic device and a computer storage medium are provided. The method includes that: region division is performed on an acquired image to obtain more than one sub-image; each sub-image is input into multiple paths of neural network and restored by using a restoration network determined for each sub-image; a restored image of each sub-image is output and obtained, so as to obtain a restored image of the acquired image.
    Type: Application
    Filed: June 8, 2021
    Publication date: September 23, 2021
    Applicant: BEIJING SENSETIME TECHNOLOGY DEVELOPMENT CO., LTD.
    Inventors: Ke YU, Xintao WANG, Chao DONG, Xiaoou TANG, Chen Change LOY
  • Patent number: 11120078
    Abstract: The present disclosure relates to a method and device for video processing, an electronic device, and a storage medium. The method comprises: determining, on the basis of paragraph information of a query text paragraph and video information of multiple videos in a video library, preselected videos associated with the query text paragraph in the multiple videos; and determining a target video in the preselected videos on the basis of video frame information of the preselected videos and of sentence information of the query text paragraph. The method for video processing of the embodiments of the present disclosure indexes videos by means of the relevance between the videos and the query text paragraph, allows the pinpointing of the target video, avoids search result redundancy, allows the processing of the query text paragraph in a natural language form, and is not limited by the inherent contents of content labels.
    Type: Grant
    Filed: August 6, 2019
    Date of Patent: September 14, 2021
    Assignee: BEIJING SENSETIME TECHNOLOGY DEVELOPMENT CO., LTD.
    Inventors: Xiaoou Tang, Dian Shao, Yu Xiong, Yue Zhao, Qingqiu Huang, Yu Qiao, Dahua Lin
  • Publication number: 20210241470
    Abstract: An image processing method includes: acquiring an image frame sequence, including a to-be-processed image frame and one or more image frames adjacent thereto, and performing image alignment on the to-be-processed image frame and each of image frames in the image frame sequence to obtain multiple pieces of aligned feature data; determining, based on the multiple pieces of alignment feature data, multiple similarity features each between a respective one of the multiple pieces of aligned feature data and aligned feature data corresponding to the to-be-processed image frame, and determining weight information of each of multiple pieces of aligned feature data based on the multiple similarity features; and fusing the multiple pieces of aligned feature data according to the weight information to obtain fusion information of the image frame sequence, the fusion information being configured to acquire a processed image frame corresponding to the to-be-processed image frame.
    Type: Application
    Filed: April 21, 2021
    Publication date: August 5, 2021
    Inventors: Xiaoou TANG, Xintao WANG, Zhuojie CHEN, Ke YU, Chao DONG, Chen Change LOY
  • Patent number: 11080569
    Abstract: A method and device for image recognition and a storage medium are provided. The method includes: a target image is acquired; feature extraction processing is performed on the target image through convolutional layers in a neural network model to obtain feature maps, and Instance Normalization (IN) and Batch Normalization (BN) processing is performed on the feature maps to obtain a recognition result of the target image; and the recognition result of the target image is output.
    Type: Grant
    Filed: October 16, 2020
    Date of Patent: August 3, 2021
    Assignee: BEIJING SENSETIME TECHNOLOGY DEVELOPMENT CO., LTD.
    Inventors: Xingang Pan, Jianping Shi, Ping Luo, Xiaoou Tang
  • Publication number: 20210049403
    Abstract: A method for image processing, an electronic device, and a storage medium are provided. The method includes the following. For each processing method in a preset set of processing methods, a first feature parameter and a second feature parameter are determined according to image data to-be-processed, where the preset set includes at least two processing methods selected from whitening methods and/or normalization methods, and the image data to-be-processed includes at least one image data. A first weighted average of the first feature parameters is determined according to a weight coefficient of each first feature parameter, and a second weighted average of the second feature parameters is determined according to a weight coefficient of each second feature parameter. The image data to-be-processed is whitened according to the first weighted average and the second weighted average.
    Type: Application
    Filed: November 2, 2020
    Publication date: February 18, 2021
    Applicant: Beijing Sensetime Technology Development Co., Ltd.
    Inventors: Xingang PAN, Ping LUO, Jianping SHI, Xiaoou TANG
  • Patent number: 10915741
    Abstract: Time domain action detecting methods and systems, electronic devices, and computer storage medium are provided. The method includes: obtaining a time domain interval in a video with an action instance and at least one adjacent segment in the time domain interval; separately extracting action features of at least two video segments in candidate segments, where the candidate segments comprises video segment corresponding to the time domain interval and adjacent segments thereof; pooling the action features of the at least two video segments in the candidate segments, to obtain a global feature of the video segment corresponding to the time domain interval; and determining, based on the global feature, an action integrity score of the video segment corresponding to the time domain interval. The embodiments of the present disclosure benefit accurately determining whether a time domain interval comprises an integral action instance, and improve the accuracy rate of action integrity identification.
    Type: Grant
    Filed: December 28, 2018
    Date of Patent: February 9, 2021
    Assignee: Beijing SenseTime Technology Development Co., Ltd
    Inventors: Xiaoou Tang, Yuanjun Xiong, Yue Zhao, Limin Wang, Zhirong Wu, Dahua Lin
  • Publication number: 20210034913
    Abstract: A method and device for image recognition and a storage medium are provided. The method includes: a target image is acquired; feature extraction processing is performed on the target image through convolutional layers in a neural network model to obtain feature maps, and Instance Normalization (IN) and Batch Normalization (BN) processing is performed on the feature maps to obtain a recognition result of the target image; and the recognition result of the target image is output.
    Type: Application
    Filed: October 16, 2020
    Publication date: February 4, 2021
    Inventors: Xingang PAN, Jianping SHI, Ping LUO, Xiaoou TANG
  • Publication number: 20200394216
    Abstract: The present disclosure relates to a method and device for video processing, an electronic device, and a storage medium. The method comprises: determining, on the basis of paragraph information of a query text paragraph and video information of multiple videos in a video library, preselected videos associated with the query text paragraph in the multiple videos; and determining a target video in the preselected videos on the basis of video frame information of the preselected videos and of sentence information of the query text paragraph. The method for video processing of the embodiments of the present disclosure indexes videos by means of the relevance between the videos and the query text paragraph, allows the pinpointing of the target video, avoids search result redundancy, allows the processing of the query text paragraph in a natural language form, and is not limited by the inherent contents of content labels.
    Type: Application
    Filed: August 6, 2019
    Publication date: December 17, 2020
    Inventors: Xiaoou TANG, Dian SHAO, Yu XIONG, Yue ZHAO, Qingqiu HUANG, Yu Qiao, Dahua LIN
  • Patent number: 10853916
    Abstract: A method and system for processing an image operates by: filtering a first real image to obtain a first feature map therefor with performances of image features improved; upscaling the obtained first feature map to improve a resolution thereof, the feature map with improved resolution forming a second feature map; and constructing, from the second feature map, a second real image having enhanced performances and a higher resolution than that of the first real image.
    Type: Grant
    Filed: June 20, 2018
    Date of Patent: December 1, 2020
    Assignee: SENSETIME GROUP LIMITED
    Inventors: Xiaoou Tang, Chao Dong, Tak Wai Hui, Chen Change Loy
  • Patent number: 10699170
    Abstract: Disclosed is a method for generating a semantic image labeling model, comprising: forming a first CNN and a second CNN, respectively; randomly initializing the first CNN; inputting a raw image and predetermined label ground truth annotations to the first CNN to iteratively update weights thereof so that a category label probability for the image, which is output from the first CNN, approaches the predetermined label ground truth annotations; randomly initializing the second CNN; inputting the category label probability to the second CNN to correct the input category label probability so as to determine classification errors of the category label probabilities; updating the second CNN by back-propagating the classification errors; concatenating the updated first and second CNNs; classifying each pixel in the raw image into one of general object categories; and back-propagating classification errors through the concatenated CNN to update weights thereof until the classification errors less than a predetermined
    Type: Grant
    Filed: January 8, 2018
    Date of Patent: June 30, 2020
    Assignee: Beijing SenseTime Technology Development Co., Ltd.
    Inventors: Xiaoou Tang, Ziwei Liu, Xiaoxiao Li, Ping Luo, Chen Change Loy
  • Publication number: 20200134375
    Abstract: A semantic segmentation model training method includes: performing, by a semantic segmentation model, image semantic segmentation on at least one unlabeled image to obtain a preliminary semantic segmentation result as the category of the unlabeled image; obtaining, by a convolutional neural network based on the category of the at least one unlabeled image and the category of at least one labeled image, sub-images respectively corresponding to the at least two images and features corresponding to the sub-images, where the at least two images comprise the at least one unlabeled image and the at least one labeled image, and the at least two sub-images carry the categories of the corresponding images; and training the semantic segmentation model on the basis of the categories of the at least two sub-images and feature distances between the at least two sub-images.
    Type: Application
    Filed: December 25, 2019
    Publication date: April 30, 2020
    Applicant: BEIJING SENSETIME TECHNOLOGY DEVELOPMENT CO., LTD.
    Inventors: Xiaohang ZHAN, Ziwei LIU, Ping LUO, Chen Change LOY, Xiaoou TANG
  • Patent number: 10579876
    Abstract: A method for identifying social relation of persons in an image, including: generating face regions for faces of the persons in the image; determining at least one spatial cue for each of the faces; extracting features related to social relation for each face from the face regions; determining a shared facial feature from the extracted features and the determined spatial cue, the determined feature being shared by multiple the social relation inferences; and predicting the social relation of the persons from the shared facial feature.
    Type: Grant
    Filed: December 29, 2017
    Date of Patent: March 3, 2020
    Assignee: BEIJING SENSETIME TECHNOLOGY DEVELOPMENT CO., LTD
    Inventors: Xiaoou Tang, Zhanpeng Zhang, Ping Luo, Chen Change Loy
  • Publication number: 20200057925
    Abstract: An image disambiguation method includes: performing image feature extraction and semantic recognition on at least two images in an image set including similar targets to obtain N K-dimensional semantic feature probability vectors, where the image set includes N images, N and K are both positive integers, and N is greater than or equal to 2; determining a differential feature combination according to the N K-dimensional semantic feature probability vectors, the differential feature combination indicating a difference between the similar targets in the at least two images in the image set; and generating a natural language for representing or prompting the difference between the similar targets in the at least two images in the image set according to the differential feature combination and image features of the at least two images in the image set.
    Type: Application
    Filed: October 24, 2019
    Publication date: February 20, 2020
    Applicant: BEIJING SENSETIME TECHNOLOGY DEVELOPMENT CO., LTD.
    Inventors: Xiaoou TANG, Yining Li, Chen Huang, Chen Change Loy
  • Patent number: 10521469
    Abstract: The present disclosure relates to an image re-ranking method, which includes: performing image searching by using an initial keyword, obtaining, by calculation, an anchor concept set of a search result according to the search result corresponding to the initial keyword, obtaining, by calculation, a weight of a correlation between anchor concepts in the anchor concept set, and forming an anchor concept graph ACG by using the anchor concepts in the anchor concept set as vertexes and the weight of the correlation between anchor concepts as a weight of a side between the vertexes; acquiring a positive training sample by using the anchor concepts, and training a classifier by using the positive training sample; obtaining a concept projection vector by using the ACG and the classifier; calculating an ACG distance between images in the search result corresponding to the initial keyword; and ranking the images according to the ACG distance.
    Type: Grant
    Filed: April 8, 2016
    Date of Patent: December 31, 2019
    Assignee: Huawei Technologies Co., Ltd.
    Inventors: Shi Qiu, Xiaogang Wang, Wenqi Ju, Jianzhuang Liu, Xiaoou Tang
  • Patent number: 10339177
    Abstract: A method for verifying facial data and a corresponding system, which comprises retrieving a plurality of source-domain datasets from a first database and a target-domain dataset from a second database different from the first database; determining a latent subspace matching with target-domain dataset best and a posterior distribution for the determined latent subspace from the target-domain dataset and the source-domain datasets; determining information shared between the target-domain data and the source-domain datasets; and establishing a Multi-Task learning model from the posterior distribution P and the shared information M on the target-domain dataset and the source-domain datasets.
    Type: Grant
    Filed: September 28, 2016
    Date of Patent: July 2, 2019
    Assignee: HUAWEI TECHNOLOGIES CO., LTD.
    Inventors: Xiaoou Tang, Chaochao Lu
  • Patent number: 10296531
    Abstract: A visual semantic complex network system and a method for generating the system have been disclosed. The system may comprise a collection device configured to retrieve a plurality of images and a plurality of texts associated with the images in accordance with given query keywords; a semantic concept determination device configured to determine semantic concepts of the retrieved images and retrieved texts for the retrieved images, respectively; a descriptor generation device configured to, from the retrieved images and texts, generate text descriptors and visual descriptors for the determined semantic concepts; and a semantic correlation device configured to determine semantic correlations and visual correlations from the generated text and visual descriptor, respectively, and to combine the determined semantic correlations and the determined visual correlations to generate the visual semantic complex network system.
    Type: Grant
    Filed: November 30, 2013
    Date of Patent: May 21, 2019
    Assignee: Beijing Sensetime Technology Development Co., Ltd.
    Inventors: Xiaoou Tang, Shi Qiu, Xiaogang Wang
  • Patent number: 10289897
    Abstract: Disclosed is an apparatus for face verification. The apparatus may comprise a feature extraction unit and a verification unit. In one embodiment, the feature extraction unit comprises a plurality of convolutional feature extraction systems trained with different face training set, wherein each of systems comprises: a plurality of cascaded convolutional, pooling, locally-connected, and fully-connected feature extraction units configured to extract facial features for face verification from face regions of face images; wherein an output unit of the unit cascade, which could be a fully-connected unit in one embodiment of the present application, is connected to at least one of previous convolutional, pooling, locally-connected, or fully-connected units, and is configured to extract facial features (referred to as deep identification-verification features or DeepID2) for face verification from the facial features in the connected units.
    Type: Grant
    Filed: December 1, 2016
    Date of Patent: May 14, 2019
    Assignee: Beijing SenseTime Technology Development Co., Ltd
    Inventors: Xiaoou Tang, Yi Sun, Xiaogang Wang