Patents by Inventor Xuehan Xiong
Xuehan Xiong has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 10776663Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can be assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.Type: GrantFiled: July 25, 2019Date of Patent: September 15, 2020Assignee: Snap Inc.Inventors: Lidiia Bogdanovych, William Brendel, Samuel Edward Hare, Fedir Poliakov, Guohui Wang, Xuehan Xiong, Jianchao Yang, Linjie Yang
-
Patent number: 10579869Abstract: A machine learning scheme can be trained on a set of labeled training images of a subject in different poses, with different textures, and with different background environments. The label or marker data of the subject may be stored as metadata to a 3D model of the subject or rendered images of the subject. The machine learning scheme may be implemented as a supervised learning scheme that can automatically identify the labeled data to create a classification model. The classification model can classify a depicted subject in many different environments and arrangements (e.g., poses).Type: GrantFiled: July 18, 2017Date of Patent: March 3, 2020Assignee: Snap Inc.Inventors: Xuehan Xiong, Zehao Xue
-
Publication number: 20200050866Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.Type: ApplicationFiled: October 16, 2019Publication date: February 13, 2020Inventors: Samuel Edward Hare, Fedir Poliakov, Guohui Wang, Xuehan Xiong, Jianchao Yang, Linjie Yang, Shah Tanmay Anilkumar
-
Patent number: 10474900Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.Type: GrantFiled: September 15, 2017Date of Patent: November 12, 2019Assignee: Snap Inc.Inventors: Samuel Edward Hare, Fedir Poliakov, Guohui Wang, Xuehan Xiong, Jianchao Yang, Linjie Yang, Shah Tanmay Anilkumar
-
Publication number: 20190332852Abstract: Systems, devices, media, and methods are presented for modeling facial representations using image segmentation with a client device. The systems and methods receive an image depicting a face, detect at least a portion of the face within the image, and identify a set of facial features within the portion of the face. The systems and methods generate a descriptor function representing the set of facial features, fit object functions of the descriptor function, identify an identification probability for each facial feature, and assign an identification to each facial feature.Type: ApplicationFiled: July 11, 2019Publication date: October 31, 2019Inventors: Jia Li, Xutao Lv, Xiaoyu Wang, Xuehan Xiong, Jiancho Yang
-
Patent number: 10402689Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can be assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.Type: GrantFiled: September 15, 2017Date of Patent: September 3, 2019Assignee: Snap Inc.Inventors: Lidiia Bogdanovych, William Brendel, Samuel Edward Hare, Fedir Poliakov, Guohui Wang, Xuehan Xiong, Jianchao Yang, Linjie Yang
-
Patent number: 10395100Abstract: Systems, devices, media, and methods are presented for modeling facial representations using image segmentation with a client device. The systems and methods receive an image depicting a face, detect at least a portion of the face within the image, and identify a set of facial features within the portion of the face. The systems and methods generate a descriptor function representing the set of facial features, fit object functions of the descriptor function, identify an identification probability for each facial feature, and assign an identification to each facial feature.Type: GrantFiled: December 19, 2018Date of Patent: August 27, 2019Assignee: Snap Inc.Inventors: Jia Li, Xutao Lv, Xiaoyu Wang, Xuehan Xiong, Jianchao Yang
-
Publication number: 20190087660Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.Type: ApplicationFiled: September 15, 2017Publication date: March 21, 2019Inventors: Samuel Edward Hare, Fedir Poliakov, Guohui Wang, Xuehan Xiong, Jianchao Yang, Linjie Yang, Shah Tanmay Anilkumar
-
Patent number: 10198626Abstract: Systems, devices, media, and methods are presented for modeling facial representations using image segmentation with a client device. The systems and methods receive an image depicting a face, detect at least a portion of the face within the image, and identify a set of facial features within the portion of the face. The systems and methods generate a descriptor function representing the set of facial features, fit object functions of the descriptor function, identify an identification probability for each facial feature, and assign an identification to each facial feature.Type: GrantFiled: October 19, 2016Date of Patent: February 5, 2019Assignee: Snap Inc.Inventors: Jia Li, Xutao Lv, Xiaoyu Wang, Xuehan Xiong, Jianchao Yang
-
Publication number: 20180107866Abstract: Systems, devices, media, and methods are presented for modeling facial representations using image segmentation with a client device. The systems and methods receive an image depicting a face, detect at least a portion of the face within the image, and identify a set of facial features within the portion of the face. The systems and methods generate a descriptor function representing the set of facial features, fit object functions of the descriptor function, identify an identification probability for each facial feature, and assign an identification to each facial feature.Type: ApplicationFiled: October 19, 2016Publication date: April 19, 2018Inventors: Jia Li, Xutao Lv, Xiaoyu Xiaoyu, Xuehan Xiong, Jianchao Yang
-
Patent number: 9928405Abstract: The present invention relates to a system for detecting and tracking facial features in images and can be used in conjunction with a camera. Given a camera, the system will detect facial landmarks in images. The present invention includes software for real time, accurate facial feature detection and tracking in unconstrained images and videos. The present invention is better, more robust and faster than existing approaches and can be implemented very efficiently allowing real-time processing, even on low-power devices, such as mobile phones.Type: GrantFiled: May 27, 2016Date of Patent: March 27, 2018Assignee: CARNEGIE MELLON UNIVERSITYInventors: Fernando De la Torre, Xuehan Xiong
-
Patent number: 9773179Abstract: A method for monitoring a vehicle operator can be executed by a controller and includes the following steps: (a) receiving image data of a vehicle operator's head; (b) tracking facial feature points of the vehicle operator based on the image data; (c) creating a 3D model of the vehicle operator's head based on the facial feature points in order to determine a 3D position of the vehicle operator's head; (d) determining a gaze direction of the vehicle operator based on a position of the facial feature points and the 3D model of the vehicle operator's head; (e) determining a gaze vector based on the gaze direction and the 3D position of the vehicle operator's head; and (f) commanding an indicator to activate when the gaze vector is outside a predetermined parameter.Type: GrantFiled: January 26, 2016Date of Patent: September 26, 2017Assignee: GM Global Technology Operations LLCInventors: Francisco Vicente, Zehua Huang, Xuehan Xiong, Fernando De La Torre, Wende Zhang, Dan Levi, Debbie E. Nachtegall
-
Patent number: 9659210Abstract: The present invention relates to a system for detecting and tracking facial features in images and can be used in conjunction with a camera. Given a camera, the system will detect facial landmarks in images. The present invention includes software for real time, accurate facial feature detection and tracking in unconstrained images and videos. The present invention is better, more robust and faster than existing approaches and can be implemented very efficiently allowing real-time processing, even on low-power devices, such as mobile phones.Type: GrantFiled: January 13, 2015Date of Patent: May 23, 2017Assignee: Carnegie Mellon UniversityInventors: Fernando De la Torre, Xuehan Xiong
-
Publication number: 20160275339Abstract: The present invention relates to a system for detecting and tracking facial features in images and can be used in conjunction with a camera. Given a camera, the system will detect facial landmarks in images. The present invention includes software for real time, accurate facial feature detection and tracking in unconstrained images and videos. The present invention is better, more robust and faster than existing approaches and can be implemented very efficiently allowing real-time processing, even on low-power devices, such as mobile phones.Type: ApplicationFiled: May 27, 2016Publication date: September 22, 2016Inventors: Fernando De la Torre, Xuehan Xiong
-
Publication number: 20160224852Abstract: A method for monitoring a vehicle operator can be executed by a controller and includes the following steps: (a) receiving image data of a vehicle operator's head; (b) tracking facial feature points of the vehicle operator based on the image data; (c) creating a 3D model of the vehicle operator's head based on the facial feature points in order to determine a 3D position of the vehicle operator's head; (d) determining a gaze direction of the vehicle operator based on a position of the facial feature points and the 3D model of the vehicle operator's head; (e) determining a gaze vector based on the gaze direction and the 3D position of the vehicle operator's head; and (f) commanding an indicator to activate when the gaze vector is outside a predetermined parameter.Type: ApplicationFiled: January 26, 2016Publication date: August 4, 2016Applicants: GM GLOBAL TECHNOLOGY OPERATIONS LLC, Carnegie Mellon UniversityInventors: Francisco Vicente, Zehua Huang, Xuehan Xiong, Fernando De La Torre, Wende Zhang, Dan Levi, Debbie E. Nachtegall