Patents by Inventor Zehao Xue
Zehao Xue has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240104815Abstract: Embodiments described herein relate to an augmented expression system to generate and cause display of a specially configured interface to present an augmented reality perspective. The augmented expression system receives image and video data of a user and tracks facial landmarks of the user based on the image and video data, in real-time to generate and present a 3-dimensional (3D) bitmoji of the user.Type: ApplicationFiled: December 4, 2023Publication date: March 28, 2024Inventors: Chen Cao, Yang Gao, Zehao Xue
-
Patent number: 11886966Abstract: Systems and methods are provided for analyzing, by a computing device, location data associated with a location of the computing device to determine that an image or video captured using a messaging application on the computing device is captured near a food-related venue or event, receiving input related to food associated with the food-related venue or event, sending the image or video and the input related to food associated with the food-related venue or event to a computing system to train a machine learning model for food detection, and updating the messaging application to comprise the trained machine learning model for food detection.Type: GrantFiled: January 6, 2023Date of Patent: January 30, 2024Assignee: SNAP INC.Inventors: Zehao Xue, Zhou Ren
-
Patent number: 11880509Abstract: Systems and methods herein describe using a neural network to identify a first set of joint location coordinates and a second set of joint location coordinates and identifying a three-dimensional hand pose based on both the first and second sets of joint location coordinates.Type: GrantFiled: January 9, 2023Date of Patent: January 23, 2024Assignee: SNAP INC.Inventors: Yuncheng Li, Jonathan M. Rodriguez, II, Zehao Xue, Yingying Wang
-
Patent number: 11875439Abstract: Embodiments described herein relate to an augmented expression system to generate and cause display of a specially configured interface to present an augmented reality perspective. The augmented expression system receives image and video data of a user and tracks facial landmarks of the user based on the image and video data, in real-time to generate and present a 3-dimensional (3D) bitmoji of the user.Type: GrantFiled: April 15, 2020Date of Patent: January 16, 2024Assignee: Snap Inc.Inventors: Chen Cao, Yang Gao, Zehao Xue
-
Publication number: 20230419188Abstract: A machine learning scheme can be trained on a set of labeled training images of a subject in different poses, with different textures, and with different background environments. The label or marker data of the subject may be stored as metadata to a 3D model of the subject or rendered images of the subject. The machine learning scheme may be implemented as a supervised learning scheme that can automatically identify the labeled data to create a classification model. The classification model can classify a depicted subject in many different environments and arrangements (e.g., poses).Type: ApplicationFiled: September 8, 2023Publication date: December 28, 2023Inventors: Xuehan Xiong, Zehao Xue
-
Patent number: 11790276Abstract: A machine learning scheme can be trained on a set of labeled training images of a subject in different poses, with different textures, and with different background environments. The label or marker data of the subject may be stored as metadata to a 3D model of the subject or rendered images of the subject. The machine learning scheme may be implemented as a supervised learning scheme that can automatically identify the labeled data to create a classification model. The classification model can classify a depicted subject in many different environments and arrangements (e.g., poses).Type: GrantFiled: May 17, 2021Date of Patent: October 17, 2023Assignee: Snap Inc.Inventors: Xuehan Xiong, Zehao Xue
-
Patent number: 11734844Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.Type: GrantFiled: August 31, 2022Date of Patent: August 22, 2023Assignee: Snap Inc.Inventors: Liuhao Ge, Zhou Ren, Yuncheng Li, Zehao Xue, Yingying Wang
-
Publication number: 20230153396Abstract: Systems and methods are provided for analyzing, by a computing device, location data associated with a location of the computing device to determine that an image or video captured using a messaging application on the computing device is captured near a food-related venue or event, receiving input related to food associated with the food-related venue or event, sending the image or video and the input related to food associated with the food-related venue or event to a computing system to train a machine learning model for food detection, and updating the messaging application to comprise the trained machine learning model for food detection.Type: ApplicationFiled: January 6, 2023Publication date: May 18, 2023Inventors: Zehao Xue, Zhou Ren
-
Publication number: 20230106140Abstract: The present invention relates to a joint automatic audio visual driven facial animation system that in some example embodiments includes a full scale state of the art Large Vocabulary Continuous Speech Recognition (LVCSR) with a strong language model for speech recognition and obtained phoneme alignment from the word lattice.Type: ApplicationFiled: December 9, 2022Publication date: April 6, 2023Inventors: Chen Cao, Xin Chen, Wei Chu, Zehao Xue
-
Patent number: 11610354Abstract: The present invention relates to a joint automatic audio visual driven facial animation system that in some example embodiments includes a full scale state of the art Large Vocabulary Continuous Speech Recognition (LVCSR) with a strong language model for speech recognition and obtained phoneme alignment from the word lattice.Type: GrantFiled: June 16, 2021Date of Patent: March 21, 2023Assignee: Snap Inc.Inventors: Chen Cao, Xin Chen, Wei Chu, Zehao Xue
-
Patent number: 11599741Abstract: Systems and methods are provided for analyzing, by a computing device, location data associated with a location of the computing device to determine that an image or video captured using a messaging application on the computing device is captured near a food-related venue or event, receiving input related to food associated with the food-related venue or event, sending the image or video and the input related to food associated with the food-related venue or event to a computing system to train a machine learning model for food detection, and updating the messaging application to comprise the trained machine learning model for food detection.Type: GrantFiled: January 28, 2020Date of Patent: March 7, 2023Assignee: SNAP INC.Inventors: Zehao Xue, Zhou Ren
-
Patent number: 11551374Abstract: Systems and methods herein describe using a neural network to identify a first set of joint location coordinates and a second set of joint location coordinates and identifying a three-dimensional hand pose based on both the first and second sets of joint location coordinates.Type: GrantFiled: September 9, 2020Date of Patent: January 10, 2023Assignee: Snap Inc.Inventors: Yuncheng Li, Jonathan M. Rodriguez, II, Zehao Xue, Yingying Wang
-
Publication number: 20220414985Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.Type: ApplicationFiled: August 31, 2022Publication date: December 29, 2022Inventors: Liuhao Ge, Zhou Ren, Yuncheng Li, Zehao Xue, Yingying Wang
-
Publication number: 20220368824Abstract: A dolly zoom effect can be applied to one or more images captured via a resource-constrained device (e.g., a mobile smartphone) by manipulating the size of a target feature while the background in the one or more images changes due to physical movement of the resource-constrained device. The target feature can be detected using facial recognition or shape detection techniques. The target feature can be resized before the size is manipulated as the background changes (e.g., changes perspective).Type: ApplicationFiled: July 29, 2022Publication date: November 17, 2022Inventors: Linjie Luo, Chongyang Ma, Zehao Xue
-
Patent number: 11468636Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.Type: GrantFiled: April 5, 2021Date of Patent: October 11, 2022Assignee: Snap Inc.Inventors: Liuhao Ge, Zhou Ren, Yuncheng Li, Zehao Xue, Yingying Wang
-
Patent number: 11418704Abstract: A dolly zoom effect can be applied to one or more images captured via a resource-constrained device (e.g., a mobile smartphone) by manipulating the size of a target feature while the background in the one or more images changes due to physical movement of the resource-constrained device. The target feature can be detected using facial recognition or shape detection techniques. The target feature can be resized before the size is manipulated as the background changes (e.g., changes perspective).Type: GrantFiled: July 20, 2020Date of Patent: August 16, 2022Assignee: Snap Inc.Inventors: Linjie Luo, Chongyang Ma, Zehao Xue
-
Publication number: 20210312681Abstract: The present invention relates to a joint automatic audio visual driven facial animation system that in some example embodiments includes a full scale state of the art Large Vocabulary Continuous Speech Recognition (LVCSR) with a strong language model for speech recognition and obtained phoneme alignment from the word lattice.Type: ApplicationFiled: June 16, 2021Publication date: October 7, 2021Inventors: Chen Cao, Xin Chen, Wei Chu, Zehao Xue
-
Patent number: 11120597Abstract: The present invention relates to a joint automatic audio visual driven facial animation system that in some example embodiments includes a full scale state of the art Large Vocabulary Continuous Speech Recognition (LVCSR) with a strong language model for speech recognition and obtained phoneme alignment from the word lattice.Type: GrantFiled: January 22, 2020Date of Patent: September 14, 2021Assignee: Snap Inc.Inventors: Chen Cao, Xin Chen, Wei Chu, Zehao Xue
-
Publication number: 20210271874Abstract: A machine learning scheme can be trained on a set of labeled training images of a subject in different poses, with different textures, and with different background environments. The label or marker data of the subject may be stored as metadata to a 3D model of the subject or rendered images of the subject. The machine learning scheme may be implemented as a supervised learning scheme that can automatically identify the labeled data to create a classification model. The classification model can classify a depicted subject in many different environments and arrangements (e.g., poses).Type: ApplicationFiled: May 17, 2021Publication date: September 2, 2021Inventors: Xuehan Xiong, Zehao Xue
-
Publication number: 20210225077Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.Type: ApplicationFiled: April 5, 2021Publication date: July 22, 2021Inventors: Liuhao Ge, Zhou Ren, Yuncheng LI, Zehao Xue, Yingying Wang