Patents by Inventor Yanghai Tsin

Yanghai Tsin has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20200326777
    Abstract: Methods and apparatus for glint-assisted gaze tracking in a VR/AR head-mounted display (HMD). Images of a user's eyes captured by gaze tracking cameras may be analyzed to detect glints (reflections on the cornea of light sources that illuminate the user's eyes) and the pupil. The glints are matched to particular ones of the light sources. The glint-light source matches are used to determine the cornea center of the eye, and the pupil center is determined. The optical axis of the eye is reconstructed from the cornea center and the pupil center, and the visual axis is then reconstructed from the optical axis and a 3D model of the user's eye. The point of gaze on the display is then determined based on the visual axis and a 3D model of the HMD.
    Type: Application
    Filed: June 26, 2020
    Publication date: October 15, 2020
    Applicant: Apple Inc.
    Inventors: Seyed Hesameddin Najafi Shoushtari, Kuen-Han Lin, Yanghai Tsin
  • Publication number: 20200273180
    Abstract: Various implementations disclosed herein include devices, systems, and methods that use event camera data to track deformable objects such as faces, hands, and other body parts. One exemplary implementation involves receiving a stream of pixel events output by an event camera. The device tracks the deformable object using this data. Various implementations do so by generating a dynamic representation of the object and modifying the dynamic representation of the object in response to obtaining additional pixel events output by the event camera. In some implementations, generating the dynamic representation of the object involves identifying features disposed on the deformable surface of the object using the stream of pixel events. The features are determined by identifying patterns of pixel events. As new event stream data is received, the patterns of pixel events are recognized in the new data and used to modify the dynamic representation of the object.
    Type: Application
    Filed: November 13, 2018
    Publication date: August 27, 2020
    Inventors: Peter KAUFMANN, Daniel KURZ, Brian AMBERG, Yanghai TSIN
  • Patent number: 10698481
    Abstract: Methods and apparatus for glint-assisted gaze tracking in a VR/AR head-mounted display (HMD). Images of a user's eyes captured by gaze tracking cameras may be analyzed to detect glints (reflections on the cornea of light sources that illuminate the user's eyes) and the pupil. The glints are matched to particular ones of the light sources. The glint-light source matches are used to determine the cornea center of the eye, and the pupil center is determined. The optical axis of the eye is reconstructed from the cornea center and the pupil center, and the visual axis is then reconstructed from the optical axis and a 3D model of the user's eye. The point of gaze on the display is then determined based on the visual axis and a 3D model of the HMD.
    Type: Grant
    Filed: September 26, 2018
    Date of Patent: June 30, 2020
    Assignee: Apple Inc.
    Inventors: Seyed Hesameddin Najafi Shoushtari, Kuen-Han Lin, Yanghai Tsin
  • Patent number: 9406137
    Abstract: Disclosed embodiments pertain to apparatus, systems, and methods for robust feature based tracking. In some embodiments, a score may be computed for a camera captured current image comprising a target object. The score may be based on one or more metrics determined from a comparison of features in the current image and a prior image captured by the camera. The comparison may be based on an estimated camera pose for the current image. In some embodiments, one of a point based, an edge based, or a combined point and edge based feature correspondence method may be selected based on a comparison of the score with a point threshold and/or a line threshold, the point and line thresholds being obtained from a model of the target. The camera pose may be refined by establishing feature correspondences using the selected method between the current image and a model image.
    Type: Grant
    Filed: May 15, 2014
    Date of Patent: August 2, 2016
    Assignee: QUALCOMM Incorporated
    Inventors: Dheeraj Ahuja, Kiyoung Kim, Yanghai Tsin, Seyed Hesameddin Najafi Shoushtari
  • Patent number: 9224205
    Abstract: A reference in an unknown environment is generated on the fly for positioning and tracking. The reference is produced in a top down process by capturing an image of a planar object with a predefined geometric shape, detecting edge pixels of the planar object, then detecting a plurality of line segments from the edge pixels. The plurality of line segments may then be used to detect the planar object in the image based on the predefined geometric shape. An initial pose of the camera with respect to the planar object is determined and tracked using the edges of the planar object.
    Type: Grant
    Filed: June 14, 2012
    Date of Patent: December 29, 2015
    Assignee: QUALCOMM Incorporated
    Inventors: Yanghai Tsin, Prasun Choudhury, Murali Ramaswamy Chari
  • Patent number: 9147122
    Abstract: A method for estimating camera pose includes: obtaining an image of a location captured via a camera, where the image includes a target object and edge line features outside of the target object; and calculating a pose of the camera with respect to the target object based on the edge line features.
    Type: Grant
    Filed: March 14, 2013
    Date of Patent: September 29, 2015
    Assignee: QUALCOMM Incorporated
    Inventors: Hui Chao, Yanghai Tsin, Faraz Mohammad Mirzaei
  • Patent number: 9116938
    Abstract: In several aspects, an electronic device and method index a repository of N documents by W words, by not storing between queries, N*W numbers that are specific to each word i and each document j, normally used to compute a score of relevance to a query, of each document j. Instead, the electronic device and method generate the N*W word-specific -document-specific numbers dynamically at query time, based on a set of W numbers corresponding to the W words, and one or more sets (e.g. x sets) of N numbers corresponding to the N documents. Query-time generation of word-specific-document-specific numbers reduces memory otherwise required, e.g. to store these numbers. Hence, in certain aspects W+xN numbers are maintained between queries, and these numbers are changed incrementally when a new document is added to the set or an existing document is removed. Maintaining W+xN numbers reduces processing otherwise required, to start from scratch.
    Type: Grant
    Filed: March 15, 2013
    Date of Patent: August 25, 2015
    Assignee: QUALCOMM Incorporated
    Inventors: Ashwin Swaminathan, Jose Jeronimo Moreira Rodrigues, Raghuraman Krishnamoorthi, Yanghai Tsin, Murali Ramaswamy Chari
  • Publication number: 20150199572
    Abstract: Embodiments disclosed pertain to object tracking based, in part, on occluding contours associated with the tracked object. In some embodiments, a 6 Degrees of Freedom (6-DoF) initial camera pose relative to a tracked object in a first image may be obtained. A 6-DoF updated camera pose relative to the tracked object for a second image subsequent to the first image may then be obtained based, at least, on the initial camera pose and one or more features associated with the tracked object and an occluding contour associated with the tracked object in the second image. The occluding contour associated with the tracked object in the second image may be derived from a closed form function.
    Type: Application
    Filed: January 16, 2014
    Publication date: July 16, 2015
    Applicant: QUALCOMM Incorporated
    Inventors: Kiyoung Kim, Yanghai Tsin
  • Patent number: 9025823
    Abstract: A method of real-time tracking of an object includes capturing a first and a second image of the object. The object is detected in the first image and movement of the object is tracked between the images. Tracking of the object includes obtaining an initial pose of the camera; projecting an image of a model object onto the second image; determining a gradient profile of the second image from an edge point of the model object along a first direction that is normal to the edge of the model object; computing a radius on the gradient profile; determining a rank order of the peaks of the gradient profile along the radius; comparing the rank order with a predetermined rank order to generate a feature candidate point; and reducing a distance along the first direction between the feature candidate point and the edge point on the edge of the model object.
    Type: Grant
    Filed: March 12, 2013
    Date of Patent: May 5, 2015
    Assignee: QUALCOMM Incorporated
    Inventors: Yanghai Tsin, Kiyoung Kim
  • Publication number: 20140369557
    Abstract: Disclosed embodiments pertain to feature based tracking. In some embodiments, a camera pose may be obtained relative to a tracked object in a first image and a predicted camera pose relative to the tracked object may be determined for a second image subsequent to the first image based, in part, on a motion model of the tracked object. An updated SE(3) camera pose may then be obtained based, in part on the predicted camera pose, by estimating a plane induced homography using an equation of a dominant plane of the tracked object, wherein the plane induced homography is used to align a first lower resolution version of the first image and a first lower resolution version of the second image by minimizing the sum of their squared intensity differences. A feature tracker may be initialized with the updated SE(3) camera pose.
    Type: Application
    Filed: April 28, 2014
    Publication date: December 18, 2014
    Applicant: QUALCOMM Incorporated
    Inventors: Guy-Richard Kayombya, Seyed Hesameddin Najafi Shoushtari, Dheeraj Ahuja, Yanghai Tsin
  • Publication number: 20140368645
    Abstract: Disclosed embodiments pertain to apparatus, systems, and methods for robust feature based tracking. In some embodiments, a score may be computed for a camera captured current image comprising a target object. The score may be based on one or more metrics determined from a comparison of features in the current image and a prior image captured by the camera. The comparison may be based on an estimated camera pose for the current image. In some embodiments, one of a point based, an edge based, or a combined point and edge based feature correspondence method may be selected based on a comparison of the score with a point threshold and/or a line threshold, the point and line thresholds being obtained from a model of the target. The camera pose may be refined by establishing feature correspondences using the selected method between the current image and a model image.
    Type: Application
    Filed: May 15, 2014
    Publication date: December 18, 2014
    Applicant: QUALCOMM Incorporated
    Inventors: Dheeraj AHUJA, Kiyoung KIM, Yanghai TSIN, Seyed Hesameddin NAJAFI SHOUSHTARI
  • Publication number: 20140270362
    Abstract: Embodiments include detection or relocalization of an object in a current image from a reference image, such as using a simple and relatively fast and invariant edge orientation based edge feature extraction, then a weak initial matching combined with a strong contextual filtering framework, and then a pose estimation framework based on edge segments. Embodiments include fast edge-based object detection using instant learning with a sufficiently large coverage area for object re-localization. Embodiments provide a good trade-off between computational efficiency of the extraction and matching processes.
    Type: Application
    Filed: March 15, 2013
    Publication date: September 18, 2014
    Applicant: QUALCOMM INCORPORATED
    Inventors: Seyed Hesameddin Najafi Shoushtari, Yanghai Tsin, Murali Ramaswamy Chari, Serafin Diaz Spindola
  • Publication number: 20140270346
    Abstract: A method of real-time tracking of an object includes capturing a first and a second image of the object. The object is detected in the first image and movement of the object is tracked between the images. Tracking of the object includes obtaining an initial pose of the camera; projecting an image of a model object onto the second image; determining a gradient profile of the second image from an edge point of the model object along a first direction that is normal to the edge of the model object; computing a radius on the gradient profile; determining a rank order of the peaks of the gradient profile along the radius; comparing the rank order with a predetermined rank order to generate a feature candidate point; and reducing a distance along the first direction between the feature candidate point and the edge point on the edge of the model object.
    Type: Application
    Filed: March 12, 2013
    Publication date: September 18, 2014
    Applicant: QUALCOMM Incorporated
    Inventors: Yanghai Tsin, Kiyoung Kim
  • Publication number: 20140280184
    Abstract: In several aspects, an electronic device and method index a repository of N documents by W words, by not storing between queries, N*W numbers that are specific to each word i and each document j, normally used to compute a score of relevance to a query, of each document j. Instead, the electronic device and method generate the N*W word-specific-document-specific numbers dynamically at query time, based on a set of W numbers corresponding to the W words, and one or more sets (e.g. x sets) of N numbers corresponding to the N documents. Query-time generation of word-specific-document-specific numbers reduces memory otherwise required, e.g. to store these numbers. Hence, in certain aspects W+xN numbers are maintained between queries, and these numbers are changed incrementally when a new document is added to the set or an existing document is removed. Maintaining W+xN numbers reduces processing otherwise required, to start from scratch.
    Type: Application
    Filed: March 15, 2013
    Publication date: September 18, 2014
    Applicant: QUALCOMM INCORPORATED
    Inventors: Ashwin Swaminathan, Jose Jeronimo Moreira Rodrigues, Raghuraman Krishnamoorthi, Yanghai Tsin, Murali Ramaswamy Chari
  • Patent number: 8798347
    Abstract: A method for compensating respiratory motion in coronary fluoroscopic images includes finding a set of transformation parameters of a parametric motion model that maximize an objective function that is a weighted normalized cross correlation function of a reference image acquired at a first time that is warped by the parametric motion model and a first incoming image acquired at a second time subsequent to the first time. The weights are calculated as a ratio of a covariance of the gradients of the reference image and the gradients of the first incoming image with respect to a root of a product of a variance of the gradients of the reference image and the variance of the gradients of the first incoming image. The parametric motion model transforms the reference image to match the first incoming image.
    Type: Grant
    Filed: March 11, 2011
    Date of Patent: August 5, 2014
    Assignee: Siemens Aktiengesellschaft
    Inventors: Ying Zhu, Yanghai Tsin, Hari Sundar, Frank Sauer
  • Publication number: 20130335575
    Abstract: A reference in an unknown environment is generated on the fly for positioning and tracking. The reference is produced in a top down process by capturing an image of a planar object with a predefined geometric shape, detecting edge pixels of the planar object, then detecting a plurality of line segments from the edge pixels. The plurality of line segments may then be used to detect the planar object in the image based on the predefined geometric shape. An initial pose of the camera with respect to the planar object is determined and tracked using the edges of the planar object.
    Type: Application
    Filed: June 14, 2012
    Publication date: December 19, 2013
    Applicant: QUALCOMM Incorporated
    Inventors: Yanghai Tsin, Prasun Choudhury, Murali Ramaswamy Chari
  • Publication number: 20130322767
    Abstract: A method for estimating camera pose includes: obtaining an image of a location captured via a camera, where the image includes a target object and edge line features outside of the target object; and calculating a pose of the camera with respect to the target object based on the edge line features.
    Type: Application
    Filed: March 14, 2013
    Publication date: December 5, 2013
    Applicant: QUALCOMM INCORPORATED
    Inventors: Hui CHAO, Yanghai TSIN, Faraz MOHAMMAD MIRZAEI
  • Patent number: 8428319
    Abstract: Automatic measurement of morphometric and motion parameters of a coronary target includes extracting reference frames from input data of a coronary target at different phases of a cardiac cycle, extracting a three-dimensional centerline model for each phase of the cardiac cycle based on the references frames and projection matrices of the coronary target, tracking a motion of the coronary target through the phases based on the three-dimensional centerline models, and determining a measurement of morphologic and motion parameters of the coronary target based on the motion.
    Type: Grant
    Filed: April 21, 2010
    Date of Patent: April 23, 2013
    Assignee: Siemens Aktiengesellschaft
    Inventors: Yanghai Tsin, Klaus J. Kirchberg, G√ľnter Lauritsch, Chenyang Xu, Jan Boese, Rui Liao
  • Patent number: 8320679
    Abstract: A method wherein images of different types of objects within a class are partitioned into region stacks. For each one of the stacks, the method: (a) applies a template to extract fragments having a predetermined size and one of a plurality of different spatial orientations, to generate extracted templates; (b) determines, from the extracted templates, a most frequent one thereof having only a first number of fragments with a common spatial orientations; (c) records the number of images having the determined most frequent extracted template; (d) repeats (b) and (c) with successively increasing number of fragments until the number of recoded images falls below a threshold; and (e) selects as a master extracted template the one of the most frequent templates having the largest recorded number of fragments. The master extracted templates for the stacks are combined into a map that is then compared with background images to remove extracted templates matching segment in the background.
    Type: Grant
    Filed: March 21, 2008
    Date of Patent: November 27, 2012
    Assignee: Siemens Corporate Research, Inc.
    Inventors: Yan Li, Yanghai Tsin, Yakup Genc
  • Patent number: 8212812
    Abstract: A method for modeling a vehicle, includes: receiving an image that includes a vehicle; and constructing a three-dimensional (3D) model of the vehicle, wherein the 3D model is constructed by: (a) taking a predetermined set of base shapes that are extracted from a subset of vehicles; (b) multiplying each of the base shapes by a parameter; (c) adding the resultant of each multiplication to form a vector that represents the vehicle's shape; (d) fitting the vector to the vehicle in the image; and (e) repeating steps (a)-(d) by modifying the parameters until a difference between a fit vector and the vehicle in the image is minimized.
    Type: Grant
    Filed: May 19, 2008
    Date of Patent: July 3, 2012
    Assignee: Siemens Corporation
    Inventors: Yanghai Tsin, Yakup Genc, Visvanathan Ramesh