Patents by Inventor Hongli Deng
Hongli Deng has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11847796Abstract: Examples are disclosed herein that relate to automatically calibrating cameras based on human detection. One example provides a computing system comprising instructions executable to receive image data comprising depth image data and two-dimensional image data of a space from a camera, detect a person in the space via the image data, determine a skeletal representation for the person via the image data, determine over a period of time a plurality of locations at which a reference point of the skeletal representation is on a ground area in the image data, determine a ground plane of the three-dimensional representation based upon the plurality of locations at which the reference point of the skeletal representation is on the ground area in the image data, and track a location of an object within the space relative to the ground plane.Type: GrantFiled: February 18, 2021Date of Patent: December 19, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Hongli Deng, Zicheng Liu
-
Publication number: 20230326076Abstract: The description relates to cameras, and camera calibration for enhancing user experiences. One example can receive a first image of a user at a first location relative to a camera. The first image can include the user's upper body but does not include the user from head to toe. The example can receive a second image of the user at a second location relative to a camera. The second image can include the user's upper body but does not include the user from head to toe. The example can estimate a distance of the second location from the first location relative to the camera and calibrate a height and tilt angle of the camera from the first image, the second image, and the estimated distance and without a full body image of the user.Type: ApplicationFiled: April 11, 2022Publication date: October 12, 2023Applicant: Microsoft Technology Licensing, LLCInventors: Hongli DENG, Duong NGUYEN, Gabriel BLANCO SALDANA, Ryan S. MENEZES
-
Patent number: 11776160Abstract: Techniques for improved camera calibration are disclosed. An image is analyzed to identify a first set of key points for an object. A virtual object is generated. The virtual object has a second set of key points. A reprojected version of the second set is fitted to the first set in 2D space until a fitting threshold is satisfied. To do so, a 3D alignment of the second set is generated in an attempt to fit (e.g., in 2D space) the second set to the first set. Another operation includes reprojecting the second set into 2D space. In response to comparing the reprojected second set to the first set, another operation includes determining whether a fitting error between those sets satisfies the fitting threshold. A specific 3D alignment of the second set is selected. The camera is calibrated based on resulting reprojection parameters.Type: GrantFiled: October 24, 2022Date of Patent: October 3, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Hongli Deng, Ryan Savio Menezes, Gabriel Blanco Saldana, Zicheng Liu
-
Publication number: 20230252783Abstract: Techniques for inferring whether an event is occurring in 3D space based on 2D image data and for maintaining a camera's calibration are disclosed. An image of an environment is accessed. Input is received, where the input includes a 2D rule imposed against a ground plane. The 2D rule includes conditions indicative of an event. A bounding box is generated and encompasses a detected object. A point within the bounding box is projected from a 2D-space image plane of the image into 3D space to generate a 3D-space point. Based on the 3D-space point, a 3D-space ground contact point is generated. That 3D-space ground contact point is reprojected onto the ground plane of the image to generate a synthesized 2D ground contact point. A location of the synthesized 2D ground contact point is determined to satisfy the conditions.Type: ApplicationFiled: April 20, 2023Publication date: August 10, 2023Inventors: Hongli DENG, Joseph Milan FILCIK, Hao YAN, Tony Ducheng JIN, Gabriel BLANCO SALDANA, Ryan Savio MENEZES
-
Patent number: 11663822Abstract: Techniques for inferring whether an event is occurring in 3D space based on 2D image data and for maintaining a camera's calibration are disclosed. An image of an environment is accessed. Input is received, where the input includes a 2D rule imposed against a ground plane. The 2D rule includes conditions indicative of an event. A bounding box is generated and encompasses a detected object. A point within the bounding box is projected from a 2D-space image plane of the image into 3D space to generate a 3D-space point. Based on the 3D-space point, a 3D-space ground contact point is generated. That 3D-space ground contact point is reprojected onto the ground plane of the image to generate a synthesized 2D ground contact point. A location of the synthesized 2D ground contact point is determined to satisfy the conditions.Type: GrantFiled: November 24, 2020Date of Patent: May 30, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Hongli Deng, Joseph Milan Filcik, Hao Yan, Tony Ducheng Jin, Gabriel Blanco Saldana, Ryan Savio Menezes
-
Publication number: 20230154224Abstract: A method to predict a traversal-time interval for traversal of a service queue comprises receiving video of a region including the service queue, recognizing in the video, via machine vision, a plurality of persons awaiting service within the region, estimating an average crossing-time interval between successive crossings, by the plurality of persons, of a fixed boundary along the service queue, wherein such estimating is based on features of the service queue and of the one or more persons awaiting service, and returning an estimate of the traversal-time interval based on a count of the persons awaiting service and on the average crossing-time interval as estimated.Type: ApplicationFiled: November 12, 2021Publication date: May 18, 2023Applicant: Microsoft Technology Licensing, LLCInventors: Chenyang LI, Hongli DENG, Gabriel BLANCO SALDANA, Joseph Milan FILCIK, Ryan Savio MENEZES
-
Publication number: 20230091536Abstract: The description relates to cameras, such as security cameras, and providing guidance for positioning cameras to achieve desired goals. One example can receive an image of a scene overlaid with transparent indicators that reflect accuracy of object detection in individual regions of the image. The example can correlate input received from the user on the display relative to the regions. The example can analyze subsequent images of the scene with rules derived from the input from the user.Type: ApplicationFiled: October 5, 2022Publication date: March 23, 2023Applicant: Microsoft Technology Licensing, LLCInventors: Hongli Deng, Ryan Savio Menezes, Schuaib Rasool, Ali Reza Emami
-
Publication number: 20230050504Abstract: Techniques for improved camera calibration are disclosed. An image is analyzed to identify a first set of key points for an object. A virtual object is generated. The virtual object has a second set of key points. A reprojected version of the second set is fitted to the first set in 2D space until a fitting threshold is satisfied. To do so, a 3D alignment of the second set is generated in an attempt to fit (e.g., in 2D space) the second set to the first set. Another operation includes reprojecting the second set into 2D space. In response to comparing the reprojected second set to the first set, another operation includes determining whether a fitting error between those sets satisfies the fitting threshold. A specific 3D alignment of the second set is selected. The camera is calibrated based on resulting reprojection parameters.Type: ApplicationFiled: October 24, 2022Publication date: February 16, 2023Inventors: Hongli DENG, Ryan Savio MENEZES, Gabriel BLANCO SALDANA, Zicheng LIU
-
Publication number: 20230036402Abstract: To improve the accuracy and efficiency of object detection through computer digital image analysis, the detection of some objects can inform the sub-portion of the digital image to which subsequent computer digital image analysis is directed to detect other objects. In such a manner object detection can be made more efficient by limiting the image area of a digital image that is analyzed. Such efficiencies can represent both computational efficiencies and communicational efficiencies arising due to the smaller quantity of digital image data that is analyzed. Additionally, the detection of some objects can render the detection of other objects more accurate by adjusting confidence thresholds based on the detection of those related objects. Relationships between objects can be utilized to inform both the image area on which subsequent object detection is performed and the confidence level of such subsequent object detection.Type: ApplicationFiled: July 22, 2021Publication date: February 2, 2023Inventors: Lijuan WANG, Zicheng LIU, Ying JIN, Hongli DENG, Kun LUO, Pei YU, Yinpeng CHEN
-
Patent number: 11496674Abstract: The description relates to cameras, such as security cameras, and providing guidance for positioning cameras to achieve desired goals. One example can receive an image of a scene overlaid with transparent indicators that reflect accuracy of object detection in individual regions of the image. The example can correlate input received from the user on the display relative to the regions. The example can analyze subsequent images of the scene with rules derived from the input from the user.Type: GrantFiled: September 18, 2020Date of Patent: November 8, 2022Assignee: Microsoft Technology Licensing, LLCInventors: Hongli Deng, Ryan Savio Menezes, Shuaib Rasool, Ali Reza Emami
-
Patent number: 11488325Abstract: Techniques for improved camera calibration are disclosed. An image is analyzed to identify a first set of key points for an object. A virtual object is generated. The virtual object has a second set of key points. A reprojected version of the second set is fitted to the first set in 2D space until a fitting threshold is satisfied. To do so, a 3D alignment of the second set is generated in an attempt to fit (e.g., in 2D space) the second set to the first set. Another operation includes reprojecting the second set into 2D space. In response to comparing the reprojected second set to the first set, another operation includes determining whether a fitting error between those sets satisfies the fitting threshold. A specific 3D alignment of the second set is selected. The camera is calibrated based on resulting reprojection parameters.Type: GrantFiled: June 17, 2020Date of Patent: November 1, 2022Assignee: Microsoft Technology Licensing, LLCInventors: Hongli Deng, Ryan Savio Menezes, Gabriel Blanco Saldana, Zicheng Liu
-
Publication number: 20220335646Abstract: Improved techniques for determining an object's 3D orientation. An image is analyzed to identify a 2D object and a first set of key points. The first set defines a first polygon. A 3D virtual object is generated. This 3D virtual object has a second set of key points defining a second polygon representing an orientation of the 3D virtual object. The second polygon is rotated a selected number of times. For each rotation, each rotated polygon is reprojected into 2D space, and a matching score is determined between each reprojected polygon and the first polygon. A specific reprojected polygon is selected whose corresponding matching score is lowest. The orientation of the 3D virtual object is set to an orientation corresponding to the specific reprojected polygon. Based on the orientation of the 3D virtual object, an area of focus of the 2D object is determined.Type: ApplicationFiled: May 9, 2022Publication date: October 20, 2022Inventors: Hongli DENG, Ryan Savio MENEZES, Gabriel BLANCO SALDANA, Zicheng LIU
-
Publication number: 20220164578Abstract: Techniques for inferring whether an event is occurring in 3D space based on 2D image data and for maintaining a camera's calibration are disclosed. An image of an environment is accessed. Input is received, where the input includes a 2D rule imposed against a ground plane. The 2D rule includes conditions indicative of an event. A bounding box is generated and encompasses a detected object. A point within the bounding box is projected from a 2D-space image plane of the image into 3D space to generate a 3D-space point. Based on the 3D-space point, a 3D-space ground contact point is generated. That 3D-space ground contact point is reprojected onto the ground plane of the image to generate a synthesized 2D ground contact point. A location of the synthesized 2D ground contact point is determined to satisfy the conditions.Type: ApplicationFiled: November 24, 2020Publication date: May 26, 2022Inventors: Hongli DENG, Joseph Milan FILCIK, Hao YAN, Tony Ducheng JIN, Gabriel BLANCO SALDANA, Ryan Savio MENEZES
-
Patent number: 11341674Abstract: Improved techniques for determining an object's 3D orientation. An image is analyzed to identify a 2D object and a first set of key points. The first set defines a first polygon. A 3D virtual object is generated. This 3D virtual object has a second set of key points defining a second polygon representing an orientation of the 3D virtual object. The second polygon is rotated a selected number of times. For each rotation, each rotated polygon is reprojected into 2D space, and a matching score is determined between each reprojected polygon and the first polygon. A specific reprojected polygon is selected whose corresponding matching score is lowest. The orientation of the 3D virtual object is set to an orientation corresponding to the specific reprojected polygon. Based on the orientation of the 3D virtual object, an area of focus of the 2D object is determined.Type: GrantFiled: June 17, 2020Date of Patent: May 24, 2022Assignee: Microsoft Technology Licensing, LLCInventors: Hongli Deng, Ryan Savio Menezes, Gabriel Blanco Saldana, Zicheng Liu
-
Publication number: 20220094844Abstract: The description relates to cameras, such as security cameras, and providing guidance for positioning cameras to achieve desired goals. One example can receive an image of a scene overlaid with transparent indicators that reflect accuracy of object detection in individual regions of the image. The example can correlate input received from the user on the display relative to the regions. The example can analyze subsequent images of the scene with rules derived from the input from the user.Type: ApplicationFiled: September 18, 2020Publication date: March 24, 2022Applicant: Microsoft Technology Licensing, LLCInventors: Hongli Deng, Ryan Savio Menezes, Shuaib Rasool, Ali Reza Emami
-
Publication number: 20210398318Abstract: Techniques for improved camera calibration are disclosed. An image is analyzed to identify a first set of key points for an object. A virtual object is generated. The virtual object has a second set of key points. A reprojected version of the second set is fitted to the first set in 2D space until a fitting threshold is satisfied. To do so, a 3D alignment of the second set is generated in an attempt to fit (e.g., in 2D space) the second set to the first set. Another operation includes reprojecting the second set into 2D space. In response to comparing the reprojected second set to the first set, another operation includes determining whether a fitting error between those sets satisfies the fitting threshold. A specific 3D alignment of the second set is selected. The camera is calibrated based on resulting reprojection parameters.Type: ApplicationFiled: June 17, 2020Publication date: December 23, 2021Inventors: Hongli DENG, Ryan Savio MENEZES, Gabriel BLANCO SALDANA, Zicheng LIU
-
Publication number: 20210397871Abstract: Improved techniques for determining an object's 3D orientation. An image is analyzed to identify a 2D object and a first set of key points. The first set defines a first polygon. A 3D virtual object is generated. This 3D virtual object has a second set of key points defining a second polygon representing an orientation of the 3D virtual object. The second polygon is rotated a selected number of times. For each rotation, each rotated polygon is reprojected into 2D space, and a matching score is determined between each reprojected polygon and the first polygon. A specific reprojected polygon is selected whose corresponding matching score is lowest. The orientation of the 3D virtual object is set to an orientation corresponding to the specific reprojected polygon. Based on the orientation of the 3D virtual object, an area of focus of the 2D object is determined.Type: ApplicationFiled: June 17, 2020Publication date: December 23, 2021Inventors: Hongli DENG, Ryan Savio MENEZES, Gabriel BLANCO SALDANA, Zicheng LIU
-
Publication number: 20210174548Abstract: Examples are disclosed herein that relate to automatically calibrating cameras based on human detection. One example provides a computing system comprising instructions executable to receive image data comprising depth image data and two-dimensional image data of a space from a camera, detect a person in the space via the image data, determine a skeletal representation for the person via the image data, determine over a period of time a plurality of locations at which a reference point of the skeletal representation is on a ground area in the image data, determine a ground plane of the three-dimensional representation based upon the plurality of locations at which the reference point of the skeletal representation is on the ground area in the image data, and track a location of an object within the space relative to the ground plane.Type: ApplicationFiled: February 18, 2021Publication date: June 10, 2021Applicant: Microsoft Technology Licensing, LLCInventors: Hongli DENG, Zicheng LIU
-
Patent number: 10957074Abstract: Examples are disclosed herein that relate to automatically calibrating cameras based on human detection. One example provides a computing system comprising instructions executable to receive image data comprising depth image data and two-dimensional image data of a space from a camera, detect a person in the space via the image data, determine a skeletal representation for the person via the image data, determine over a period of time a plurality of locations at which a reference point of the skeletal representation is on a ground area in the image data, determine a ground plane of the three-dimensional representation based upon the plurality of locations at which the reference point of the skeletal representation is on the ground area in the image data, and track a location of an object within the space relative to the ground plane.Type: GrantFiled: January 29, 2019Date of Patent: March 23, 2021Assignee: Microsoft Technology Licensing, LLCInventors: Hongli Deng, Zicheng Liu
-
Publication number: 20200242805Abstract: Examples are disclosed herein that relate to automatically calibrating cameras based on human detection. One example provides a computing system comprising instructions executable to receive image data comprising depth image data and two-dimensional image data of a space from a camera, detect a person in the space via the image data, determine a skeletal representation for the person via the image data, determine over a period of time a plurality of locations at which a reference point of the skeletal representation is on a ground area in the image data, determine a ground plane of the three-dimensional representation based upon the plurality of locations at which the reference point of the skeletal representation is on the ground area in the image data, and track a location of an object within the space relative to the ground plane.Type: ApplicationFiled: January 29, 2019Publication date: July 30, 2020Applicant: Microsoft Technology Licensing, LLCInventors: Hongli DENG, Zicheng LIU