Patents by Inventor Harpreet Singh Sawhney
Harpreet Singh Sawhney has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240127522Abstract: Examples are disclosed that relate to generating expressive avatars using multi-modal three-dimensional face modeling and tracking. One example includes a computer system comprising a processor coupled to a storage system that stores instructions. Upon execution by the processor, the instructions cause the processor to receive initialization data describing an initial state of a facial model. The instructions further cause the processor to receive a plurality of multi-modal data signals. The instructions further cause the processor to perform a fitting process using the initialization data and the plurality of multi-modal data signals. The instructions further cause the processor to determine a set of parameters based on the fitting process, wherein the determined set of parameters describes an updated state of the facial model.Type: ApplicationFiled: December 6, 2022Publication date: April 18, 2024Applicant: Microsoft Technology Licensing, LLCInventors: Harpreet Singh SAWHNEY, Benjamin Eliot LUNDELL, Anshul Bhavesh SHAH, Calin CRISTIAN, Charles Thomas HEWITT, Tadas BALTRUSAITIS, Mladen RADOJEVIC, Kosta GRUJCIC, Ivan STOJILJKOVIC, Paul Malcolm MCILROY, John Ishola OLAFENWA, Jouya JADIDIAN, Kenneth Mitchell JAKUBZAK
-
Patent number: 11836294Abstract: Examples are disclosed that relate to representing recorded hand motion. One example provides a computing device comprising a logic subsystem and a storage subsystem comprising instructions executable by the logic subsystem to receive a recorded representation of hand motion determined relative to a virtual model aligned to a first instance of an object, receive image data corresponding to an environment, and recognize a second instance of the object in the environment. The instructions are further executable to align the virtual model to the second instance of the object, and output a parametric representation of hand motion for display relative to the virtual model as aligned to the second instance of the object, such that the parametric representation is spatially consistent with the recorded representation of hand motion relative to the virtual model as aligned to the first instance of the object.Type: GrantFiled: November 22, 2022Date of Patent: December 5, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Harpreet Singh Sawhney, Ning Xu
-
Publication number: 20230079335Abstract: Examples are disclosed that relate to representing recorded hand motion. One example provides a computing device comprising a logic subsystem and a storage subsystem comprising instructions executable by the logic subsystem to receive a recorded representation of hand motion determined relative to a virtual model aligned to a first instance of an object, receive image data corresponding to an environment, and recognize a second instance of the object in the environment. The instructions are further executable to align the virtual model to the second instance of the object, and output a parametric representation of hand motion for display relative to the virtual model as aligned to the second instance of the object, such that the parametric representation is spatially consistent with the recorded representation of hand motion relative to the virtual model as aligned to the first instance of the object.Type: ApplicationFiled: November 22, 2022Publication date: March 16, 2023Applicant: Microsoft Technology Licensing, LLCInventors: Harpreet Singh SAWHNEY, Ning XU
-
Patent number: 11562598Abstract: Examples are disclosed that relate to representing recorded hand motion. One example provides a computing device comprising a logic subsystem and a storage subsystem comprising instructions executable by the logic subsystem to receive a recorded representation of hand motion determined relative to a virtual model aligned to a first instance of an object, receive image data corresponding to an environment, and recognize a second instance of the object in the environment. The instructions are further executable to align the virtual model to the second instance of the object, and output a parametric representation of hand motion for display relative to the virtual model as aligned to the second instance of the object, such that the parametric representation is spatially consistent with the recorded representation of hand motion relative to the virtual model as aligned to the first instance of the object.Type: GrantFiled: August 1, 2019Date of Patent: January 24, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Harpreet Singh Sawhney, Ning Xu
-
Publication number: 20230019745Abstract: Examples are disclosed that relate to computer-based tracking of a process performed by a user. In one example, multi-modal sensor information is received via a plurality of sensors. A world state of a real-world physical environment and a user state in the real-world physical environment are tracked based on the multi-modal sensor information. A process being performed by the user within a working domain is recognized based on the world state and the user state. A current step in the process is detected based on the world state and the user state. Domain-specific instructions directing the user how to perform an expected action are presented via a user interface device. A user action is detected based on the world state and the user state. Based on the user action differing from the expected action, domain-specific guidance to perform the expected action is presented via the user interface device.Type: ApplicationFiled: July 15, 2021Publication date: January 19, 2023Applicant: Microsoft Technology Licensing, LLCInventors: Harpreet Singh SAWHNEY, Bugra TEKIN
-
Patent number: 11538282Abstract: Examples are disclosed that relate to representing recorded hand motion. One example provides a computing device comprising a logic subsystem and a storage subsystem comprising instructions executable by the logic subsystem to receive a recorded representation of hand motion determined relative to a virtual model aligned to a first instance of an object, receive image data corresponding to an environment, and recognize a second instance of the object in the environment. The instructions are further executable to align the virtual model to the second instance of the object, and output a parametric representation of hand motion for display relative to the virtual model as aligned to the second instance of the object, such that the parametric representation is spatially consistent with the recorded representation of hand motion relative to the virtual model as aligned to the first instance of the object.Type: GrantFiled: August 1, 2019Date of Patent: December 27, 2022Assignee: Microsoft Technology Licensing, LLCInventors: Harpreet Singh Sawhney, Ning Xu
-
Patent number: 11514648Abstract: An image data annotation system automatically annotates a physical object within individual images frames of an image sequence with relevant object annotations based on a three-dimensional (3D) model of the physical object. Annotating the individual image frames with object annotations includes updating individual image frames within image input data to generate annotated image data that is suitable for reliably training a DNN object detection architecture. Exemplary object annotations that the image data annotation system can automatically apply to individual image frames include, inter alia, object pose, image pose, object masks, 3D bounding boxes composited over the physical object, 2D bounding boxes composited over the physical object, and/or depth map information.Type: GrantFiled: December 23, 2020Date of Patent: November 29, 2022Assignee: MICROSOFT TECHNOLOGY LICENSING, LLCInventors: Harpreet Singh Sawhney, Ning Xu, Amol Ashok Ambardekar, Moses Obadeji Olafenwa
-
Publication number: 20220198753Abstract: An image data annotation system automatically annotates a physical object within individual images frames of an image sequence with relevant object annotations based on a three-dimensional (3D) model of the physical object. Annotating the individual image frames with object annotations includes updating individual image frames within image input data to generate annotated image data that is suitable for reliably training a DNN object detection architecture. Exemplary object annotations that the image data annotation system can automatically apply to individual image frames include, inter alia, object pose, image pose, object masks, 3D bounding boxes composited over the physical object, 2D bounding boxes composited over the physical object, and/or depth map information.Type: ApplicationFiled: December 23, 2020Publication date: June 23, 2022Inventors: Harpreet Singh SAWHNEY, Ning XU, Amol Ashok AMBARDEKAR, Moses Obadeji OLAFENWA
-
Patent number: 11132845Abstract: A method for object recognition includes, at a computing device, receiving an image of a real-world object. An identity of the real-world object is recognized using an object recognition model trained on a plurality of computer-generated training images. A digital augmentation model corresponding to the real-world object is retrieved, the digital augmentation model including a set of augmentation-specific instructions. A pose of the digital augmentation model is aligned with a pose of the real-world object. An augmentation is provided, the augmentation associated with the real-world object and specified by the augmentation-specific instructions.Type: GrantFiled: May 22, 2019Date of Patent: September 28, 2021Assignee: Microsoft Technology Licensing, LLCInventors: Harpreet Singh Sawhney, Andrey Konin, Bilha-Catherine W. Githinji, Amol Ashok Ambardekar, William Douglas Guyman, Muhammad Zeeshan Zia, Ning Xu, Sheng Kai Tang, Pedro Urbina Escos
-
Patent number: 11106949Abstract: A computing device, including a processor configured to receive a first video including a plurality of frames. For each frame, the processor may determine that a target region of the frame includes a target object. The processor may determine a surrounding region within which the target region is located. The surrounding region may be smaller than the frame. The processor may identify one or more features located in the surrounding region. From the one or more features, the processor may generate one or more manipulated object identifiers. For each of a plurality of pairs of frames, the processor may determine a respective manipulated object movement between a first manipulated object identifier of the first frame and a second manipulated object identifier of the second frame. The processor may classify at least one action performed in the first video based on the plurality of manipulated object movements.Type: GrantFiled: March 22, 2019Date of Patent: August 31, 2021Assignee: Microsoft Technology Licensing, LLCInventors: Muhammad Zeeshan Zia, Federica Bogo, Harpreet Singh Sawhney, Huseyin Coskun, Bugra Tekin
-
Patent number: 11074477Abstract: A computing system and method for identifying related visual content of a collection of visual media files comprising one or more image files and video files includes monitoring inputs to the computing system, the inputs associated with a user interaction with electronic content using the computer system, identifying a visual media file in the collection of visual media files relevant to the electronic content based on a semantic label assigned to the visual media file by the computing system, creating a representative image of the identified visual media file, and displaying the representative image for selection. The computing system enables a selection of the displayed representative image for association of the identified visual media file with the electronic content.Type: GrantFiled: July 20, 2017Date of Patent: July 27, 2021Assignee: SRI InternationalInventors: Harpreet Singh Sawhney, Jayakrishnan Eledath, Mayank Bansal, Bogdan C. Matei, Xutao Lv, Chaitanya Desai, Timothy Shields
-
Publication number: 20200372715Abstract: A method for object recognition includes, at a computing device, receiving an image of a real-world object. An identity of the real-world object is recognized using an object recognition model trained on a plurality of computer-generated training images. A digital augmentation model corresponding to the real-world object is retrieved, the digital augmentation model including a set of augmentation-specific instructions. A pose of the digital augmentation model is aligned with a pose of the real-world object. An augmentation is provided, the augmentation associated with the real-world object and specified by the augmentation-specific instructions.Type: ApplicationFiled: May 22, 2019Publication date: November 26, 2020Applicant: Microsoft Technology Licensing, LLCInventors: Harpreet Singh SAWHNEY, Andrey KONIN, Bilha-Catherine W. GITHINJI, Amol Ashok AMBARDEKAR, William Douglas GUYMAN, Muhammad Zeeshan ZIA, Ning XU, Sheng Kai TANG, Pedro URBINA ESCOS
-
Publication number: 20200311396Abstract: Examples are disclosed that relate to representing recorded hand motion. One example provides a computing device comprising instructions executable by a logic subsystem to receive video data capturing hand motion relative to an object, determine a first pose of the object, and associate a first coordinate system with the object based on the first pose. The instructions are further executable to determine a representation of the hand motion in the first coordinate system, the representation having a time-varying pose relative to the first pose of the object, and configure the representation for display relative to a second instance of the object having a second pose in a second coordinate system, with a time-varying pose relative to the second pose that is spatially consistent with the time-varying pose relative to the first pose.Type: ApplicationFiled: March 25, 2019Publication date: October 1, 2020Applicant: Microsoft Technology Licensing, LLCInventors: Marc Andre Leon POLLEFEYS, Sudipta Narayan SINHA, Harpreet Singh SAWHNEY, Bugra TEKIN, Federica BOGO
-
Publication number: 20200311397Abstract: Examples are disclosed that relate to representing recorded hand motion. One example provides a computing device comprising a logic subsystem and a storage subsystem comprising instructions executable by the logic subsystem to receive a recorded representation of hand motion determined relative to a virtual model aligned to a first instance of an object, receive image data corresponding to an environment, and recognize a second instance of the object in the environment. The instructions are further executable to align the virtual model to the second instance of the object, and output a parametric representation of hand motion for display relative to the virtual model as aligned to the second instance of the object, such that the parametric representation is spatially consistent with the recorded representation of hand motion relative to the virtual model as aligned to the first instance of the object.Type: ApplicationFiled: August 1, 2019Publication date: October 1, 2020Applicant: Microsoft Technology Licensing, LLCInventors: Harpreet Singh SAWHNEY, Ning XU
-
Publication number: 20200302245Abstract: A computing device, including a processor configured to receive a first video including a plurality of frames. For each frame, the processor may determine that a target region of the frame includes a target object. The processor may determine a surrounding region within which the target region is located. The surrounding region may be smaller than the frame. The processor may identify one or more features located in the surrounding region. From the one or more features, the processor may generate one or more manipulated object identifiers. For each of a plurality of pairs of frames, the processor may determine a respective manipulated object movement between a first manipulated object identifier of the first frame and a second manipulated object identifier of the second frame. The processor may classify at least one action performed in the first video based on the plurality of manipulated object movements.Type: ApplicationFiled: March 22, 2019Publication date: September 24, 2020Applicant: Microsoft Technology Licensing, LLCInventors: Muhammad Zeeshan ZIA, Federica BOGO, Harpreet Singh SAWHNEY, Huseyin COSKUN, Bugra TEKIN
-
Patent number: 10726264Abstract: Techniques for localizing a device based on images captured by the device. The techniques include receiving, from a device via a data communication network, image frame data for frames captured by an imaging camera included in the device, the frames including a first frame, automatically detecting real-world objects captured in the image frame data, automatically classifying the detected real-world objects as being associated with respective object classes, automatically identifying object class and instance dependent keypoints for the real-world objects based on the object classes associated with the real-world objects, and estimating a pose of the device for the first frame based on at least the identified object class and instance dependent keypoints.Type: GrantFiled: June 25, 2018Date of Patent: July 28, 2020Assignee: Microsoft Technology Licensing, LLCInventors: Harpreet Singh Sawhney, Marc André Léon Pollefeys
-
Patent number: 10691743Abstract: A computing system for realizing visual content of an image collection executes feature detection algorithms and semantic reasoning techniques on the images in the collection to elicit a number of different types of visual features of the images. The computing system indexes the visual features and provides technologies for multi-dimensional content-based clustering, searching, and iterative exploration of the image collection using the visual features and/or the visual feature indices.Type: GrantFiled: August 5, 2014Date of Patent: June 23, 2020Assignee: SRI InternationalInventors: Harpreet Singh Sawhney, Jayakrishnan Eledath, Mayank Bansal, Bogdan C. Matei, Xutao Lv, Chaitanya Desai, Timothy Shields
-
Patent number: 10679063Abstract: A computing system for recognizing salient events depicted in a video utilizes learning algorithms to detect audio and visual features of the video. The computing system identifies one or more salient events depicted in the video based on the audio and visual features.Type: GrantFiled: September 4, 2015Date of Patent: June 9, 2020Assignee: SRI InternationalInventors: Hui Cheng, Ajay Divakaran, Elizabeth Shriberg, Harpreet Singh Sawhney, Jingen Liu, Ishani Chakraborty, Omar Javed, David Chisolm, Behjat Siddiquie, Steven S. Weiner
-
Publication number: 20190392212Abstract: Techniques for localizing a device based on images captured by the device. The techniques include receiving, from a device via a data communication network, image frame data for frames captured by an imaging camera included in the device, the frames including a first frame, automatically detecting real-world objects captured in the image frame data, automatically classifying the detected real-world objects as being associated with respective object classes, automatically identifying object class and instance dependent keypoints for the real-world objects based on the object classes associated with the real-world objects, and estimating a pose of the device for the first frame based on at least the identified object class and instance dependent keypoints.Type: ApplicationFiled: June 25, 2018Publication date: December 26, 2019Applicant: MICROSOFT TECHNOLOGY LICENSING, LLCInventors: Harpreet Singh SAWHNEY, Marc André Léon POLLEFEYS
-
Patent number: 10268900Abstract: A system for object detection and tracking includes technologies to, among other things, detect and track moving objects, such as pedestrians and/or vehicles, in a real-world environment, handle static and dynamic occlusions, and continue tracking moving objects across the fields of view of multiple different cameras.Type: GrantFiled: February 27, 2018Date of Patent: April 23, 2019Assignee: SRI InternationalInventors: Ajay Divakaran, Qian Yu, Amir Tamrakar, Harpreet Singh Sawhney, Jiejie Zhu, Omar Javed, Jingen Liu, Hui Cheng, Jayakrishnan Eledath