Patents by Inventor Harpreet Sawhney
Harpreet Sawhney has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 10963504Abstract: Zero-shot content detection includes building/training a semantic space by embedding word-based document descriptions of a plurality of documents into a multi-dimensional space using a semantic embedding technique; detecting a plurality of features in the multimodal content by applying feature detection algorithms to the multimodal content; determining respective word-based concept descriptions for concepts identified in the multimodal content using the detected features; embedding the respective word-based concept descriptions into the semantic space; and in response to a content detection action, (i) embedding/mapping words representative of the content detection action into the semantic space, (ii) automatically determining, without the use of training examples, concepts in the semantic space relevant to the content detection action based on the embedded words, and (iii) identifying portions of the multimodal content responsive to the content detection action based on the concepts in the semantic space detType: GrantFiled: February 13, 2017Date of Patent: March 30, 2021Assignee: SRI InternationalInventors: Hui Cheng, Jingen Liu, Harpreet Sawhney, Mohamed Elhoseiny
-
Patent number: 10769491Abstract: Techniques are disclosed for identifying discriminative, fine-grained features of an object in an image. In one example, an input device receives an image. A machine learning system includes a model comprising a first set, a second set, and a third set of filters. The machine learning system applies the first set of filters to the received image to generate an intermediate representation of the received image. The machine learning system applies the second set of filters to the intermediate representation to generate part localization data identifying sub-parts of an object and one or more regions of the image in which the sub-parts are located. The machine learning system applies the third set of filters to the intermediate representation to generate classification data identifying a subordinate category to which the object belongs. The system uses the part localization and classification data to perform fine-grained classification of the object.Type: GrantFiled: August 31, 2018Date of Patent: September 8, 2020Assignee: SRI InternationalInventors: Bogdan Calin Mihai Matei, Xiyang Dai, John Benjamin Southall, Nhon Hoc Trinh, Harpreet Sawhney
-
Publication number: 20190073560Abstract: Techniques are disclosed for identifying discriminative, fine-grained features of an object in an image. In one example, an input device receives an image. A machine learning system includes a model comprising a first set, a second set, and a third set of filters. The machine learning system applies the first set of filters to the received image to generate an intermediate representation of the received image. The machine learning system applies the second set of filters to the intermediate representation to generate part localization data identifying sub-parts of an object and one or more regions of the image in which the sub-parts are located. The machine learning system applies the third set of filters to the intermediate representation to generate classification data identifying a subordinate category to which the object belongs. The system uses the part localization and classification data to perform fine-grained classification of the object.Type: ApplicationFiled: August 31, 2018Publication date: March 7, 2019Inventors: Bogdan Calin Mihai Matei, Xiyang Dai, John Benjamin Southall, Nhon Hoc Trinh, Harpreet Sawhney
-
Publication number: 20190065492Abstract: Zero-shot content detection includes building/training a semantic space by embedding word-based document descriptions of a plurality of documents into a multi-dimensional space using a semantic embedding technique; detecting a plurality of features in the multimodal content by applying feature detection algorithms to the multimodal content; determining respective word-based concept descriptions for concepts identified in the multimodal content using the detected features; embedding the respective word-based concept descriptions into the semantic space; and in response to a content detection action, (i) embedding/mapping words representative of the content detection action into the semantic space, (ii) automatically determining, without the use of training examples, concepts in the semantic space relevant to the content detection action based on the embedded words, and (iii) identifying portions of the multimodal content responsive to the content detection action based on the concepts in the semantic space detType: ApplicationFiled: February 13, 2017Publication date: February 28, 2019Inventors: Hui Cheng, Jingen Liu, Harpreet Sawhney, Mohamed Elhoseiny
-
Publication number: 20180075774Abstract: A method and system for analyzing at least one food item on a food plate is disclosed. A plurality of images of the food plate is received by an image capturing device. A description of the at least one food item on the food plate is received by a recognition device. The description is at least one of a voice description and a text description. At least one processor extracts a list of food items from the description; classifies and segments the at least one food item from the list using color and texture features derived from the plurality of images; and estimates the volume of the classified and segmented at least one food item. The processor is also configured to estimate the caloric content of the at least one food item.Type: ApplicationFiled: November 20, 2017Publication date: March 15, 2018Inventors: Manika PURI, Zhiwei Zhu, Jeffrey Lubin, Tom Pschar, Ajay Divakaran, Harpreet Sawhney
-
Patent number: 9734414Abstract: A system and method for efficiently locating in 3D an object of interest in a target scene using video information captured by a plurality of cameras. The system and method provide for multi-camera visual odometry wherein pose estimates are generated for each camera by all of the cameras in the multi-camera configuration. Furthermore, the system and method can locate and identify salient landmarks in the target scene using any of the cameras in the multi-camera configuration and compare the identified landmark against a database of previously identified landmarks. In addition, the system and method provide for the integration of video-based pose estimations with position measurement data captured by one or more secondary measurement sensors, such as, for example, Inertial Measurement Units (IMUs) and Global Positioning System (GPS) units.Type: GrantFiled: August 25, 2015Date of Patent: August 15, 2017Assignee: SRI InternationalInventors: Supun Samarasekera, Rakesh Kumar, Taragay Oskiper, Zhiwei Zhu, Oleg Naroditsky, Harpreet Sawhney
-
Patent number: 9563623Abstract: Embodiments of the present invention are directed towards methods and apparatus for generating a common operating picture of an event based on the event-specific information extracted from data collected from a plurality of electronic information sources. In some embodiments, a method for generating a common operating picture of an event includes collecting data, comprising image data and textual data, from a plurality of electronic information sources, extracting information related to an event from the data, said extracted information comprising image descriptors, visual features, and categorization tags, by applying statistical analysis and semantic analysis, aligning the extracted information to generate aligned information, recognizing event-specific information for the event based on the aligned information, and generating a common operating picture of the event based on the event-specific information.Type: GrantFiled: May 21, 2015Date of Patent: February 7, 2017Assignee: SRI InternationalInventors: Harpreet Sawhney, Jayakrishnan Eledath, Ajay Divakaran, Mayank Bansal, Hui Cheng
-
Publication number: 20160078303Abstract: A system and method for efficiently locating in 3D an object of interest in a target scene using video information captured by a plurality of cameras. The system and method provide for multi-camera visual odometry wherein pose estimates are generated for each camera by all of the cameras in the multi-camera configuration. Furthermore, the system and method can locate and identify salient landmarks in the target scene using any of the cameras in the multi-camera configuration and compare the identified landmark against a database of previously identified landmarks. In addition, the system and method provide for the integration of video-based pose estimations with position measurement data captured by one or more secondary measurement sensors, such as, for example, Inertial Measurement Units (IMUs) and Global Positioning System (GPS) units.Type: ApplicationFiled: August 25, 2015Publication date: March 17, 2016Inventors: Supun Samarasekera, Rakesh Kumar, Taragay Oskiper, Zhiwei Zhu, Oleg Naroditsky, Harpreet Sawhney
-
Publication number: 20150254231Abstract: A computer-implemented method comprising collecting data from a plurality of information sources, identifying a geographic location associated with the data and forming a corresponding event according to the geographic location, correlating the data and the event with one or more topics based at least partly on the identified geographic location and storing the correlated data and event and inferring the associated geographic location if the data does not comprise explicit location information, including matching the data against a database of geo-referenced data.Type: ApplicationFiled: May 21, 2015Publication date: September 10, 2015Inventors: Harpreet Sawhney, Jayakrishnan Eledath, Ajay Divakaran, Mayank Bansal, Hui Cheng
-
Patent number: 9121713Abstract: A system and method for efficiently locating in 3D an object of interest in a target scene using video information captured by a plurality of cameras. The system and method provide for multi-camera visual odometry wherein pose estimates are generated for each camera by all of the cameras in the multi-camera configuration. Furthermore, the system and method can locate and identify salient landmarks in the target scene using any of the cameras in the multi-camera configuration and compare the identified landmark against a database of previously identified landmarks. In addition, the system and method provide for the integration of video-based pose estimations with position measurement data captured by one or more secondary measurement sensors, such as, for example, Inertial Measurement Units (IMUs) and Global Positioning System (GPS) units.Type: GrantFiled: April 19, 2012Date of Patent: September 1, 2015Assignee: SRI InternationalInventors: Supun Samarasekera, Rakesh Kumar, Taragay Oskiper, Zhiwei Zhu, Oleg Naroditsky, Harpreet Sawhney
-
Patent number: 8995717Abstract: A computer implemented method for deriving an attribute entity network (AEN) from video data is disclosed, comprising the steps of: extracting at least two entities from the video data; tracking the trajectories of the at least two entities to form at least two tracks; deriving at least one association between at least two entities by detecting at least one event involving the at least two entities, said detecting of at least one event being based on detecting at least one spatio-temporal motion correlation between the at least two entities; and constructing the AEN by creating a graph wherein the at least two objects form at least two nodes and the at least one association forms a link between the at least two nodes.Type: GrantFiled: August 29, 2012Date of Patent: March 31, 2015Assignee: SRI InternationalInventors: Hui Cheng, Jiangjian Xiao, Harpreet Sawhney
-
Patent number: 8655056Abstract: A computer implemented method for matching video data to a database containing a plurality of video fingerprints of the type described above, comprising the steps of calculating at least one fingerprint representing at least one query frame from the video data; indexing into the database using the at least one calculated fingerprint to find a set of candidate fingerprints; applying a score to each of the candidate fingerprints; selecting a subset of candidate fingerprints as proposed frames by rank ordering the candidate fingerprints; and attempting to match at least one fingerprint of at least one proposed frame.Type: GrantFiled: May 17, 2013Date of Patent: February 18, 2014Assignee: SRI InternationalInventors: Gajinder Singh, Manika Puri, Jeffrey Lubin, Harpreet Sawhney
-
Publication number: 20130259361Abstract: A computer implemented method for matching video data to a database containing a plurality of video fingerprints of the type described above, comprising the steps of calculating at least one fingerprint representing at least one query frame from the video data; indexing into the database using the at least one calculated fingerprint to find a set of candidate fingerprints; applying a score to each of the candidate fingerprints; selecting a subset of candidate fingerprints as proposed frames by rank ordering the candidate fingerprints; and attempting to match at least one fingerprint of at least one proposed frame.Type: ApplicationFiled: May 17, 2013Publication date: October 3, 2013Inventors: GAJINDER SINGH, MANIKA PURI, JEFFREY LUBIN, HARPREET SAWHNEY
-
Publication number: 20130260345Abstract: A method and system for analyzing at least one food item on a food plate is disclosed. A plurality of images of the food plate is received by an image capturing device. A description of the at least one food item on the food plate is received by a recognition device. The description is at least one of a voice description and a text description. At least one processor extracts a list of food items from the description; classifies and segments the at least one food item from the list using color and texture features derived from the plurality of images; and estimates the volume of the classified and segmented at least one food item. The processor is also configured to estimate the caloric content of the at least one food item.Type: ApplicationFiled: March 22, 2013Publication date: October 3, 2013Applicant: SRI InternationalInventors: MANIKA PURI, ZHIWEI ZHU, JEFFREY LUBIN, TOM PSCHAR, AJAY DIVAKARAN, HARPREET SAWHNEY
-
Publication number: 20120321137Abstract: A computer implemented method for deriving an attribute entity network (AEN) from video data is disclosed, comprising the steps of: extracting at least two entities from the video data; tracking the trajectories of the at least two entities to form at least two tracks; deriving at least one association between at least two entities by detecting at least one event involving the at least two entities, said detecting of at least one event being based on detecting at least one spatio-temporal motion correlation between the at least two entities; and constructing the AEN by creating a graph wherein the at least two objects form at least two nodes and the at least one association forms a link between the at least two nodes.Type: ApplicationFiled: August 29, 2012Publication date: December 20, 2012Applicant: SRI INTERNATIONALInventors: HUI CHENG, JIANGJIAN XIAO, HARPREET SAWHNEY
-
Patent number: 8294763Abstract: A computer implemented method for deriving an attribute entity network (AEN) from video data is disclosed, comprising the steps of extracting at least two entities from the video data, tracking the trajectories of the at least two entities to form at least two tracks, deriving at least one association between at least two entities by detecting at least one event involving the at least two entities, where the detecting of at least one event is based on detecting at least one spatiotemporal motion correlation between the at least two entities, and constructing the AEN by creating a graph wherein the at least two objects form at least two nodes and the at least one association forms a link between the at least two nodes.Type: GrantFiled: November 14, 2008Date of Patent: October 23, 2012Assignee: SRI InternationalInventors: Hui Cheng, Jiangjian Xiao, Harpreet Sawhney
-
Patent number: 8289390Abstract: A sentient system combines detection, tracking, and immersive visualization of a cluttered and crowded environment, such as an office building, terminal, or other enclosed site using a network of stereo cameras. A guard monitors the site using a live 3D model, which is updated from different directions using the multiple video streams. As a person moves within the view of a camera, the system detects its motion and tracks the person's path, it hands off the track to the next camera when the person goes out of that camera's view. Multiple people can be tracked simultaneously both within and across cameras, with each track shown on a map display. The track system includes a track map browser that displays the tracks of all moving objects as well as a history of recent tracks and a video flashlight viewer that displays live immersive video of any person that is being tracked.Type: GrantFiled: July 28, 2005Date of Patent: October 16, 2012Assignee: SRI InternationalInventors: Manoj Aggarwal, Keith Hanna, Harpreet Sawhney, Vincent Paragano, Rakesh Kumar, Supun Samarasekera, Aydin Arpa, Thomas Germano, Tao Zhao, David Kalokitis, David R. Patterson
-
Publication number: 20120206596Abstract: A system and method for efficiently locating in 3D an object of interest in a target scene using video information captured by a plurality of cameras. The system and method provide for multi-camera visual odometry wherein pose estimates are generated for each camera by all of the cameras in the multi-camera configuration. Furthermore, the system and method can locate and identify salient landmarks in the target scene using any of the cameras in the multi-camera configuration and compare the identified landmark against a database of previously identified landmarks. In addition, the system and method provide for the integration of video-based pose estimations with position measurement data captured by one or more secondary measurement sensors, such as, for example, Inertial Measurement Units (IMUs) and Global Positioning System (GPS) units.Type: ApplicationFiled: April 19, 2012Publication date: August 16, 2012Applicant: SRI INTERNATIONALInventors: SUPUN SAMARASEKERA, RAKESH KUMAR, TARAGAY OSKIPER, ZHIWEI ZHU, OLEG NARODITSKY, HARPREET SAWHNEY
-
Patent number: 8174568Abstract: A system and method for efficiently locating in 3D an object of interest in a target scene using video information captured by a plurality of cameras. The system and method provide for multi-camera visual odometry wherein pose estimates are generated for each camera by all of the cameras in the multi-camera configuration. Furthermore, the system and method can locate and identify salient landmarks in the target scene using any of the cameras in the multi-camera configuration and compare the identified landmark against a database of previously identified landmarks. In addition, the system and method provide for the integration of video-based pose estimations with position measurement data captured by one or more secondary measurement sensors, such as, for example, Inertial Measurement Units (IMUs) and Global Positioning System (GPS) units.Type: GrantFiled: December 3, 2007Date of Patent: May 8, 2012Assignee: SRI InternationalInventors: Supun Samarasekera, Rakesh Kumar, Taragay Oskiper, Zhiwei Zhu, Oleg Naroditsky, Harpreet Sawhney
-
Patent number: 8063936Abstract: According to an aspect of the invention, a system for providing immersive surveillance a site has a plurality of cameras each producing a respective raw video of a respective portion of the site. A processing component receives the raw video from the cameras and generates processed video from it. A visualization engine is coupled to the processing system, and receives the processed video therefrom. The visualization engine renders real-time images corresponding to a view of the site in which at least a portion of the processed video is overlaid onto a rendering of an image based on a computer model of the site. The visualization engine displays the images in real time to a viewer. The processing component comprises first and second filter modules. The second filter module processes video received as output from the first filter module. A controller component controls all transmission of data and video between the first and second filter modules.Type: GrantFiled: June 1, 2005Date of Patent: November 22, 2011Assignee: L-3 Communications CorporationInventors: Supun Samarasekera, Vincent Paragano, Harpreet Sawhney, Manoj Aggarwal, Keith Hanna, Rakesh Kumar, Aydin Arpa, Philip Miller