Patents by Inventor Rajath Shetty
Rajath Shetty has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240095460Abstract: In various examples, systems and methods that use dialogue systems associated with various machine systems and applications are described. For instance, the systems and methods may receive text data representing speech, such as a question associated with a vehicle or other machine type. The systems and methods then use a retrieval system(s) to retrieve a question/answer pair(s) associated with the text data and/or contextual information associated with the text data. In some examples, the contextual information is associated with a knowledge base associated with or corresponding to the vehicle. The systems and methods then generate a prompt using the text data, the question/answer pair(s), and/or the contextual information. Additionally, the systems and methods determine, using a language model(s) and based at least on the prompt, an output associated with the text data. For instance, the output may include information that answers the question associated with the vehicle.Type: ApplicationFiled: September 19, 2022Publication date: March 21, 2024Inventors: Peng Xu, Mostofa Patwary, Rajath Shetty, Niral Lalit Pathak, Ratin Kumar, Bryan Catanzaro, Mohammad Shoeybi
-
Patent number: 11934955Abstract: Systems and methods for more accurate and robust determination of subject characteristics from an image of the subject. One or more machine learning models receive as input an image of a subject, and output both facial landmarks and associated confidence values. Confidence values represent the degrees to which portions of the subject's face corresponding to those landmarks are occluded, i.e., the amount of uncertainty in the position of each landmark location. These landmark points and their associated confidence values, and/or associated information, may then be input to another set of one or more machine learning models which may output any facial analysis quantity or quantities, such as the subject's gaze direction, head pose, drowsiness state, cognitive load, or distraction state.Type: GrantFiled: October 31, 2022Date of Patent: March 19, 2024Assignee: NVIDIA CorporationInventors: Nuri Murat Arar, Niranjan Avadhanam, Nishant Puri, Shagan Sah, Rajath Shetty, Sujay Yadawadkar, Pavlo Molchanov
-
Publication number: 20240087561Abstract: In various examples, techniques for using scene-aware context for dialogue systems and applications are described herein. For instance, systems and methods are disclosed that process audio data representing speech in order to determine an intent associated with the speech. Systems and methods are also disclosed that process sensor data representing at least a user in order to determine a point of interest associated with the user. In some examples, the point of interest may include a landmark, a person, and/or any other object within an environment. The systems and methods may then generate a context associated with the point of interest. Additionally, the systems and methods may process the intent and the context using one or more language models. Based on the processing, the language model(s) may output data associated with the speech.Type: ApplicationFiled: September 12, 2022Publication date: March 14, 2024Inventors: Niral Lalit Pathak, Rajath Shetty, Ratin Kumar
-
Publication number: 20240022601Abstract: In various examples, techniques are described for detecting whether spoofing attacks are occurring using multiple sensors. Systems and methods are disclosed that include at least a first sensor having a first pose to capture a first perspective view of a user and a second sensor having a second pose to capture a second perspective view of the user. The first sensor and/or the second sensor may include an image sensor, a depth sensor, and/or the like. The systems and methods include a neural network that is configured to analyze first sensor data generated by the first sensor and second sensor data generated by the second sensor to determine whether a spoofing attack is occurring. The systems and methods may also perform one or more processes, such as facial recognition, based on whether the spoofing attack is occurring.Type: ApplicationFiled: July 12, 2022Publication date: January 18, 2024Inventors: Manoj Kumar Yennapureddy, Shagan Sah, Rajath Shetty
-
Publication number: 20230244941Abstract: Systems and methods for determining the gaze direction of a subject and projecting this gaze direction onto specific regions of an arbitrary three-dimensional geometry. In an exemplary embodiment, gaze direction may be determined by a regression-based machine learning model. The determined gaze direction is then projected onto a three-dimensional map or set of surfaces that may represent any desired object or system. Maps may represent any three-dimensional layout or geometry, whether actual or virtual. Gaze vectors can thus be used to determine the object of gaze within any environment. Systems can also readily and efficiently adapt for use in different environments by retrieving a different set of surfaces or regions for each environment.Type: ApplicationFiled: April 10, 2023Publication date: August 3, 2023Inventors: Nuri Murat Arar, Hairong Jiang, Nishant Puri, Rajath Shetty, Niranjan Avadhanam
-
Patent number: 11704814Abstract: In various examples, an adaptive eye tracking machine learning model engine (“adaptive-model engine”) for an eye tracking system is described. The adaptive-model engine may include an eye tracking or gaze tracking development pipeline (“adaptive-model training pipeline”) that supports collecting data, training, optimizing, and deploying an adaptive eye tracking model that is a customized eye tracking model based on a set of features of an identified deployment environment. The adaptive-model engine supports ensembling the adaptive eye tracking model that may be trained on gaze vector estimation in surround environments and ensemble based on a plurality of eye tracking variant models and a plurality of facial landmark neural network metrics.Type: GrantFiled: May 13, 2021Date of Patent: July 18, 2023Assignee: NVIDIA CorporationInventors: Nuri Murat Arar, Niranjan Avadhanam, Hairong Jiang, Nishant Puri, Rajath Shetty, Shagan Sah
-
Patent number: 11688074Abstract: In various examples, a background of an object may be modified to generate a training image. A segmentation mask may be generated and used to generate an object image that includes image data representing the object. The object image may be integrated into a different background and used for data augmentation in training a neural network. Data augmentation may also be performed using hue adjustment (e.g., of the object image) and/or rendering three-dimensional capture data that corresponds to the object from selected views. Inference scores may be analyzed to select a background for an image to be included in a training dataset. Backgrounds may be selected and training images may be added to a training dataset iteratively during training (e.g., between epochs). Additionally, early or late fusion nay be employed that uses object mask data to improve inferencing performed by a neural network trained using object mask data.Type: GrantFiled: September 30, 2020Date of Patent: June 27, 2023Assignee: NVIDIA CorporationInventors: Nishant Puri, Sakthivel Sivaraman, Rajath Shetty, Niranjan Avadhanam
-
Patent number: 11657263Abstract: Systems and methods for determining the gaze direction of a subject and projecting this gaze direction onto specific regions of an arbitrary three-dimensional geometry. In an exemplary embodiment, gaze direction may be determined by a regression-based machine learning model. The determined gaze direction is then projected onto a three-dimensional map or set of surfaces that may represent any desired object or system. Maps may represent any three-dimensional layout or geometry, whether actual or virtual. Gaze vectors can thus be used to determine the object of gaze within any environment. Systems can also readily and efficiently adapt for use in different environments by retrieving a different set of surfaces or regions for each environment.Type: GrantFiled: August 28, 2020Date of Patent: May 23, 2023Assignee: NVIDIA CorporationInventors: Nuri Murat Arar, Hairong Jiang, Nishant Puri, Rajath Shetty, Niranjan Avadhanam
-
Publication number: 20230078171Abstract: Systems and methods for more accurate and robust determination of subject characteristics from an image of the subject. One or more machine learning models receive as input an image of a subject, and output both facial landmarks and associated confidence values. Confidence values represent the degrees to which portions of the subject's face corresponding to those landmarks are occluded, i.e., the amount of uncertainty in the position of each landmark location. These landmark points and their associated confidence values, and/or associated information, may then be input to another set of one or more machine learning models which may output any facial analysis quantity or quantities, such as the subject's gaze direction, head pose, drowsiness state, cognitive load, or distraction state.Type: ApplicationFiled: October 31, 2022Publication date: March 16, 2023Inventors: Nuri Murat Arar, Niranjan Avadhanam, Nishant Puri, Shagan Sah, Rajath Shetty, Sujay Yadawadkar, Pavlo Molchanov
-
Publication number: 20220366568Abstract: In various examples, an adaptive eye tracking machine learning model engine (“adaptive-model engine”) for an eye tracking system is described. The adaptive-model engine may include an eye tracking or gaze tracking development pipeline (“adaptive-model training pipeline”) that supports collecting data, training, optimizing, and deploying an adaptive eye tracking model that is a customized eye tracking model based on a set of features of an identified deployment environment. The adaptive-model engine supports ensembling the adaptive eye tracking model that may be trained on gaze vector estimation in surround environments and ensemble based on a plurality of eye tracking variant models and a plurality of facial landmark neural network metrics.Type: ApplicationFiled: May 13, 2021Publication date: November 17, 2022Inventors: Nuri Murat Arar, Niranjan Avadhanam, Hairong Jiang, Nishant Puri, Rajath Shetty, Shagan Sah
-
Patent number: 11487968Abstract: Systems and methods for more accurate and robust determination of subject characteristics from an image of the subject. One or more machine learning models receive as input an image of a subject, and output both facial landmarks and associated confidence values. Confidence values represent the degrees to which portions of the subject's face corresponding to those landmarks are occluded, i.e., the amount of uncertainty in the position of each landmark location. These landmark points and their associated confidence values, and/or associated information, may then be input to another set of one or more machine learning models which may output any facial analysis quantity or quantities, such as the subject's gaze direction, head pose, drowsiness state, cognitive load, or distraction state.Type: GrantFiled: August 27, 2020Date of Patent: November 1, 2022Assignee: NVIDIA CorporationInventors: Nuri Murat Arar, Niranjan Avadhanam, Nishant Puri, Shagan Sah, Rajath Shetty, Sujay Yadawadkar, Pavlo Molchanov
-
Publication number: 20220101047Abstract: In various examples, a background of an object may be modified to generate a training image. A segmentation mask may be generated and used to generate an object image that includes image data representing the object. The object image may be integrated into a different background and used for data augmentation in training a neural network. Data augmentation may also be performed using hue adjustment (e.g., of the object image) and/or rendering three-dimensional capture data that corresponds to the object from selected views. Inference scores may be analyzed to select a background for an image to be included in a training dataset. Backgrounds may be selected and training images may be added to a training dataset iteratively during training (e.g., between epochs). Additionally, early or late fusion nay be employed that uses object mask data to improve inferencing performed by a neural network trained using object mask data.Type: ApplicationFiled: September 30, 2020Publication date: March 31, 2022Inventors: Nishant Puri, Sakthivel Sivaraman, Rajath Shetty, Niranjan Avadhanam
-
Publication number: 20210182625Abstract: Systems and methods for more accurate and robust determination of subject characteristics from an image of the subject. One or more machine learning models receive as input an image of a subject, and output both facial landmarks and associated confidence values. Confidence values represent the degrees to which portions of the subject's face corresponding to those landmarks are occluded, i.e., the amount of uncertainty in the position of each landmark location. These landmark points and their associated confidence values, and/or associated information, may then be input to another set of one or more machine learning models which may output any facial analysis quantity or quantities, such as the subject's gaze direction, head pose, drowsiness state, cognitive load, or distraction state.Type: ApplicationFiled: August 27, 2020Publication date: June 17, 2021Inventors: Nuri Murat Arar, Niranjan Avadhanam, Nishant Puri, Shagan Sah, Rajath Shetty, Sujay Yadawadkar, Pavlo Molchanov
-
Publication number: 20210182609Abstract: Systems and methods for determining the gaze direction of a subject and projecting this gaze direction onto specific regions of an arbitrary three-dimensional geometry. In an exemplary embodiment, gaze direction may be determined by a regression-based machine learning model. The determined gaze direction is then projected onto a three-dimensional map or set of surfaces that may represent any desired object or system. Maps may represent any three-dimensional layout or geometry, whether actual or virtual. Gaze vectors can thus be used to determine the object of gaze within any environment. Systems can also readily and efficiently adapt for use in different environments by retrieving a different set of surfaces or regions for each environment.Type: ApplicationFiled: August 28, 2020Publication date: June 17, 2021Inventors: Nuri Murat Arar, Hairong Jiang, Nishant Puri, Rajath Shetty, Niranjan Avadhanam