Patents by Inventor Niranjan Avadhanam

Niranjan Avadhanam has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20250259459
    Abstract: In various examples, estimated field of view or gaze information of a user may be projected external to a vehicle and compared to vehicle perception information corresponding to an environment outside of the vehicle. As a result, interior monitoring of a driver or occupant of the vehicle may be used to determine whether the driver or occupant has processed or seen certain object types, environmental conditions, or other information exterior to the vehicle. For a more holistic understanding of the state of the user, attentiveness and/or cognitive load of the user may be monitored to determine whether one or more actions should be taken. As a result, notifications, AEB system activations, and/or other actions may be determined based on a more complete state of the user as determined based on cognitive load, attentiveness, and/or a comparison between external perception of the vehicle and estimated perception of the user.
    Type: Application
    Filed: April 28, 2025
    Publication date: August 14, 2025
    Inventors: Nuri Murat Arar, Niranjan Avadhanam, Yuzhuo Ren
  • Publication number: 20250252594
    Abstract: In various examples, one or more validity checks that model one or more aspects of human physiology may be applied to frames of detected human features to detect and respond to the presence of faults. Example validity checks include human feature constraints derived from the kinematics of human motion, anatomical and spatial constraints, consistency across detection modalities, and/or others. The present techniques may be utilized to validate human features detected by various computer vision tasks, such as those involving pose estimation, facial detection, gesture recognition, and/or activity monitoring, to name a few examples.
    Type: Application
    Filed: February 7, 2024
    Publication date: August 7, 2025
    Inventors: Richard Bramley, Niranjan Avadhanam, Nishant Puri, Rajath Shetty, Prashant Chandrasekhar
  • Patent number: 12382180
    Abstract: Stitching of multiple images into a composite representation can be performed using a set of stitching parameters determined based, at least in part, upon a subjective stitching quality assessment value. A stitched image can be compared against its constituent images to obtain one or more objective quality metrics. These objective quality metrics can be fed, as input, to a trained classifier, which can infer a subjective quality assessment metric for the stitched (or otherwise composited) image. This subjective quality assessment metric can be used to adjust one or more compositing parameter values in order to provide at least a minimum subjective quality assessment value for composited images.
    Type: Grant
    Filed: July 20, 2021
    Date of Patent: August 5, 2025
    Assignee: Nvidia Corporation
    Inventors: Yuzhuo Ren, Niranjan Avadhanam
  • Publication number: 20250222934
    Abstract: In various examples, circadian rhythm-based data augmentation for drowsiness detection systems and applications are provided. Embodiments described herein may produce an estimated circadian rhythm for a test subject and/or vehicle driver or other machine operator or occupant, and use the pattern of that circadian rhythm to correct, confirm, calibrate, or otherwise augment drowsiness assessments derived from video image data. The position of a person in the context of their process C circadian cycle may be used as indication of their level of drowsiness. An estimated process C circadian cycle may be used to generate more accurate ground truth training data for training machine learning models, and may be used by real-time, in-vehicle drowsiness detection systems that infer driver drowsiness levels based on captured images. In various embodiments, a circadian rhythm drowsiness estimate may be used to correct, calibrate, augment, and/or replace a drowsiness score predicted by a machine learning model.
    Type: Application
    Filed: January 9, 2024
    Publication date: July 10, 2025
    Inventors: Yuzhuo REN, Niranjan AVADHANAM, Varsha HEDAU, Zhengmin ZHANG, Shelly GOEL
  • Publication number: 20250221647
    Abstract: In various examples, circadian rhythm-based data augmentation for drowsiness detection systems and applications are provided. Embodiments described herein may produce an estimated circadian rhythm for a test subject and/or vehicle driver or other machine operator or occupant, and use the pattern of that circadian rhythm to correct, confirm, calibrate, or otherwise augment drowsiness assessments derived from video image data. The position of a person in the context of their process C circadian cycle may be used as indication of their level of drowsiness. An estimated process C circadian cycle may be used to generate more accurate ground truth training data for training machine learning models, and may be used by real-time, in-vehicle drowsiness detection systems that infer driver drowsiness levels based on captured images. In various embodiments, a circadian rhythm drowsiness estimate may be used to correct, calibrate, augment, and/or replace a drowsiness score predicted by a machine learning model.
    Type: Application
    Filed: January 9, 2024
    Publication date: July 10, 2025
    Inventors: Yuzhuo REN, Niranjan AVADHANAM, Varsha HEDAU, Zhengmin ZHANG, Shelly GOEL
  • Patent number: 12351119
    Abstract: Systems and methods are disclosed herein for implementation of a vehicle command operation system that may use multi-modal technology to authenticate an occupant of the vehicle to authorize a command and receive natural language commands for vehicular operations. The system may utilize sensors to receive data indicative of a voice command from an occupant of the vehicle. The system may receive second sensor data to aid in the determination of the corresponding vehicular operation in response to the received command. The system may retrieve authentication data for the occupants of the vehicle. The system authenticates the occupant to authorize a vehicular operation command using a neural network based on at least one of the first sensor data, the second sensor data, and the authentication data. Responsive to the authentication, the system may authorize the operation to be performed in the vehicle based on the vehicular operation command.
    Type: Grant
    Filed: December 6, 2022
    Date of Patent: July 8, 2025
    Assignee: NVIDIA Corporation
    Inventors: Sumit Bhattacharya, Jason Conrad Roche, Niranjan Avadhanam
  • Publication number: 20250191466
    Abstract: In various examples, audio alerts of emergency response vehicles may be detected and classified using audio captured by microphones of an autonomous or semi-autonomous machine in order to identify travel directions, locations, and/or types of emergency response vehicles in the environment. For example, a plurality of microphone arrays may be disposed on an autonomous or semi-autonomous machine and used to generate audio signals corresponding to sounds in the environment. These audio signals may be processed to determine a location and/or direction of travel of an emergency response vehicle (e.g., using triangulation). Additionally, to identify siren types—and thus emergency response vehicle types corresponding thereto—the audio signals may be used to generate representations of a frequency spectrum that may be processed using a deep neural network (DNN) that outputs probabilities of alert types being represented by the audio data.
    Type: Application
    Filed: January 29, 2025
    Publication date: June 12, 2025
    Inventors: Ambrish Dantrey, Atousa Torabi, Anshul Jain, Ram Ganapathi, Abhijit Patait, Revanth Reddy Nalla, Niranjan Avadhanam
  • Patent number: 12327413
    Abstract: In various examples, color statistic(s) from ground projections are used to harmonize color between reference and target frames representing an environment. The reference and target frames may be projected onto a representation of the ground (e.g., a ground plane) of the environment, an overlapping region between the projections may be identified, and the portion of each projection that lands in the overlapping region may be taken as a corresponding ground projection. Color statistics (e.g., mean, variance, standard deviation, kurtosis, skew, correlation(s) between color channels) may be computed from the ground projections (or a portion thereof, such as a majority cluster) and used to modify the colors of the target frame to have updated color statistics that match those from the ground projection of the reference frame, thereby harmonizing color across the reference and target frames.
    Type: Grant
    Filed: October 4, 2022
    Date of Patent: June 10, 2025
    Assignee: NVIDIA Corporation
    Inventors: Yuzhuo Ren, Dawid Stanislaw Pajak, Niranjan Avadhanam, Guangli Dai
  • Publication number: 20250170958
    Abstract: Systems and methods for a self-adjusting vehicle mirror. The mirror automatically locates the face of the driver or another passenger, and orients the mirror to provide the driver/passenger face with a desired view from the mirror. The mirror may continue to reorient itself as the driver or passenger shifts position, to continuously provide a desired field of view even as he or she changes position over time. In certain embodiments, the mirror system of the disclosure can be a self-contained system, with the mirror, mirror actuator, camera, and computing device all contained within the mirror housing as a single integrated unit.
    Type: Application
    Filed: January 27, 2025
    Publication date: May 29, 2025
    Inventors: Feng Hu, Niranjan Avadhanam, Ratin Kumar, Simon John Baker
  • Patent number: 12314854
    Abstract: Systems and methods for determining the gaze direction of a subject and projecting this gaze direction onto specific regions of an arbitrary three-dimensional geometry. In an exemplary embodiment, gaze direction may be determined by a regression-based machine learning model. The determined gaze direction is then projected onto a three-dimensional map or set of surfaces that may represent any desired object or system. Maps may represent any three-dimensional layout or geometry, whether actual or virtual. Gaze vectors can thus be used to determine the object of gaze within any environment. Systems can also readily and efficiently adapt for use in different environments by retrieving a different set of surfaces or regions for each environment.
    Type: Grant
    Filed: April 10, 2023
    Date of Patent: May 27, 2025
    Assignee: NVIDIA Corporation
    Inventors: Nuri Arar Murat, Hairong Jiang, Nishant Puri, Rajath Shetty, Niranjan Avadhanam
  • Publication number: 20250156717
    Abstract: Apparatuses, systems, and techniques are described to determine locations of objects using images including digital representations of those objects. In at least one embodiment, a gaze of one or more occupants of a vehicle is determined independently of a location of one or more sensors used to detect those occupants.
    Type: Application
    Filed: January 15, 2025
    Publication date: May 15, 2025
    Inventors: Feng Hu, Niranjan Avadhanam, Yuzhuo Ren, Sujay Yadawadkar, Sakthivel Sivaraman, Hairong Jiang, Siyue Wu
  • Publication number: 20250148810
    Abstract: In various examples, systems and methods are disclosed herein for a vehicle command operation system that may use technology across multiple modalities to cause vehicular operations to be performed in response to determining a focal point based on a gaze of an occupant. The system may utilize sensors to receive first data indicative of an eye gaze of an occupant of the vehicle. The system may utilize sensors to receive second data indicative of other data from the occupant. The system may then calculate a gaze vector based on the data indicative of the eye gaze of the occupant. The system may determine a focal point based on the gaze vector. In response to determining the focal point, the system causes an operation to be performed in the vehicle based on the second data.
    Type: Application
    Filed: January 13, 2025
    Publication date: May 8, 2025
    Inventors: Jason Conrad Roche, Niranjan Avadhanam
  • Publication number: 20250142208
    Abstract: In various examples, an image processing pipeline may switch between different operating or switching modes based on speed of ego-motion and/or the active gear (e.g., park vs. drive) of a vehicle or other ego-machine in which an RGB/IR camera is being used. For example, a first operating or switching mode that toggles between IR and RGB imaging modes at a fixed frame rate or interval may be used when the vehicle is in motion, in a particular gear (e.g., drive), and/or traveling above a threshold speed. In another example, a second operating or switching mode that toggles between IR and RGB imaging modes based on detected light intensity may be used when the vehicle is in stationary, in park (or out of gear), and/or traveling below a threshold speed.
    Type: Application
    Filed: October 25, 2023
    Publication date: May 1, 2025
    Inventors: Sakthivel SIVARAMAN, Rajath SHETTY, Animesh KHEMKA, Niranjan AVADHANAM
  • Publication number: 20250136130
    Abstract: Various embodiments of the present disclosure relate to operator assistance based on extracting natural language characters from one or more sensed objects. For instance, particular embodiments may generate a natural language utterance based on extracting natural language text in a nearby traffic sign. In an illustrative example, particular embodiments may detect, via object detection and within image data, one or more regions of the image data depicting the traffic sign. Particular embodiments can then extract one or more first natural language characters represented in the traffic sign based at least on performing optical character recognition within the one or more regions of the image data in response to detecting the one or more regions of the image data depicting the traffic sign.
    Type: Application
    Filed: November 1, 2023
    Publication date: May 1, 2025
    Inventors: Rajath SHETTY, Ratin KUMAR, Niral Lalit PATHAK, Niranjan AVADHANAM
  • Publication number: 20250136134
    Abstract: Various embodiments of the present disclosure relate to operator assistance based on operator monitoring. For instance, during long drives, a driver may become drowsy or may not otherwise be alert. As such, particular embodiments have the capability of starting a conversation with the driver based on driver interests and/or detecting that the driver is getting drowsy. In an illustrative example, a Driver Monitoring System (DMS) camera of a vehicle may employ a component that derives pixel-level information showing head nodding, hands dropping, or the like. Based on image pattern characteristics in the image data, particular embodiments generate a score representing an alertness level. A representation of the alertness level can be provided as input to a machine learning model so that the model may generate a suitable natural language or other response, such as starting a conversation with personalized trivia, sending a control signal to honk a horn, or the like.
    Type: Application
    Filed: November 1, 2023
    Publication date: May 1, 2025
    Inventors: Rajath SHETTY, Ratin KUMAR, Niral Lalit PATHAK, Niranjan AVADHANAM
  • Patent number: 12288403
    Abstract: In various examples, estimated field of view or gaze information of a user may be projected external to a vehicle and compared to vehicle perception information corresponding to an environment outside of the vehicle. As a result, interior monitoring of a driver or occupant of the vehicle may be used to determine whether the driver or occupant has processed or seen certain object types, environmental conditions, or other information exterior to the vehicle. For a more holistic understanding of the state of the user, attentiveness and/or cognitive load of the user may be monitored to determine whether one or more actions should be taken. As a result, notifications, AEB system activations, and/or other actions may be determined based on a more complete state of the user as determined based on cognitive load, attentiveness, and/or a comparison between external perception of the vehicle and estimated perception of the user.
    Type: Grant
    Filed: March 20, 2024
    Date of Patent: April 29, 2025
    Assignee: NVIDIA Corporation
    Inventors: Nuri Murat Arar, Niranjan Avadhanam, Yuzhuo Ren
  • Patent number: 12283187
    Abstract: In various examples, audio alerts of emergency response vehicles may be detected and classified using audio captured by microphones of an autonomous or semi-autonomous machine in order to identify travel directions, locations, and/or types of emergency response vehicles in the environment. For example, a plurality of microphone arrays may be disposed on an autonomous or semi-autonomous machine and used to generate audio signals corresponding to sounds in the environment. These audio signals may be processed to determine a location and/or direction of travel of an emergency response vehicle (e.g., using triangulation). Additionally, to identify siren types—and thus emergency response vehicle types corresponding thereto—the audio signals may be used to generate representations of a frequency spectrum that may be processed using a deep neural network (DNN) that outputs probabilities of alert types being represented by the audio data.
    Type: Grant
    Filed: September 6, 2023
    Date of Patent: April 22, 2025
    Assignee: NVIDIA Corporation
    Inventors: Ambrish Dantrey, Atousa Torabi, Anshul Jain, Ram Ganapathi, Abhijit Patait, Revanth Reddy Nalla, Niranjan Avadhanam
  • Publication number: 20250121843
    Abstract: Approaches presented herein provide for the automated determination of a level of impairment of a person, as may be relevant to the performance of a task. A light and camera-based system can be used to determine factors such as gaze nystagmus that are indicative of inebriation or impairment. A test system can simulate motion of a light using a determined pattern, and capture image data of at least the eye region of a person attempting to follow the motion. The captured image data can be analyzed using a neural network to infer at least one behavior of the user, and the behavior determination(s) can be used to determine a capacity or level of impairment of a user. An appropriate action can be taken, such as to allow a person with full capacity to operate a vehicle or perform a task, or to block access to such operation or performance if the person is determined to be impaired beyond an allowable amount.
    Type: Application
    Filed: October 11, 2023
    Publication date: April 17, 2025
    Inventors: Niranjan Avadhanam, Yuzhuo Ren
  • Publication number: 20250124734
    Abstract: Interactions with virtual systems may be difficult when users inadvertently fail to provide sufficient information to proceed with their requests. Certain types of inputs, such as auditory inputs, may lack sufficient information to properly provide a response to the user. Additional information, such as image data, may enable user gestures or poses to supplement the auditory inputs to enable response generation without requesting additional information from users.
    Type: Application
    Filed: December 23, 2024
    Publication date: April 17, 2025
    Inventors: Sakthivel Sivaraman, Nishant Puri, Yuzhuo Ren, Atousa Torabi, Shubhadeep Das, Niranjan Avadhanam, Sumit Kumar Bhattacharya, Jason Roche
  • Publication number: 20250117980
    Abstract: In various examples, infrared image data (e.g., frames of an infrared video feed) may be colorized by applying the infrared image data and/or a corresponding edge map to a generator of a generative adversarial network (GAN). The GAN may be trained with or without paired ground truth RGB and infrared (and/or edge map) images. In an example of the latter scenario, a first generator G(IR)?RGB and a second generator G(RGB)?IR may be trained in a first chain, their positions may be swapped in a second chain, and the second chain may be trained. In some embodiments, edges may be emphasized by weighting edge pixels (e.g., determined from a corresponding edge map) higher than non-edge pixels when backpropagating loss. After training, G(IR)?RGB may be used to generate RGB image data from infrared image data (and/or a corresponding edge map).
    Type: Application
    Filed: October 10, 2023
    Publication date: April 10, 2025
    Inventors: Yuzhuo REN, Niranjan AVADHANAM