Patents by Inventor Ritwik Giri

Ritwik Giri has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 12272371
    Abstract: Real-time audio enhancement for a target speaker may be performed. An embedding of a sample of speaker audio is created using a trained neural network that performs voice identification. The embedding is then concatenated with the input features of a trained machine learning model for audio enhancement. The audio enhancement model can recognize and enhance a target speaker's speech in a real-time implementation, as the embedding is in the same feature space of the audio enhancement model.
    Type: Grant
    Filed: June 30, 2021
    Date of Patent: April 8, 2025
    Assignee: Amazon Technologies, Inc.
    Inventors: Ritwik Giri, Shrikant Venkataramani, Jean-Marc Valin, Mehmet Umut Isik, Arvindh Krishnaswamy
  • Publication number: 20250111857
    Abstract: Examples herein provide an approach to enhance an audio mixture of a teleconference application by switching between noise suppression modes using a single model. Specifically, a machine learning (ML) model may be configured to, in response to receiving an audio mixture representation as input, suppress either a background noise of the audio mixture or suppress all noise of the audio mixture except a user's voice. In some examples, the ML model may be trained on speech and background noise training data during a training phase. In addition, the ML model may be trained on a user's voice during an enrollment phase. In addition, during an inference phase, the ML model may enhance the audio mixture by suppressing a portion of the audio mixture.
    Type: Application
    Filed: September 29, 2023
    Publication date: April 3, 2025
    Inventors: Ritwik Giri, Zhepei Wang, Devansh Shah, Jean-Marc Valin, Michael Mark Goodwin
  • Patent number: 12236968
    Abstract: A system comprises an ear-worn electronic device configured to be worn by a wearer. The ear-worn electronic device comprises a processor and memory coupled to the processor. The memory is configured to store an annoying sound dictionary representative of a plurality of annoying sounds pre-identified by the wearer. A microphone is coupled to the processor and configured to monitor an acoustic environment of the wearer. A speaker or a receiver is coupled to the processor. The processor is configured to identify different background noises present in the acoustic environment, determine which of the background noises correspond to one or more of the plurality of annoying sounds, and attenuate the one or more annoying sounds in an output signal provided to the speaker or receiver.
    Type: Grant
    Filed: December 1, 2023
    Date of Patent: February 25, 2025
    Assignee: STARKEY LABORATORIES, INC.
    Inventors: Ritwik Giri, Karim Helwani, Tao Zhang
  • Patent number: 12205039
    Abstract: A group masked autoencoder may be implemented for anomaly detection. An autoencoder network model may be trained without supervision and applied to output an estimated joint probability distribution of normality for a group of frames of time series data. The estimated joint probability distribution may be used to determine an anomaly score for the time series data. An anomaly may be detected according to the anomaly score and a result that indicates a detected anomaly may be provided.
    Type: Grant
    Filed: November 2, 2020
    Date of Patent: January 21, 2025
    Assignee: Amazon Technologies, Inc.
    Inventors: Ritwik Giri, Srikanth Venkata Tenneti, Karim Helwani, Fangzhou Cheng, Mehmet Umut Isik, Arvindh Krishnaswamy
  • Patent number: 12175434
    Abstract: Systems, methods, and apparatuses for detecting anomalies using clusters are described. In some examples, a method includes receiving a request to perform anomaly detection using a plurality of clusters; receiving a data point; determining when the received data point is a part of one of the plurality of clusters utilizing a distance to centers of the one or more clusters, wherein: when the received data point is determined to belong to a normal cluster, assigning the received data point to the determined cluster, updating the cluster, and updating a history for the cluster, when the received data point is determined to belong to an anomalous cluster, raising an anomaly, updating the cluster, and updating a history for the cluster, and when the received data point is determined to not belong to any cluster, raising an anomaly.
    Type: Grant
    Filed: September 30, 2020
    Date of Patent: December 24, 2024
    Assignee: Amazon Technologies, Inc.
    Inventors: Srikanth Venkata Tenneti, Arvindh Krishnaswamy, Karim Helwani, Mehmet Umut Isik, Ritwik Giri, Fangzhou Cheng, Aparna Pandey
  • Patent number: 12014748
    Abstract: Techniques for training and using a machine learning model for estimation of reverberation in a multi-task learning framework are described. According to some embodiments, the multi-task learning framework improves the performance of the machine learning model by estimating the amount of reverberation present in an input audio recording as a secondary task to the primary task of generating a clean speech portion of the input audio recording. In one embodiment, a model architecture is selected that takes a noisy reverberant recording as an input and outputs an estimate of a clean (e.g., de-reverberated) signal, an estimate of noise (e.g., background noise), and an estimate of the reverb only portion, with the secondary task of estimating the reverb only portion acting as a regularizer that improves the machine learning model's performance in enhancing the reverberant (e.g., and noisy) input speech.
    Type: Grant
    Filed: August 7, 2020
    Date of Patent: June 18, 2024
    Assignee: Amazon Technologies, Inc.
    Inventors: Ritwik Giri, Mehmet Umut Isik, Neerad Dilip Phansalkar, Jean-Marc Valin, Karim Helwani, Arvindh Krishnaswamy
  • Patent number: 12008457
    Abstract: Audio processing may be performed with a convolutional neural network that includes positional embeddings. Audio data may be received at an audio processing system. A convolutional neural network that concatenates frequency-positional embeddings at an input layer may be used to process the audio data. A result of processing the audio data through the convolutional neural network may be used to perform an audio processing task.
    Type: Grant
    Filed: September 29, 2020
    Date of Patent: June 11, 2024
    Assignee: Amazon Technologies, Inc.
    Inventors: Mehmet Umut Isik, Ritwik Giri, Neerad Dilip Phansalkar, Jean-Marc Valin, Karim Helwani, Arvindh Krishnaswamy
  • Publication number: 20240127840
    Abstract: A system comprises an ear-worn electronic device configured to be worn by a wearer. The ear-worn electronic device comprises a processor and memory coupled to the processor. The memory is configured to store an annoying sound dictionary representative of a plurality of annoying sounds pre-identified by the wearer. A microphone is coupled to the processor and configured to monitor an acoustic environment of the wearer. A speaker or a receiver is coupled to the processor. The processor is configured to identify different background noises present in the acoustic environment, determine which of the background noises correspond to one or more of the plurality of annoying sounds, and attenuate the one or more annoying sounds in an output signal provided to the speaker or receiver.
    Type: Application
    Filed: December 1, 2023
    Publication date: April 18, 2024
    Inventors: Ritwik Giri, Karim Helwani, Tao Zhang
  • Publication number: 20240096346
    Abstract: A plurality of talker embedding vectors may be derived that correspond to a plurality of talkers in an input audio stream. Each talker embedding vector may represent respective voice characteristics of a respective talker. The talker embedding vectors may be generated based on, for example, a pre-enrollment process or a cluster-based embedding vector derivation process. A plurality of instances of a personalized noise suppression model may be executed on the input audio stream. Each instance of the personalized noise suppression model may employ a respective talker embedding vector. A plurality of single-talker audio streams may be generated by the plurality of instances of the personalized noise suppression model. A plurality of single-talker transcriptions may be generated based on the plurality of single-talker audio streams. The plurality of single-talker transcriptions may be merged into a multi-talker output transcription.
    Type: Application
    Filed: June 27, 2022
    Publication date: March 21, 2024
    Inventors: Masahito Togami, Ritwik Giri, Michael Mark Goodwin, Arvindh . Krishnaswamy, Siddhartha Shankara Rao
  • Patent number: 11875812
    Abstract: A system comprises an ear-worn electronic device configured to be worn by a wearer. The ear-worn electronic device comprises a processor and memory coupled to the processor. The memory is configured to store an annoying sound dictionary representative of a plurality of annoying sounds pre-identified by the wearer. A microphone is coupled to the processor and configured to monitor an acoustic environment of the wearer. A speaker or a receiver is coupled to the processor. The processor is configured to identify different background noises present in the acoustic environment, determine which of the background noises correspond to one or more of the plurality of annoying sounds, and attenuate the one or more annoying sounds in an output signal provided to the speaker or receiver.
    Type: Grant
    Filed: July 20, 2022
    Date of Patent: January 16, 2024
    Assignee: STARKEY LABORATORIES, INC.
    Inventors: Ritwik Giri, Karim Helwani, Tao Zhang
  • Publication number: 20230421973
    Abstract: A method, comprising receiving at least one sound at an electronic device. The at least one sound is enhanced for the at least one user based on a compound metric. The compound metric is calculated using at least two sound metrics selected from an engineering metric, a perceptual metric, and a physiological metric. The engineering metric comprises a difference between an output signal and a desired signal. At least one of the perceptual metric and the physiological metric is based at least in part on input sensed from the at least one user in response to the received at least one sound.
    Type: Application
    Filed: September 12, 2023
    Publication date: December 28, 2023
    Inventors: Joao Felipe Santos, Tao Zhang, Yan Zhao, Buye Xu, Ritwik Giri
  • Patent number: 11812223
    Abstract: A method, comprising receiving at least one sound at an electronic device. The at least one sound is enhanced for the at least one user based on a compound metric. The compound metric is calculated using at least two sound metrics selected from an engineering metric, a perceptual metric, and a physiological metric. The engineering metric comprises a difference between an output signal and a desired signal. At least one of the perceptual metric and the physiological metric is based at least in part on input sensed from the at least one user in response to the received at least one sound.
    Type: Grant
    Filed: June 7, 2022
    Date of Patent: November 7, 2023
    Assignee: STARKEY LABORATORIES, INC.
    Inventors: Joao Felipe Santos, Tao Zhang, Yan Zhao, Buye Xu, Ritwik Giri
  • Patent number: 11545134
    Abstract: Techniques for the generation of dubbed audio for an audio/video are described.
    Type: Grant
    Filed: December 10, 2019
    Date of Patent: January 3, 2023
    Assignee: Amazon Technologies, Inc.
    Inventors: Marcello Federico, Robert Enyedi, Yaser Al-Onaizan, Roberto Barra-Chicote, Andrew Paul Breen, Ritwik Giri, Mehmet Umut Isik, Arvindh Krishnaswamy, Hassan Sawaf
  • Publication number: 20220392473
    Abstract: A system comprises an ear-worn electronic device configured to be worn by a wearer. The ear-worn electronic device comprises a processor and memory coupled to the processor. The memory is configured to store an annoying sound dictionary representative of a plurality of annoying sounds pre-identified by the wearer. A microphone is coupled to the processor and configured to monitor an acoustic environment of the wearer. A speaker or a receiver is coupled to the processor. The processor is configured to identify different background noises present in the acoustic environment, determine which of the background noises correspond to one or more of the plurality of annoying sounds, and attenuate the one or more annoying sounds in an output signal provided to the speaker or receiver.
    Type: Application
    Filed: July 20, 2022
    Publication date: December 8, 2022
    Inventors: Ritwik Giri, Karim Helwani, Tao Zhang
  • Patent number: 11521637
    Abstract: Post-filtering may be performed for ratio masks as part of audio enhancement. Audio data may be received. A machine learning model may be applied to generate gain values for different spectrum bands of the audio data. The gain values may then be modified using an envelope post-filter according to a monotonically increasing function applied to the gain values to produce modified gain values used to generate an enhanced version of the audio data.
    Type: Grant
    Filed: September 29, 2020
    Date of Patent: December 6, 2022
    Assignee: Amazon Technologies, Inc.
    Inventors: Jean-Marc Valin, Mehmet Umut Isik, Neerad Dilip Phansalkar, Ritwik Giri, Karim Helwani, Arvindh Krishnaswamy
  • Publication number: 20220369046
    Abstract: A method, comprising receiving at least one sound at an electronic device. The at least one sound is enhanced for the at least one user based on a compound metric. The compound metric is calculated using at least two sound metrics selected from an engineering metric, a perceptual metric, and a physiological metric. The engineering metric comprises a difference between an output signal and a desired signal. At least one of the perceptual metric and the physiological metric is based at least in part on input sensed from the at least one user in response to the received at least one sound.
    Type: Application
    Filed: June 7, 2022
    Publication date: November 17, 2022
    Inventors: Joao Felipe Santos, Tao Zhang, Yan Zhao, Buye Xu, Ritwik Giri
  • Patent number: 11423922
    Abstract: A system comprises an ear-worn electronic device configured to be worn by a wearer. The ear-worn electronic device comprises a processor and memory coupled to the processor. The memory is configured to store an annoying sound dictionary representative of a plurality of annoying sounds pre-identified by the wearer. A microphone is coupled to the processor and configured to monitor an acoustic environment of the wearer. A speaker or a receiver is coupled to the processor. The processor is configured to identify different background noises present in the acoustic environment, determine which of the background noises correspond to one or more of the plurality of annoying sounds, and attenuate the one or more annoying sounds in an output signal provided to the speaker or receiver.
    Type: Grant
    Filed: December 17, 2020
    Date of Patent: August 23, 2022
    Assignee: Starkey Laboratories, Inc.
    Inventors: Ritwik Giri, Karim Helwani, Tao Zhang
  • Patent number: 11363390
    Abstract: A method, comprising receiving at least one sound at an electronic device. The at least one sound is enhanced for the at least one user based on a compound metric. The compound metric is calculated using at least two sound metrics selected from an engineering metric, a perceptual metric, and a physiological metric. The engineering metric comprises a difference between an output signal and a desired signal. At least one of the perceptual metric and the physiological metric is based at least in part on input sensed from the at least one user in response to the received at least one sound.
    Type: Grant
    Filed: October 19, 2020
    Date of Patent: June 14, 2022
    Assignee: Starkey Laboratories, Inc.
    Inventors: Joao Felipe Santos, Tao Zhang, Yan Zhao, Buye Xu, Ritwik Giri
  • Publication number: 20220100187
    Abstract: Systems, methods, and apparatuses for providing an estimation of remaining useful life are described. In some examples, a method for providing an estimation of remaining useful life includes receiving a request to determine a remaining useful life of a managed device before maintenance using a trained machine learning model; receiving sensor data from the managed device; applying the trained machine learning model to the received information to generate a prediction of a time to failure of the managed device and a confidence interval for that prediction; and providing the prediction of a time to failure and confidence interval for that prediction to a requester.
    Type: Application
    Filed: September 30, 2020
    Publication date: March 31, 2022
    Inventors: Mehmet Umut ISIK, Oleksandr GLADKOV, Ritwik GIRI, Arvindh KRISHNASWAMY
  • Publication number: 20220101193
    Abstract: Systems, methods, and apparatuses for selecting a model are described. In some examples, a method of selecting a model includes receiving a request to perform model selection; evaluating a plurality of models to select model by: generating a plurality of metrics for each of the trained plurality of models, the plurality of metrics including at least two of an forewarning time metric of how much in advance of a failure an alert can be raised by the model, event recall metric of how many failure events were alerted to in advance of failure, an event precision metric of a ratio of true and false positives, and an area under a receiver operating characteristic (ROC) curve, calculating, for each of the trained plurality of models, a weighted harmonic mean from the at least two metrics, and selecting one of plurality of models based on the calculated weighted harmonic means; and generating and providing a report regarding the selected trained model.
    Type: Application
    Filed: September 30, 2020
    Publication date: March 31, 2022
    Inventors: Karim HELWANI, Srikanth Venkata TENNETI, Arvindh KRISHNASWAMY, Ritwik GIRI, Mehmet Umut ISIK, Aparna PANDEY, Fangzhou CHENG