Patents by Inventor Jonathan Le

Jonathan Le has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11978476
    Abstract: A system and method for detecting anomalous sound are disclosed. The method includes receiving a spectrogram of an audio signal with elements defined by values in a time-frequency domain of the spectrogram. Each of the values corresponds to an element of the spectrogram that is identified by a coordinate in the time-frequency domain. The time-frequency domain of the spectrogram is partitioned into a context region and a target region. The context region and the target region are processed by a neural network using an attentive neural process to recover values of the spectrogram for elements with coordinates in the target region. The recovered values of the elements of the target region are compared with values of elements of the partitioned target region. An anomaly score is determined based on the comparison. The anomaly score is used for performing a control action.
    Type: Grant
    Filed: September 19, 2021
    Date of Patent: May 7, 2024
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Gordon Wichern, Ankush Chakrabarty, Zhong-Qiu Wang, Jonathan Le Roux
  • Patent number: 11978435
    Abstract: This invention relates generally to speech processing and more particularly to end-to-end automatic speech recognition (ASR) that utilizes long contextual information. Some embodiments of the invention provide a system and a method for end-to-end ASR suitable for recognizing long audio recordings such as lecture and conversational speeches. This disclosure includes a Transformer-based ASR system that utilizes contextual information, wherein the Transformer accepts multiple utterances at the same time and predicts transcript for the last utterance. This is repeated in a sliding-window fashion with one-utterance shifts to recognize the entire recording. In addition, some embodiments of the present invention may use acoustic and/or text features obtained from only the previous utterances spoken by the same speaker as the last utterance when the long audio recording includes multiple speakers.
    Type: Grant
    Filed: October 13, 2020
    Date of Patent: May 7, 2024
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Takaaki Hori, Niko Moritz, Chiori Hori, Jonathan Le Roux
  • Patent number: 11969258
    Abstract: A surgical system includes at least one light emitter generating light at a first intensity and an array of light sensors including a least one row of light sensors, individual light sensors in the row of light sensors adapted to generate a signal including a non-pulsatile component. The system also includes a controller coupled to the array of light sensors, the controller including an analyzer to determine the magnitudes of the non-pulsatile components at the individual light sensors in the row of light sensors, to determine if the non-pulsatile component transitions from a higher magnitude to a lower magnitude and from a lower magnitude to a higher magnitude, and if so, to determine if the first intensity should be changed to a second intensity.
    Type: Grant
    Filed: July 17, 2020
    Date of Patent: April 30, 2024
    Assignee: Briteseed, LLC
    Inventors: Amal Chaturvedi, Hariharan Subramanian, Jonathan Gunn, Mayank Vijayvergia, Shetha Shukair, Paul Le Rolland
  • Publication number: 20240111055
    Abstract: There is provided continuous wave time of flight, CW-ToF, camera system comprising: one or more lasers for outputting laser light; one or more imaging sensors, the one or more image sensors each comprising a plurality of imaging pixels for accumulating charge based on incident light comprising reflected laser light off a first surface of an object; and a distance determination system coupled to the one or more imaging sensors and configured to: acquire a first set of charge samples from the one or more imaging sensors in respect of the object by: a) driving the one or more lasers to output laser light modulated with a first modulation signal, wherein the first modulation signal has a first frequency; and b) after step a, reading out image sensor values indicative of charge accumulated by at least some of the plurality of imaging pixels of the one or more imaging sensors; acquire a second set of charge samples from the one or more imaging sensors in respect of the object by: c) driving the one or more lasers t
    Type: Application
    Filed: September 30, 2022
    Publication date: April 4, 2024
    Inventors: Javier CALPE-MARAVILLA, Filiberto Pla, Jonathan Hurwitz, Nicolas Le Dortz
  • Patent number: 11936786
    Abstract: Provided is novel technology for secure security data transmission and more particularly for registering network-enabled security devices such as IP cameras to a security server over a public network such as to a cloud-based security service. An enrolment server is provided that is logged into using a computing device to request and receive an activation code for the security device. The activation code is then provided to the security device, e.g. directly by the computing device. The Security device authenticates itself based on the activation code and in one example provides a public key that will be used to verify its registration. Data transmissions by the device are secured in part on the basis of its registration.
    Type: Grant
    Filed: June 15, 2022
    Date of Patent: March 19, 2024
    Inventors: Jonathan Doyon, Simon Le Bourdais-Cabana, Sebastien Nadeau, Siaka Baro, Martin Tardif
  • Publication number: 20240055012
    Abstract: A system and method for reverberation reduction is disclosed. A first Deep Neural Network (DNN) produces a first estimate of a target direct-path signal from a mixture of acoustic signals that include the target direct-path signal and a reverberation of the target direct-path signal. A filter modeling a room impulse response (RIR) for the first estimate is estimated. The filter when applied to the first estimate of the target direct-path signal generates a result closest to a residual between the mixture of the acoustic signals and the first estimate of the target direct-path signal according to a distance function. The estimated filter is used for modeling the RIR.
    Type: Application
    Filed: August 15, 2022
    Publication date: February 15, 2024
    Inventors: Zhong-Qiu Wang, Gordon Wichern, Jonathan Le Roux
  • Publication number: 20240046085
    Abstract: An artificial intelligence (AI) low-latency processing system is provided. The low-latency processing system includes a processor; and a memory having instructions stored thereon. The low-latency processing system is configured to collect a sequence of frames jointly including information dispersed among at least some frames in the sequence of frames, execute a timing neural network trained to identify an early subsequence of frames in the sequence of frames including at least a portion of the information indicative of the information, and execute a decoding neural network trained to decode the information from the portion of the information in the subsequence of frames, wherein the timing neural network is jointly trained with the decoding neural network to iteratively identify the smallest number of subframes from the beginning of a training sequence of frames containing a portion of training information sufficient to decode the training information.
    Type: Application
    Filed: August 4, 2022
    Publication date: February 8, 2024
    Applicant: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Chiori Hori, Jonathan Le Roux, Anoop Cherian, 02139 Marks
  • Publication number: 20240003737
    Abstract: A system and a method for detecting anomalous sound are disclosed. The method includes receiving an audio signal from a sound source in a recording environment. The sound source and the recording environment are characterized by a set of attributes including a first attribute pertaining to a first attribute type and a second attribute pertaining to a second attribute type. A multi-head neural network is trained to extract from the received audio signal a first embedding vector indicative of the first attribute type and a second embedding vector indicative of the second attribute type. The first embedding vector is compared with a first set of embedding vectors to classify attributes of the first attribute type and the second embedding vector is compared with a second set of embedding vectors to classify attributes of the second attribute type, to determine a result of anomaly detection.
    Type: Application
    Filed: March 23, 2023
    Publication date: January 4, 2024
    Inventors: Gordon Wichern, Satvik Venkatesh, Aswin Shanmugam Subramanian, Jonathan Le Roux
  • Publication number: 20230394725
    Abstract: A system and method for automatically generating and rendering a report data structure is provided. The report data structure is formed in a platform independent manner that includes all data for transactions used in the report. The system analyzes the transactions to be included in the report and selects the type of display component based on a ranking score to best highlight the data contained therein.
    Type: Application
    Filed: August 21, 2023
    Publication date: December 7, 2023
    Inventors: Manuel Deschamps Rascon, Mark Eli Moreau Roseboom, Jonathan Le, Michael Furtak, Jeffrey Hall Seibert, JR., Wayne Chang
  • Patent number: 11810552
    Abstract: The present disclosure provides an artificial intelligence (AI) system for sequence-to-sequence modeling with attention adapted for streaming applications. The AI system comprises at least one processor; and memory having instructions stored thereon that, when executed by the processor, cause the AI system to process each input frame in a sequence of input frames through layers of a deep neural network (DNN) to produce a sequence of outputs. At least some of the layers of the DNN include a dual self-attention module having a dual non-causal and causal architecture attending to non-causal frames and causal frames. Further, the AI system renders the sequence of outputs.
    Type: Grant
    Filed: July 2, 2021
    Date of Patent: November 7, 2023
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Niko Moritz, Takaaki Hori, Jonathan Le Roux
  • Patent number: 11798574
    Abstract: A speech separation device (12) of a speech separation system includes a feature amount extraction unit (121) configured to extract time-series data of a speech feature amount of mixed speech, a block division unit (122) configured to divide the time-series data of the speech feature amount into blocks having a certain time width, a speech separation neural network (1b) configured to create time-series data of a mask of each of a plurality of speakers from the time-series data of the speech feature amount divided into blocks, and a speech restoration unit (123) configured to restore the speech data of each of the plurality of speakers from the time-series data of the mask and the time-series data of the speech feature amount of the mixed speech.
    Type: Grant
    Filed: January 12, 2021
    Date of Patent: October 24, 2023
    Assignees: MITSUBISHI ELECTRIC CORPORATION, MITSUBISHI ELECTRIC RESEARCH LABORATORIES, INC.
    Inventors: Ryo Aihara, Toshiyuki Hanazawa, Yohei Okato, Gordon P Wichern, Jonathan Le Roux
  • Patent number: 11790930
    Abstract: A system and method for reverberation reduction is disclosed. A first Deep Neural Network (DNN) produces a first estimate of a target direct-path signal from a mixture of acoustic signals that include the target direct-path signal and a reverberation of the target direct-path signal. A filter modeling a room impulse response (RIR) for the first estimate is estimated. The filter when applied to the first estimate of the target direct-path signal generates a result closest to a residual between the mixture of the acoustic signals and the first estimate of the target direct-path signal according to a distance function. A mixture with reduced reverberation of the target direct-path signal is obtained by removing the result of applying the filter to the first estimate of the target direct-path signal from the received mixture. A second DNN produces a second estimate of the target direct-path signal from the mixture with reduced reverberation.
    Type: Grant
    Filed: March 10, 2022
    Date of Patent: October 17, 2023
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Zhong-Qiu Wang, Gordon Wichern, Jonathan Le Roux
  • Publication number: 20230326478
    Abstract: Embodiments of the present disclosure disclose a system and method for extraction of a target sound signal. The system collects collect a mixture of sound signals. The system selects a query identifying the target sound signal to be extracted from the mixture of sound signals, the query comprising one or more identifiers. Each identifier is present in a predetermined set of one or more identifiers and defines at least one of mutually inclusive and mutually exclusive characteristics of the mixture of sound signals. The system determined one or more logical operators connecting the extracted one or more identifiers. The system transforms the one or more identifiers and the extracted logical operators into a digital representation. The system executes a neural network trained to extract the target sound signal by mixing the digital representation with intermediate outputs of intermediate layers of the neural network.
    Type: Application
    Filed: October 9, 2022
    Publication date: October 12, 2023
    Inventors: Gordon Wichern, Efthymios Tzinis, Aswin Shanmugam Subramanian, Jonathan Le Roux
  • Publication number: 20230306980
    Abstract: A system and method for low-latency audio signal enhancement is provided. An input mixture of audio signals is partitioned into a sequence of overlapping frames by using a first sliding window method. The first sliding window method comprises a first window function having a first width associated with a window of the corresponding frame and a shift length associated with shifting of the window of the first sliding window method. Each frame is then processed using a first DNN, a frequency domain causal linear filter and a second DNN, to generate final enhanced overlapping frames for each of the processed frames. The final enhanced overlapping frames are then combined using a second sliding window method associated with a second window function having a second width less than the first width and the same shift length as the first sliding window method.
    Type: Application
    Filed: October 10, 2022
    Publication date: September 28, 2023
    Inventors: Zhong Qiu Wang, Gordon Wichern, Jonathan Le Roux
  • Patent number: 11769282
    Abstract: A system and method for automatically generating and rendering a report data structure is provided. The report data structure is formed in a platform independent manner that includes all data for transactions used in the report. The system analyzes the transactions to be included in the report and selects the type of display component based on a ranking score to best highlight the data contained therein.
    Type: Grant
    Filed: October 14, 2020
    Date of Patent: September 26, 2023
    Assignee: Digits Financial, Inc.
    Inventors: Manuel Deschamps Rascon, Mark Eli Moreau Roseboom, Jonathan Le, Michael Furtak, Jeffrey Hall Seibert, Jr., Wayne Chang
  • Patent number: 11756551
    Abstract: An audio processing system is provided. The audio processing system comprises an input interface configured to accept an audio signal. Further, the audio processing system comprises a memory configured to store a neural network trained to determine different types of attributes of multiple concurrent audio events of different origins, wherein the types of attributes include time-dependent and time-agnostic attributes of speech and non-speech audio events. Further, the audio processing system comprises a processor configured to process the audio signal with the neural network to produce metadata of the audio signal, the metadata including one or multiple attributes of one or multiple audio events in the audio signal.
    Type: Grant
    Filed: October 7, 2020
    Date of Patent: September 12, 2023
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Niko Moritz, Gordon Wichern, Takaaki Hori, Jonathan Le Roux
  • Publication number: 20230283950
    Abstract: Embodiments of the present disclosure disclose a system and method for localization of a target sound event. The system collects a first digital representation of an acoustic mixture of sounds of a plurality of sound events, by using an acoustic sensor. The system receives a second digital representation of a sound corresponding to the target sound event. Further, the first digital representation and the second digital representation are processed by a neural network to produce a localization information indicative of a location of an origin of the target sound event with respect to a location of the acoustic sensor.
    Type: Application
    Filed: March 7, 2022
    Publication date: September 7, 2023
    Applicant: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Gordon Wichern, Olga Slizovskaia, Jonathan Le Roux
  • Patent number: 11635299
    Abstract: A navigation system for providing driving instructions to a driver of a vehicle traveling on a route is provided. The driving instructions are generated by executing a multimodal fusion method that comprises extracting features from sensor measurements, annotating the features with directions for the vehicle to follow the route with respect to objects sensed by the sensors, and encoding the annotated features with a multimodal attention neural network to produce encodings. The encodings are transformed into a common latent space, and the transformed encodings are fused using an attention mechanism producing an encoded representation of the scene. The method further comprises decoding the encoded representation with a sentence generation neural network to generate a driving instruction and submitting the driving instruction to an output device.
    Type: Grant
    Filed: February 6, 2020
    Date of Patent: April 25, 2023
    Inventors: Chiori Hori, Anoop Cherian, Siheng Chen, Tim Marks, Jonathan Le Roux, Takaaki Hori, Bret Harsham, Anthony Vetro, Alan Sullivan
  • Publication number: 20230086355
    Abstract: A system and method for detecting anomalous sound are disclosed. The method includes receiving a spectrogram of an audio signal with elements defined by values in a time-frequency domain of the spectrogram. Each of the values corresponds to an element of the spectrogram that is identified by a coordinate in the time-frequency domain. The time-frequency domain of the spectrogram is partitioned into a context region and a target region. The context region and the target region are processed by a neural network using an attentive neural process to recover values of the spectrogram for elements with coordinates in the target region. The recovered values of the elements of the target region are compared with values of elements of the partitioned target region. An anomaly score is determined based on the comparison. The anomaly score is used for performing a control action.
    Type: Application
    Filed: September 19, 2021
    Publication date: March 23, 2023
    Inventors: Gordon Wichern, Ankush Chakrabarty, Zhong-Qiu Wang, Jonathan Le Roux
  • Patent number: 11582485
    Abstract: Embodiments of the present disclosure discloses a scene-aware video encoder system. The scene-aware encoder system transforms a sequence of video frames of a video of a scene into a spatio-temporal scene graph. The spatio-temporal scene graph includes nodes representing one or multiple static and dynamic objects in the scene. Each node of the spatio-temporal scene graph describes an appearance, a location, and/or a motion of each of the objects (static and dynamic objects) at different time instances. The nodes of the spatio-temporal scene graph are embedded into a latent space using a spatio-temporal transformer encoding different combinations of different nodes of the spatio-temporal scene graph corresponding to different spatio-temporal volumes of the scene. Each node of the different nodes encoded in each of the combinations is weighted with an attention score determined as a function of similarities of spatio-temporal locations of the different nodes in the combination.
    Type: Grant
    Filed: February 7, 2022
    Date of Patent: February 14, 2023
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Anoop Cherian, Chiori Hori, Jonathan Le Roux, Tim Marks, Alan Sullivan