Patents Assigned to Deepgram, Inc.
  • Patent number: 12380880
    Abstract: An end-to-end automatic speech recognition (ASR) system can be constructed by fusing a first ASR model with a transformer. The input of the transformer is a learned layer generated by the first ASR model. The fused ASR model and transformer can be treated as a single end-to-end model and trained as a single model. In some embodiments, the end-to-end speech recognition system can be trained using a teacher-student training technique by selectively truncating portions of the first ASR model and/or the transformer components and selectively freezing various layers during the training passes.
    Type: Grant
    Filed: April 3, 2023
    Date of Patent: August 5, 2025
    Assignee: Deepgram, Inc.
    Inventors: Andrew Nathan Seagraves, Deepak Subburam, Adam Joseph Sypniewski, Scott Ivan Stephenson, Jacob Edward Cutter, Michael Joseph Sypniewski, Daniel Lewis Shafer
  • Patent number: 12334075
    Abstract: Modern automatic speech recognition (ASR) systems can utilize artificial intelligence (AI) models to service ASR requests. The number and scale of AI models used in a modern ASR system can be substantial. The process of configuring and reconfiguring hardware to execute various AI models corresponding to a substantial number of ASR requests can be time consuming and inefficient. Among other features, the described technology utilizes batching of ASR requests, splitting of the ASR requests, and/or parallel processing to efficiently use hardware tasked with executing AI models corresponding to ASR requests. In one embodiment, the compute graphs of ASR tasks are used to batch the ASR requests. The corresponding AI models of each batch can be loaded into hardware, and batches can be processed in parallel. In some embodiments, the ASR requests are split, batched, and processed in parallel.
    Type: Grant
    Filed: October 14, 2022
    Date of Patent: June 17, 2025
    Assignee: Deepgram, Inc.
    Inventors: Adam Joseph Sypniewski, Joshua Gevirtz, Nikola Lazar Whallon, Anthony John Deschamps, Scott Ivan Stephenson
  • Patent number: 11676579
    Abstract: Systems and methods are disclosed for generating internal state representations of a neural network during processing and using the internal state representations for classification or search. In some embodiments, the internal state representations are generated from the output activation functions of a subset of nodes of the neural network. The internal state representations may be used for classification by training a classification model using internal state representations and corresponding classifications. The internal state representations may be used for search, by producing a search feature from an search input and comparing the search feature with one or more feature representations to find the feature representation with the highest degree of similarity.
    Type: Grant
    Filed: October 16, 2020
    Date of Patent: June 13, 2023
    Assignee: Deepgram, Inc.
    Inventors: Jeff Ward, Adam Sypniewski, Scott Stephenson
  • Patent number: 11367433
    Abstract: Systems and methods are disclosed for end-to-end neural networks for speech recognition and classification and additional machine learning techniques that may be used in conjunction or separately. Some embodiments comprise multiple neural networks, directly connected to each other to form an end-to-end neural network. One embodiment comprises a convolutional network, a first fully-connected network, a recurrent network, a second fully-connected network, and an output network. Some embodiments are related to generating speech transcriptions, and some embodiments relate to classifying speech into a number of classifications.
    Type: Grant
    Filed: May 29, 2020
    Date of Patent: June 21, 2022
    Assignee: Deepgram, Inc.
    Inventors: Adam Sypniewski, Jeff Ward, Scott Stephenson
  • Patent number: 10847138
    Abstract: Systems and methods are disclosed for generating internal state representations of a neural network during processing and using the internal state representations for classification or search. In some embodiments, the internal state representations are generated from the output activation functions of a subset of nodes of the neural network. The internal state representations may be used for classification by training a classification model using internal state representations and corresponding classifications. The internal state representations may be used for search, by producing a search feature from an search input and comparing the search feature with one or more feature representations to find the feature representation with the highest degree of similarity.
    Type: Grant
    Filed: May 21, 2019
    Date of Patent: November 24, 2020
    Assignee: Deepgram, Inc.
    Inventors: Jeff Ward, Adam Sypniewski, Scott Stephenson
  • Patent number: 10720151
    Abstract: Systems and methods are disclosed for end-to-end neural networks for speech recognition and classification and additional machine learning techniques that may be used in conjunction or separately. Some embodiments comprise multiple neural networks, directly connected to each other to form an end-to-end neural network. One embodiment comprises a convolutional network, a first fully-connected network, a recurrent network, a second fully-connected network, and an output network. Some embodiments are related to generating speech transcriptions, and some embodiments relate to classifying speech into a number of classifications.
    Type: Grant
    Filed: August 22, 2018
    Date of Patent: July 21, 2020
    Assignee: Deepgram, Inc.
    Inventors: Adam Sypniewski, Jeff Ward, Scott Stephenson
  • Publication number: 20200035222
    Abstract: Systems and methods are disclosed for end-to-end neural networks for speech recognition and classification and additional machine learning techniques that may be used in conjunction or separately. Some embodiments comprise multiple neural networks, directly connected to each other to form an end-to-end neural network. One embodiment comprises a convolutional network, a first fully-connected network, a recurrent network, a second fully-connected network, and an output network. Some embodiments are related to generating speech transcriptions, and some embodiments relate to classifying speech into a number of classifications.
    Type: Application
    Filed: August 22, 2018
    Publication date: January 30, 2020
    Applicant: Deepgram, Inc.
    Inventors: Adam Sypniewski, Jeff Ward, Scott Stephenson
  • Publication number: 20200035224
    Abstract: Systems and methods are disclosed for generating internal state representations of a neural network during processing and using the internal state representations for classification or search. In some embodiments, the internal state representations are generated from the output activation functions of a subset of nodes of the neural network. The internal state representations may be used for classification by training a classification model using internal state representations and corresponding classifications. The internal state representations may be used for search, by producing a search feature from an search input and comparing the search feature with one or more feature representations to find the feature representation with the highest degree of similarity.
    Type: Application
    Filed: May 21, 2019
    Publication date: January 30, 2020
    Applicant: Deepgram, Inc.
    Inventors: Jeff Ward, Adam Sypniewski, Scott Stephenson
  • Patent number: 10540959
    Abstract: Systems and methods are disclosed for customizing a neural network for a custom dataset, when the neural network has been trained on data from a general dataset. The neural network may comprise an output layer including one or more nodes corresponding to candidate outputs. The values of the nodes in the output layer may correspond to a probability that the candidate output is the correct output for an input. The values of the nodes in the output layer may be adjusted for higher performance when the neural network is used to process data from a custom dataset.
    Type: Grant
    Filed: December 26, 2018
    Date of Patent: January 21, 2020
    Assignee: Deepgram, Inc.
    Inventors: Jeff Ward, Adam Sypniewski, Scott Stephenson
  • Patent number: 10380997
    Abstract: Systems and methods are disclosed for generating internal state representations of a neural network during processing and using the internal state representations for classification or search. In some embodiments, the internal state representations are generated from the output activation functions of a subset of nodes of the neural network. The internal state representations may be used for classification by training a classification model using internal state representations and corresponding classifications. The internal state representations may be used for search, by producing a search feature from an search input and comparing the search feature with one or more feature representations to find the feature representation with the highest degree of similarity.
    Type: Grant
    Filed: August 22, 2018
    Date of Patent: August 13, 2019
    Assignee: Deepgram, Inc.
    Inventors: Jeff Ward, Adam Sypniewski, Scott Stephenson
  • Patent number: 10210860
    Abstract: Systems and methods are disclosed for customizing a neural network for a custom dataset, when the neural network has been trained on data from a general dataset. The neural network may comprise an output layer including one or more nodes corresponding to candidate outputs. The values of the nodes in the output layer may correspond to a probability that the candidate output is the correct output for an input. The values of the nodes in the output layer may be adjusted for higher performance when the neural network is used to process data from a custom dataset.
    Type: Grant
    Filed: August 22, 2018
    Date of Patent: February 19, 2019
    Assignee: Deepgram, Inc.
    Inventors: Jeff Ward, Adam Sypniewski, Scott Stephenson