Abstract: Methods and systems for training one or more neural networks for transcription and for transcribing a media file using the trained one or more neural networks are provided. One of the methods includes: segmenting the media file into a plurality of segments; inputting each segment, one segment at a time, of the plurality of segments into a first neural network trained to perform speech recognition; extracting outputs, one segment at a time, from one or more layers of the first neural network; and training a second neural network to generate a predicted-WER (word error rate) of a plurality of transcription engines for each segment based at least on outputs from the one or more layers of the first neural network.

Abstract: Disclosed are systems and methods to detect and track an object across frames of a video. One of the disclosed methods includes: detecting a first group of one or more objects, using a first neural network, in each frame of the video, wherein each detected head of the first group comprises a leading and a trailing edge; grouping the leading and trailing edges of the one or more objects into groups of leading edges and groups of trailing edges based at least on coordinates of the leading and trailing edges; generating a list of no-edge-detect frames by identifying frames of the video missing a group of leading edges or a group of trailing edges; analyzing the no-edge-detect frames in the list of no-edge-detect frames, using an optical image classification engine, to detect a second group of one or more objects in the no-edge-detect frames; and merging the first and second groups of one or more objects to form a merged list of detected objects in the video.

Abstract: Methods and systems for training one or more neural networks for transcription and for transcribing a media file using the trained one or more neural networks are provided. One of the methods includes: segmenting the media file into a plurality of segments; inputting each segment, one segment at a time, of the plurality of segments into a first neural network trained to perform speech recognition; extracting outputs, one segment at a time, from one or more layers of the first neural network; and training a second neural network to generate a predicted-WER (word error rate) of a plurality of transcription engines for each segment based at least on outputs from the one or more layers of the first neural network.

Abstract: Disclosed are systems and methods to detect and track an object across frames of a video. One of the disclosed methods includes: detecting a first group of one or more objects, using a first neural network, in each frame of the video, wherein each detected head of the first group comprises a leading and a trailing edge; grouping the leading and trailing edges of the one or more objects into groups of leading edges and groups of trailing edges based at least on coordinates of the leading and trailing edges; generating a list of no-edge-detect frames by identifying frames of the video missing a group of leading edges or a group of trailing edges; analyzing the no-edge-detect frames in the list of no-edge-detect frames, using an optical image classification engine, to detect a second group of one or more objects in the no-edge-detect frames; and merging the first and second groups of one or more objects to form a merged list of detected objects in the video.

Abstract: Methods and systems for training one or more neural networks for transcription and for transcribing a media file using the trained one or more neural networks are provided. One of the methods includes: segmenting the media file into a plurality of segments; inputting each segment, one segment at a time, of the plurality of segments into a first neural network trained to perform speech recognition; extracting outputs, one segment at a time, from one or more layers of the first neural network; and training a second neural network to generate a predicted-WER (word error rate) of a plurality of transcription engines for each segment based at least on outputs from the one or more layers of the first neural network.

Abstract: User authentication is an extremely important process in many applications and industries. Because of its importance, most security-sensitive user authentication processes employ an automatic multi-factor authentication process that involves confirming a SMS message, answering a security question, entering a PIN, etc. However, even these auto multi-factor authentication processes are vulnerable to attack and hack. For example, some facial recognition authentication processes can be defeated using a picture. Voice print can also be duplicated using a previous recording of the user's voice. As such, most financial institutions employ some form of human involvement (on top of multi-factor authentication) to authenticate a user for high security sensitive situations. The cost for performing authentication with human involvement can be very expensive.

Abstract: Methods and systems for classifying a multimedia file using interclass data is disclosed. One of the methods includes receiving, from a first transcription engine, one or more transcription results of one or more audio segments of the multimedia file; identifying a first transcription result for a first audio segment having a low confidence of accuracy; identifying a first image data of the multimedia file corresponding to the first segment; receiving, from an image classification engine trained to classify image data, an image classification result of one or more portions of the first image data in response to requesting the image classification engine to classify the first image data; and selecting, based at least on the image classification result of the one or more portions of the first image data, a second transcription engine to re-classify the first audio segment.

Abstract: Provided herein are embodiments of systems and methods for classifying one or more objects in an image. One of the methods includes: receiving object classification results of the image from one or more classification engines, the object classification results comprise classification of one or more objects and confidence scores associated with the one or more objects; aggregating the object classification results from the one or more classification engines to generate a list of confidence scores associated with each of the one or more objects, the list of confidence scores comprises one or more confidence scores from one or more classification engines; calculating an overall certainty score for each of the one or more objects based at least on the list of confidence scores; and generating a first orchestrated classification result based the overall certainty score for each of the one or more object.

Abstract: Methods and systems for training one or more neural networks for transcription and for transcribing a media file using the trained one or more neural networks are provided. One of the methods includes: segmenting the media file into a plurality of segments; extracting, using a first neural network, audio features of a first and second segment of the plurality of segments; and identifying, using a second neural network, a best-candidate engine for each of the first and second segments based at least on audio features of the first and second segments. A best-candidate engine is a neural network having a highest predicted transcription accuracy among a collection of neural networks.

Abstract: Methods and systems for classifying a multimedia file using interclass data is disclosed. One of the methods can use classification results from one or more engines of different classes to select a different engine for the original classification task. For example, given an audio segment with associated metadata and image data, the disclosed interclass method can use the classification results from a topic classification of metadata and/or an image classification result of the image data as inputs for selecting a new transcription engine to transcribe the audio segment.

Abstract: Disclosed are systems and methods to detect and track an object across frames of a video. One of the disclosed methods includes: detecting a first group of one or more objects, using a first neural network, in each frame of the video, wherein each detected head of the first group comprises a leading and a trailing edge; grouping the leading and trailing edges of the one or more objects into groups of leading edges and groups of trailing edges based at least on coordinates of the leading and trailing edges; generating a list of no-edge-detect frames by identifying frames of the video missing a group of leading edges or a group of trailing edges; analyzing the no-edge-detect frames in the list of no-edge-detect frames, using an optical image classification engine, to detect a second group of one or more objects in the no-edge-detect frames; and merging the first and second groups of one or more objects to form a merged list of detected objects in the video.

Abstract: Methods and systems for transcribing a media file using a human intelligence task service and/or reinforcement learning are provided. The disclosed systems and methods provide opportunities for a segment of the input media file to be automatically re-analyzed, re-transcribed, and/or modified for re-transcription using a human intelligence task (HIT) service for verification and/or modification of the transcription results. The segment can also be reanalyzed, reconstructed, and re-transcribed using a reinforcement learning enabled transcription model.

Abstract: Provided herein are embodiments of systems and methods for classifying one or more objects in an image. One of the methods includes: receiving object classification results of the image from one or more classification engines, the object classification results comprise classification of one or more objects and confidence scores associated with the one or more objects; aggregating the object classification results from the one or more classification engines to generate a list of confidence scores associated with each of the one or more objects, the list of confidence scores comprises one or more confidence scores from one or more classification engines; calculating an overall certainty score for each of the one or more objects based at least on the list of confidence scores; and generating a first orchestrated classification result based the overall certainty score for each of the one or more object.

Abstract: Methods and systems for training an engine prediction neural network is disclosed. One of the methods can include: extracting image features of a first ground truth image using outputs of one or more layers of an image classification neural network; classifying the first ground truth image using a plurality of candidate neural networks; determining a classification accuracy score of a classification result of the first ground truth image for each candidate neural network of the plurality of candidate neural networks; and training the engine prediction neural network to predict the best candidate engine by associating the image features of the first ground truth image with the classification accuracy score of each candidate neural network.

Abstract: Methods and systems for classifying a multimedia file using interclass data is disclosed. One of the methods can use classification results from one or more engines of different classes to select a different engine for the original classification task. For example, given an audio segment with associated metadata and image data, the disclosed interclass method can use the classification results from a topic classification of metadata and/or an image classification result of the image data as inputs for selecting a new transcription engine to transcribe the audio segment.

Abstract: Methods and systems for classifying a multimedia file using interclass data is disclosed. One of the methods includes receiving, from a first transcription engine, one or more transcription results of one or more audio segments of the multimedia file; identifying a first transcription result for a first audio segment having a low confidence of accuracy; identifying a first image data of the multimedia file corresponding to the first segment; receiving, from an image classification engine trained to classify image data, an image classification result of one or more portions of the first image data in response to requesting the image classification engine to classify the first image data; and selecting, based at least on the image classification result of the one or more portions of the first image data, a second transcription engine to re-classify the first audio segment.

Abstract: Methods and systems for training one or more neural networks for transcription and for transcribing a media file using the trained one or more neural networks are provided. One of the methods includes: segmenting the media file into a plurality of segments; extracting, using a first neural network, audio features of a first and second segment of the plurality of segments; and identifying, using a second neural network, a best-candidate engine for each of the first and second segments based at least on audio features of the first and second segments. A best-candidate engine is a neural network having a highest predicted transcription accuracy among a collection of neural networks.

Abstract: A system and method for publishing endorsed media is disclosed herein. The system receives a media file and then transcribes a portion of the media file to produce a transcribed script for the media portion. The system then extracts one or more sentiments based on the transcribed script and media data of the media file. Next, the system associates the one or more extracted sentiments to one or more keywords in the transcribed script. Trending topic around the Internet is then determined. In one aspect, the system then publishes the media portion over the network based on a match between the trending topic and the one or more keywords associated with the one or more extracted sentiments.

Abstract: Methods and systems for transcribing a media file using reinforcement learning are provided. In one aspect, the method includes: identifying a low confidence of accuracy portion from a transcription result of the media file; constructing a phoneme sequence that includes an audio segment corresponding to the identified low confidence of accuracy portion, based on at least on a reward function; creating a new audio waveform based at least on the constructed phoneme sequence; and generating a new transcription using a transcription engine based on the new audio waveform.

Abstract: A method of transcription a media file is provided. The method includes: generating a feature profile for a media file; segmenting the media file into a first and a second segment based at least on portion(s) of the feature profile corresponding to the first and second segments, identifying one or more transcription engines having a predicted high level of accuracy, using a trained machine learning transcription model, for the first and second segments; requesting a first and a second transcription engine from the identified one or more transcription engines to transcribe the first and second segments, respectively; receiving a first and a second transcribed portion from the first and second transcription engines; and generating a merged transcription using the first and second transcribed portions.