Abstract: Methods for executing an open-ended audio tracking system are disclosed. A feedback loop between an audio foundational model (AFM) and a large language model (LLM) enables for both detection of low-level sound events in real-time and detection of high-level acoustic scenes, which are then used to generate additional text-based event descriptions that are applied in a subsequent iteration cycle of the system. The AFM may resemble a contrastive language-audio pre-training (CLAP) model that is configured to sound event detection, while the LLM receives the particular sound events that were detected and categorizes those events into an acoustic sound category that explains the environmental context of the sound events.

Abstract: Training of an audio foundation model (AFM) is performed using a dataset constructed using low-level audio property control and high-level composition planning. A plurality of digital audio compositions are generated, using a large language model (LLM) as a planner agent. The planner agent is prompted to generate composition plans defining logical combinations of foreground and background digital sounds, event occurrences within the compositions, and digital sound properties. The foreground and background digital sounds have consistent audio quality. An audio composition tool generates the plurality of digital audio compositions according to the composition plans. Descriptive text is generated for each of the digital audio compositions using a summarizer agent. The summarizer agent is implemented as an LLM, prompted to describe the digital audio compositions. The compositions and the corresponding descriptive text are combined to form audio-text pairs.

Abstract: A method and system is disclosed for tuning a machine learning classifier. An object class requirement may be provided and include rank thresholds. The object class requirements may also include a range goal that defines a minimum distance from the object the machine learning algorithm should not provide false positive results. A base classifier may be trained using a weighted loss function that includes one or more weight values that are computed using the one or more object class requirements. An output of the weighted loss function may be evaluated using an objective function which may be established using the one or more object class requirements. The one or more weights may also be re-tuned using the weighted loss function if the output of the weighted loss function does not converge within a predetermined loss threshold.

Abstract: Active learning for anomalous event detection and classification in industrial applications. Initial samples from an industrial environment may be received. Unlabeled samples may then be classified using a target query strategy to determine the top-ranked relevant or most important samples. The top-ranked samples may then be manually annotated. A model may then be optimized based on the initial samples from the industrial environment and the annotated top-ranked samples. The model's performance may then be evaluated.

Abstract: Knowledge-based audio-text modeling via automatic multimodal graph construction is performed. An audio dataset is received, the audio dataset including clips of audio data, wherein each of the clips of the audio data is paired with corresponding metadata descriptive of the audio contents of the respective clip of the audio data. Graph nodes of interest are identified from a sematic network, the graph nodes being descriptive of semantics of the knowledge domain of the contents of the audio dataset. A large language model (LLM) is utilized for categorizing the metadata into the graph nodes and for inferring supplemental data for the graph nodes for which there is no metadata, producing an extracted knowledge graph. The extracted knowledge graph is validated utilizing the LLM to perform relation verification of edges between the graph nodes of the extracted knowledge graph, thereby mitigating hallucination effects in the categorizing and inferring of the supplemental data.

Abstract: Methods and systems for generating training data for training a contrastive language-audio machine-learning model. A plurality of audio segments are retrieved from a speech emotion recognition (SER) database along with metadata associated with the audio segments. The metadata of each audio segment includes an emotion class. Words or terms associated with emotions are retrieved from a lexicon. A large language model (LLM) is executed on (i) the classes of emotion associated with the audio segments and (ii) the words or terms from the lexicon. This generates a plurality of text captions associated with emotion, which are stored in a caption pool. For each audio segment retrieved from the SER database, that audio segment is paired with one or more of the text captions from the caption pool that were generated based on the emotion class associated with that audio segment. This yields audio-text pairs for training a contrastive learning model.

Abstract: A method and system is disclosed for creating a machine learning model that is reconfigurable. A fixed parameter model is created to include fixed feature values obtained during a training process for the machine learning model. The fixed parameter model may include a fixed base classifier used by the machine learning model to classify objects detected by an ultra-sonic system within a vicinity of a vehicle. A configurable parameter model may be created to include feature values that are different from the fixed feature values, the configurable parameter model including a modified base classifier. A vehicle controller may receive and update the fixed parameter model with the configurable parameter model. The machine learning model may be updated to use the configurable parameter model to classify the objects detected by the ultra-sonic system.

Abstract: A method and system is disclosed for tuning a machine learning classifier. An object class requirement may be provided and include rank thresholds. The object class requirements may also include a range goal that defines a minimum distance from the object the machine learning algorithm should not provide false positive results. A base classifier may be trained using a weighted loss function that includes one or more weight values that are computed using the one or more object class requirements. An output of the weighted loss function may be evaluated using an objective function which may be established using the one or more object class requirements. The one or more weights may also be re-tuned using the weighted loss function if the output of the weighted loss function does not converge within a predetermined loss threshold.

Abstract: A system and method is disclosed for classifying one or more objects within a vicinity of a vehicle. Ultra-sonic data may be received from a plurality of ultra-sonic sensors and may comprise echo signals indicating one or more objects that are proximally located within a vicinity of a vehicle. One or more features may be calculated from the ultra-sonic data using one or more signal processing algorithms unique to each of the plurality of ultra-sonic sensors. The one more features may be combined using a second-level signal processing algorithm to determine geometric relations for the one or more objects. The one or more features may then be statistically aggregated at an object level. The one or more objects may then be classified using a machine learning algorithm that compares an input of each of the one or more features to a trained classifier.