Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing language machine learning model (LLM) as autonomous reasoners to navigate and execute multiple layers of a computerized bio-activity discovery pipeline of a tech-bio exploration system. In particular, the disclosed systems can utilize an LLM that learns to access one or more tech-bio exploration tools to execute one or more processes and/or tasks in a bio-activity discovery pipeline. For instance, the disclosed systems can provide an interactive query prompt interface to enable users to provide tech-bio queries (as prompts) and utilize the LLM with the prompts to execute one or more tasks in the bio-activity discovery pipeline to generate and/or retrieve bio-activity data for the query. Moreover, the disclosed systems can utilize one or more LLMs to autonomously utilize and/or interact with one or more tech-bio tools in the bio-activity discovery pipeline to generate and/or obtain bio-activity data.

Abstract: A method of real-time video event detection includes: obtaining, based on a natural language query, a query vector; performing multimodal feature extraction on a video stream to obtain a video vector, obtaining a similarity score by comparing the query vector to the video vector; comparing the similarity score to a predetermined threshold; and activating, based on the similarity score being above the predetermined threshold, an action trigger. The multimodal feature extraction is performed using a plurality of overlapping windows that include sequential frames of the video stream.

Abstract: A method may include receiving, by a virtual assistant of a user device, an input from a user, the virtual assistant being based on software. The method may include obtaining, by the virtual assistant of the user device and via a sensor of the user device, audio information or video information of the user. The method may include determining, by the virtual assistant of the user device, an identity of the user based on the audio information or the video information of the user and a set of facial embeddings and speech embeddings that is correlated with the user, the set of facial embeddings and speech embeddings being generated using a facial embedding model, a speech embedding model, and a sound source localization model. The method may include performing, by the virtual assistant of the user device, an action based on the input and the identity of the user.