Abstract: Disclosed are apparatuses, systems, and techniques that leverage one or more language models (LMs)—such as large language models (LLMs—for efficient multi-speaker speech recognition. The techniques include processing, using a speaker diarization model, an audio feature to generate a first association of the audio feature with one or more prospective speakers, the audio feature being representative of one or more spoken words. The techniques further include providing, to an LM, a first prompt requesting the LM to identify a second association of the one or more spoken words with the one or more prospective speakers and receiving, from the LM, a first response identifying the second association of the one or more spoken words with the one or more prospective speakers. The techniques further include determining, using the first association and the second association, one or more speakers that produced the one or more spoken words.

Abstract: Disclosed are apparatuses, systems, and techniques that may use machine learning for implementing speaker recognition, verification, and/or diarization. The techniques include receiving a first set of audio data channels (ADCs) jointly capturing a speech produced by one or more speakers and obtaining, using the first set of ADCs, a second set of one or more ADCs. Individual ADCs of the second set of ADCs represent one or more channels of the first set of ADCs, and at least one channel of the second set of ADCs represents a cluster of two or more ADCs of the first set of ADCs, the two of more ADCs being selected based on similarity of audio data of the two or more ADCs. The techniques further include processing, using an audio processing neural network model, the second set of ADCs to obtain an association of the speech to the one or more speakers.

Abstract: Disclosed are apparatuses, systems, and techniques that may use machine learning for implementing speaker recognition, verification, and/or diarization. The techniques include processing audio data channels (ADCs) using a voice detection model to determine voice activity likelihoods (VALs) that individual ADCs include speech, obtaining, using VALs, a second set of ADC(s), and processing, using an audio processing a neural network (NN) model, the second set of ADCs to obtain association of the speech to the one or more speakers. The techniques also include generating a plurality of embeddings associated with the ADCs, processing the plurality of embeddings to obtain aggregated embedding(s) that represent audio data of multiple ADCs, and processing the aggregated embedding(s), using the audio processing NN model, to obtain association of the speech to the one or more speakers.

Abstract: A method for free flow fever screening is presented. The method includes capturing a plurality of frames from thermal data streams and visual data streams related to a same scene to define thermal data frames and visual data frames, detecting and tracking a plurality of individuals moving in a free-flow setting within the visual data frames, and generating a tracking identification for each individual of the plurality of individuals present in a field-of-view of the one or more cameras across several frames of the plurality of frames. The method further includes fusing the thermal data frames and the visual data frames, measuring, by a fever-screener, a temperature of each individual of the plurality of individuals within and across the plurality of frames derived from the thermal data streams and the visual data streams, and generating a notification when a temperature of an individual exceeds a predetermined threshold temperature.

Abstract: A method for performing resource orchestration for microservices-based 5G applications in a dynamic, heterogenous, multi-tiered compute and network environment is presented.

Abstract: Methods and systems for executing an application include extending a container orchestration system application programming interface (API) to handle objects that specify components of an application. An application representation is executed using the extended container orchestration system API, including the instantiation of one or more services that define a data stream path from a sensor to a device.

Abstract: Systems and methods are provided for deploying applications within a wireless network infrastructure, including initiating, by a centralized control module in a pre-configured hardware unit having a 5G wireless communication module, edge computing device, centralized control module, and data processing module with access to cloud resources, a setup procedure upon receiving a deployment command, the setup procedure including activating the 5G wireless communication module to establish a network connection. User equipment for communication with sensors and cameras is deployed using an edge device through the network connection. Application deployment is managed using a centralized control module including an edge cloud optimizer for allocating resources between an edge computing device and the cloud resources based on real-time analysis of network conditions and application requirements.

Abstract: A method for real-time cross-spectral object association and depth estimation is presented. The method includes synthesizing, by a cross-spectral generative adversarial network (CS-GAN), visual images from different data streams obtained from a plurality of different types of sensors, applying a feature-preserving loss function resulting in real-time pairing of corresponding cross-spectral objects, and applying dual bottleneck residual layers with skip connections to accelerate real-time inference and to accelerate convergence during model training.

Abstract: A pull-based communication method for microservices-based real-time streaming video analytics pipelines is provided. The method includes receiving a plurality of frames from a plurality of cameras, each camera including a camera sidecar, arranging a plurality of detectors in layers such that a first detector layer includes detectors with detector sidecars and detector business logic, and the second detector layer includes detectors with only sidecars, arranging a plurality of extractors in layers such that a first extractor layer includes extractors with extractor sidecars and extractor business logic, and the second extractor layer includes extractors with only sidecars, and enabling a mesh controller, during registration, to selectively assign inputs to one or more of the detector sidecars of the first detector layer and one or more of the extractor sidecars of the first extractor layer to pull data items for processing.

Abstract: Systems and methods for scaling in a container orchestration platform are described that include configuring an autoscaler in a control plane of the container orchestration platform to receive stream data from a data exchange system that is measuring stream processing of a pipeline of microservices for an application. The systems and methods further include controlling a number of deployment pods in at least one node of the container orchestration platform to meet requirements for the application provided by the pipeline of microservices.

Abstract: Methods and systems for camera configuration include configuring an image capture configuration parameter of a camera according to a multi-objective reinforcement learning aggregated reward function. Respective quality estimates for analytics are determined after configuring the image capture parameters. The aggregated reward function is updated based on the quality estimates.

Abstract: A computer implemented method is provided for resource management of stream analytics at each individual node that includes computing a mean of output processing rate of microservices in a pipeline; and evaluating a state of each microservice of the microservices in the pipeline. The computer implemented method also includes selecting a single microservice from the pipeline for updating resources for an action that changes the state in single the microservice that is selected; and performing resource allocation update for the selected microservice. The computer implemented method may also include updating the state of the selected microservice.

Abstract: Systems and methods are provided for dynamically tuning camera parameters in a video analytics system to optimize analytics accuracy. A camera captures a current scene, and optimal camera parameter settings are learned and identified for the current scene using a Reinforcement Learning (RL) engine. The learning includes defining a state within the RL engine as a tuple of two vectors: a first representing current camera parameter values and a second representing measured values of frames of the current scene. Quality of frames is estimated using a quality estimator, and camera parameters are adjusted based on the quality estimator and the RL engine for optimization. Effectiveness of tuning is determined using perceptual Image Quality Assessment (IQA) to quantify a quality measure. Camera parameters are adaptively tuned in real-time based on learned optimal camera parameter settings, state, quality measure, and set of actions, to optimize the analytics accuracy for video analytics tasks.

Abstract: Systems and methods are provided for increasing accuracy of video analytics tasks in real-time by acquiring a video using video cameras, and identifying fluctuations in the accuracy of video analytics applications across consecutive frames of the video. The identified fluctuations are quantified based on an average relative difference of true-positive detection counts across consecutive frames. Fluctuations in accuracy are reduced by applying transfer learning to a deep learning model initially trained using images, and retraining the deep learning model using video frames. A quality of object detections is determined based on an amount of track-ids assigned by a tracker across different video frames. Optimization of the reduction of fluctuations includes iteratively repeating the identifying, the quantifying, the reducing, and the determining the quality of object detections until a threshold is reached. Model predictions for each frame in the video are generated using the retrained deep learning model.

Abstract: A method for implementing application self-optimization in serverless edge computing environments is presented. The method includes requesting deployment of an application pipeline on data received from a plurality of sensors, the application pipeline including a plurality of microservices, enabling communication between a plurality of pods and a plurality of analytics units (AUs), each pod of the plurality of pods including a sidecar, determining whether each of the plurality of AUs maintains any state to differentiate between stateful AUs and stateless AUs, scaling the stateful AUs and the stateless AUs, enabling communication directly between the sidecars of the plurality of pods, and reusing and resharing common AUs of the plurality of AUs across different applications.

Abstract: A pull-based communication method for microservices-based real-time streaming video analytics pipelines is provided. The method includes receiving a plurality of frames from a plurality of cameras, each camera including a camera sidecar, arranging a plurality of detectors in layers such that a first detector layer includes detectors with detector sidecars and detector business logic, and the second detector layer includes detectors with only sidecars, arranging a plurality of extractors in layers such that a first extractor layer includes extractors with extractor sidecars and extractor business logic, and the second extractor layer includes extractors with only sidecars, and enabling a mesh controller, during registration, to selectively assign inputs to one or more of the detector sidecars of the first detector layer and one or more of the extractor sidecars of the first extractor layer to pull data items for processing.

Abstract: A method is provided for managing applications for sensors. In one embodiment, the method includes loading a plurality of applications and links for communicating with a plurality of sensors on a platform having an interface for entry of a requested use case; and copying a configuration from a grouping of application instances being applied to a first sensor performing in a function comprising of the requested use case. The method may further include applying the configuration for the grouping of application instances to a second set of sensors to automatically conform the plurality of sensors on the platform to perform the requested use case.

Abstract: Methods and systems for reserving resources include determining a state of a distributed computing system based on resource needs of an application that is executed on the distributed computing system and system resource constraints. An action is determined using the state of the distributed computing system as an input to a trained reinforcement learning model. A resource request is issued for the application to reserve resources based on the action.

Abstract: Systems and methods for network bandwidth optimization, including transmitting sensor data from one or more sensors over a wireless network into a generated network slice, submitting a Quality-of-Service (QoS) request for one or more applications by specifying desired network slice characteristics, and predicting network bandwidth needed for granting the QoS request for the one or more applications using a cost function based on magnitude, direction, and frequency of error. Time-varying network bandwidth usage is continuously monitored, and new QoS requests for the one or more applications are periodically requested based on the monitoring. An updated prediction for updated bandwidth needed for the new QoS request is generated using the cost function, and network bandwidth reservations are iteratively adjusted based on the updated prediction for the new QoS request to provide an amount of network resources to the one or more applications to support the new QoS request.

Abstract: Methods and systems for managing communications include identifying a system condition in a distributed computing system comprising a first microservice in communication with a second microservice. A communications method is identified responsive to the identified system condition using a reinforcement learning model that associates communication methods with system conditions. The identified communications method is implemented for communications between the first microservice and the second microservice, such that the first microservice and the second microservice use the identified communications method to transmit data.