Abstract: Methods and systems for action detection include encoding a text feature of an input textual description of an action using a visual language model (VLM). A video feature of an input video is encoded using the VLM. The action in the video is recognized, based on the text feature and the video feature, to localize the action within the video. A person performing the action is located within the video using the VLM.

Abstract: Systems and methods for a multi-entity tracking transformer model (MCTR). To train the MCTR, processing track embeddings and detection embeddings of video feeds obtained from multiple cameras to generate updated track embeddings with a tracking module. The updated track embeddings can be associated with the detection embeddings to generate track-detection associations (TDA) for each camera view and camera frame with an association module. A cost module can calculate a differentiable loss from the TDA by combining a detection loss, a track loss and an auxiliary track loss. A model trainer can train the MCTR using the differentiable loss and contiguous video segments sampled from a training dataset to track multiple objects with multiple cameras.

Abstract: A radiofrequency (RF) generator for use in a tissue puncture system for puncturing a tissue in a body is disclosed. The RF generator includes an RF energy source to couple to an active electrode of an RF puncture device and to couple to a return electrode and a controller coupled to the RF energy source. The controller causes the RF energy source to generate each of a puncture signal and a test signal, the test signal having a lower power than the puncture signal, measures an impedance of the test signal, determine a state of the active electrode based on the impedance measurement, and displays the state on a display device.

Abstract: Systems and methods are provided for encoding and decoding images using differentiable JPEG compression, including converting images from RGB color space to YCbCr color space to obtain a luminance and chrominance channels, and applying chroma subsampling to the chrominance channels to reduce resolution. The YCbCr image is divided into pixel blocks and a DCT is performed on the pixel blocks to obtain DCT coefficients. DCT coefficients are quantized using a scaled quantization table to reduce precision, and quantized DCT coefficients are encoded using lossless entropy coding, forming a compressed JPEG file decoded by reversing the lossless entropy coding to obtain quantized DCT coefficients, which are dequantized using the scaled quantization table to restore the precision. The dequantized DCT coefficients are converted back to a spatial domain using an IDCT, the chrominance channels are upsampled to original resolution, and the YCbCr image is converted back to the RGB color space.

Abstract: Systems and methods for optimizing multi-camera multi-entity artificial intelligence tracking systems. Visual and location information of entities from video feeds received from multiple cameras can be obtained by employing an entity detection model and re-identification model. Likelihood scores that entity detections belong to an entity track can be predicted from the visual and location information. The entity detections predicted into entity tracks can be processed by employing combinatorial optimization of the likelihood scores by identifying assumptions from the likelihood scores, entity detections, and the entity tracks, filtering the assumptions with unsatisfiable problems to obtain a filtered assumptions set, and optimizing an answer set by utilizing the filtered assumptions set and the likelihood scores to maximize an overall score and obtain optimized entity tracks. Multiple entities can be monitored by utilizing the optimized entity tracks.

Abstract: Systems and methods are provided for optimizing video compression using end-to-end learning, including capturing, using an edge device, raw video frames from a video clip and determining maximum network bandwidth. Predicting, using a control network implemented on the edge device, optimal codec parameters, based on dynamic network conditions and content of the video clip, encoding, using a differentiable surrogate model of a video codec, the video clip using the predicted codec parameters and to propagate gradients from a server-side vision model to adjust the codec parameters. Decoding, using a server, the video clip and analyzing the video clip with a deep vision model located on the server, transmitting, using a feedback mechanism, analysis from the deep vision model back to the control network to facilitate end-to-end training of the system. Adjusting the encoding parameters based on the analysis from the deep vision model received from the feedback mechanism.

Abstract: Methods and systems of training a machine learning model include identifying an object or person related to an action in a first video. The object or person is copied from the first video to a second video to generate a third video. A machine learning model is trained using the first video and the third video.

Abstract: Systems and methods are provided for optimizing video compression for remote vehicle control, including capturing, capturing video and sensor data from a vehicle using a plurality of sensors and high-resolution cameras, analyzing the captured video to identify critical regions within frames of the video using an attention-based module. Current network bandwidth is assessed and future bandwidth availability is predicted. Video compression parameters are predicted based on an analysis of the video and an assessment of the current network bandwidth using a control network, and the video is compressed based on the predicted parameters with an adaptive video compression module. The compressed video and sensor data is transmitted to a remote-control center, and received video and sensor data is decoded at the remote-control center. The vehicle is autonomously or remotely controlled from the remote-control center based on the decoded video and sensor data.

Abstract: Methods and systems for training a model include performing spatial augmentation on an unlabeled input video to generate spatially augmented video. Temporal augmentation is performed on the input video to generate temporally augmented video. Predictions are generated, using a model that was pre-trained on a labeled dataset, for the unlabeled input video, the spatially augmented video, and the temporally augmented video. Parameters of the model are adapted using the predictions while enforcing temporal consistency, temporal consistency, and historical consistency. The model may be used for action recognition in a healthcare context, with recognition results being used for determining whether patients are performing a rehabilitation exercise correctly.

Abstract: Methods and systems for tracking movement include performing person detection in frames from multiple video streams to identify detection images. Visual and location information from the detection images are combined to generate scores for pairs of detection images across the multiple video streams and across frames of respective video streams. A pairwise detection graph is generated using the detection images as nodes and the scores as weighted edges. A current view of the multiple video streams is changed to a next view of the multiple video streams, responsive to a determination that a score between consecutive frames of the view is below a threshold value and that a score between coincident frames of the current view and the next view is above the threshold value.

Abstract: Methods and systems for tracking movement include performing person detection in frames from multiple video streams to identify detection images. Visual and location information from the detection images are combined to generate scores for pairs of detection images across the multiple video streams and across frames of respective video streams. A pairwise detection graph is generated using the detection images as nodes and the scores as weighted edges. Movement of an individual is tracked based a constrained answer set programming problem, with constraints determined based on matching scores and logical assumptions. An action responsive to the tracked movement is performed. Tracking of movement of a patient in a healthcare facility can be used to inform treatment decisions by healthcare professionals.

Abstract: Methods and systems for temporal action localization include processing a video stream to identify an action and a start time and a stop time for the action using a neural network model that separately processes information of appearance and motion modalities from the video stream using transformer branches that include a self-attention and a cross-attention between the appearance and motion modalities. An action is performed responsive to the identified action.

Abstract: A system and method for a distributed solar power generation and communications system that includes one or more solar power generation and communications system units mounted on utility poles and distributed across a region. Each solar power generation and communications system unit includes one or more solar panels, a micro-converter, a meter, and a modem. Each unit is configured to communicate solar power production/consumption data to a remote monitoring and forecast system that estimates the unit's power production/consumption over a future period of time. A system and method is also provided for a solar power generation and communications system unit that includes a protective shroud and installation processes for mounting the unit on a utility pole and electrically coupling the unit to an utility electric grid.