Abstract: A system, computer readable medium and method trains a video transformer, using a machine learning engine, for human action recognition in a video. The method includes sampling video clips with varying temporal resolutions in global views and sampling the video clips from different spatiotemporal windows in local views. The machine learning engine is configured to match the global and local views in a framework of student-teacher networks to learn cross-view correspondence between local and global views, and to learn motion correspondence between varying temporal resolutions. The video transformer can output for display video clips in a manner that emphasizes attention to the recognized human action.

Abstract: A system, method, computer readable storage medium for a computer vision system includes at least one video camera, and video processor circuitry. The method includes inputting a stream of video data and generating a sequence of image frames, segmenting and tracking, by the video analysis apparatus, object instances in the stream of video data, including receiving the sequence of image frames, analyzing the sequence of image frames using a video instance segmentation transformer to obtain a video instance mask sequence from the sequence of image frames, the transformer having a backbone network, a transformer encoder-decoder, and an instance matching and segmentation block, The encoder contains a multi-scale spatio-temporal split attention module to capture spatio-temporal feature relationships at multiple scales across multiple frames. The decoder contains a temporal attention block for enhancing a temporal consistency of transformer queries. The method includes displaying the video instance mask sequence.

Abstract: An apparatus, computer readable storage medium and method of generating a diverse set of images from few-shot images, includes a parameter input receiving values for control parameters to control an extent to which each reference image impacts a newly generated image. The apparatus involves an image generation deep learning network for generating an image for each of the values for the control parameters. The deep learning network has an encoder, a transformer-based fusion block, and a decoder. The transformer-based fusion block includes a mapping network that computes meta-weights from features extracted from the reference images and the control parameters, and a cross-attention block to generate modulation weights based on the meta-weights. An output displays high-quality and diverse images generated based on the values for the control parameter.

Abstract: A video system and method for person search includes video cameras for capturing video images, a display device, and a computer system. The computer system including a deep learning network to determine person images, from among the video images, matching a target query person. The deep learning network having a person detection branch, a person re-identification branch, and an attention-aware relation mixer connected to the person detection branch and to the person re-identification branch. The attention-aware relation mixer including a relation mixer having a spatial and channel mixer that performs spatial attention followed by spatial mixing (tokenized multi-layered perceptron) and channel attention followed by channel mixing (channel multi-layered perceptron), and a joint spatio-channel attention layer that utilizes 3D attention weights to modulate 3D spatio-channel region of interest features and aggregate the features with output of the relation mixer.

Abstract: A mobile device and mobile application, in which the mobile device includes a camera having an image capture circuit operating in a mode to capture a RAW image burst, and processing circuitry, including a neural network engine, to generate a single enhanced image from the RAW image burst. The neural network engine executing program instructions including an edge boosting feature alignment stage to remove inter-frame spatial and color misalignment from the RAW image burst to obtain aligned burst frames, a pseudo-burst feature fusion stage to create a set of pseudo-burst features that combine complementary information from the aligned burst frames, and an adaptive group upsampling stage to progressively increase spatial resolution while merging the set of pseudo-burst features and output the single enhanced image. The mobile application and mobile device perform super-resolution, low-light image enhancement, and burst denoising using a RAW image burst.

Abstract: A system, method and computer readable medium for emergency health response, including sensors for measuring health conditions of a user, a local machine learning device to predict abnormalities in health status of the user based on the measurements, a communications device for transmitting an emergency alert message to emergency response providers that are within range of the communications device, and for receiving response messages from emergency response providers that are available to provide emergency treatment. A health condition controller selecting a provider. When the provider is a hospital, the subject vehicle will set its destination to the hospital and will transmit health status information of the user to the provider. When the provider is an emergency response vehicle, the subject vehicle will communicate coordinates as a meeting destination for meeting the provider response vehicle and will transmit health status information of the user to the provider response vehicle.

Abstract: A system, computer-readable storage medium and method for prognosis of head and neck cancer, includes an input for receiving electronic health records (EHR) of a patient, an input for receiving multimodal images of a head and neck area of the patient, a feature extraction module for converting the electronic health records and multimodal images into at least one feature vector, a hybrid machine learning architecture that includes a multi-task logistic regression (MTLR) model and a multi-layer artificial neural network, the hybrid architecture takes as input the at least one feature vector and outputs a final risk score of prognosis for head and neck cancer for the patient.

Abstract: A method, apparatus, and system for detecting DeepFake videos, includes an input device for inputting a potential DeepFake video, the input device inputs a sequence of video frames of the video, and processing circuitry. The processing circuitry detects faces frame by frame in the video to obtain consecutive face images, creates UV texture maps from the face images, inputs both face images and corresponding UV texture maps, extracts image feature maps, by a convolution neural network (CNN) backbone, from the input face images and corresponding UV texture maps and forms an input data structure, receives the input data structure, by a video transformer model that includes multiple encoders, and computes, by the video transformer model, a classification of the video as being Real or Fake. A display device plays back the potential DeepFake video and an indication that the video is Real or Fake.

Abstract: A system and computer readable storage medium for automated handwriting generation, including a text input device for inputting a text query having at least one textual word string, an image input device for inputting a handwriting sample with characters in a writing style of a user, and a computer implemented deep learning transformer model including an encoder network and a decoder network in which each are a hybrid of convolution and multi-head self-attention networks. The encoder produces a sequence of style feature embeddings from the input handwriting sample. The decoder takes the sequence of style feature embeddings in order to convert the at least one textual word string into a generated handwritten image having substantially same writing style as the handwriting sample. An output device to output the generated handwriting image.

Abstract: A system and computer readable storage medium for automated handwriting generation, including a text input device for inputting a text query having at least one textual word string, an image input device for inputting a handwriting sample with characters in a writing style of a user, and a computer implemented deep learning transformer model including an encoder network and a decoder network in which each are a hybrid of convolution and multi-head self-attention networks. The encoder produces a sequence of style feature embeddings from the input handwriting sample. The decoder takes the sequence of style feature embeddings in order to convert the at least one textual word string into a generated handwritten image having substantially same writing style as the handwriting sample. An output device to output the generated handwriting image.