Abstract: Machine learning is performed using synthetic data for neural network training using vectors. Facial images are obtained for a neural network training dataset. Facial elements from the facial images are encoded into vector representations of the facial elements. A generative adversarial network (GAN) generator is trained to provide one or more synthetic vectors based on the one or more vector representations, wherein the one or more synthetic vectors enable avoidance of discriminator detection in the GAN. The training a GAN further comprises determining a generator accuracy using the discriminator. The generator accuracy can enable a classifier, where the classifier comprises a multi-layer perceptron. Additional synthetic vectors are generated in the GAN, wherein the additional synthetic vectors avoid discriminator detection. A machine learning neural network is trained using the additional synthetic vectors.

Abstract: Disclosed techniques include in-vehicle drowsiness analysis using blink-rate. Video of an individual is obtained within a vehicle using an image capture device. The video is analyzed using one or more processors to detect a blink event based on a classifier for a blink that was determined. Using temporal analysis, the blink event is determined by identifying that eyes of the individual are closed for a frame in the video. Using the blink event and one or more other blink events, blink-rate information is determined using the one or more processors. Based on the blink-rate information, a drowsiness metric is calculated using the one or more processors. The vehicle is manipulated based on the drowsiness metric. A blink duration of the individual for the blink event is evaluated. The blink-rate information is compensated. The compensating is based on demographic information for the individual.

Abstract: Machine learning is performed using synthetic data for neural network training using vectors. Facial images are obtained for a neural network training dataset. Facial elements from the facial images are encoded into vector representations of the facial elements. A generative adversarial network (GAN) generator is trained to provide one or more synthetic vectors based on the one or more vector representations, wherein the one or more synthetic vectors enable avoidance of discriminator detection in the GAN. The training a GAN further comprises determining a generator accuracy using the discriminator. The generator accuracy can enable a classifier, where the classifier comprises a multi-layer perceptron. Additional synthetic vectors are generated in the GAN, wherein the additional synthetic vectors avoid discriminator detection. A machine learning neural network is trained using the additional synthetic vectors.

Abstract: Disclosed techniques include in-vehicle drowsiness analysis using blink-rate. Video of an individual is obtained within a vehicle using an image capture device. The video is analyzed using one or more processors to detect a blink event based on a classifier for a blink that was determined. Using temporal analysis, the blink event is determined by identifying that eyes of the individual are closed for a frame in the video. Using the blink event and one or more other blink events, blink-rate information is determined using the one or more processors. Based on the blink-rate information, a drowsiness metric is calculated using the one or more processors. The vehicle is manipulated based on the drowsiness metric. A blink duration of the individual for the blink event is evaluated. The blink-rate information is compensated. The compensating is based on demographic information for the individual.

Abstract: Drowsiness mental state analysis is performed using blink rate. Video is obtained of an individual or group. The individual or group can be within a vehicle. The video is analyzed to detect a blink event based on a classifier, where the blink event is determined by identifying that eyes are closed for a frame in the video. A blink duration is evaluated for the blink event. Blink-rate information is determined using the blink event and one or more other blink events. The evaluating can include evaluating blinking for a group of people. The blink-rate information is compensated to determine drowsiness, based on the temporal distribution mapping of the blink-rate information. Mental states of the individual are inferred for the blink event based on the blink event, the blink duration of the individual, and the blink-rate information that was compensated. The compensating is biased based on demographic information of the individual.

Abstract: Drowsiness mental state analysis is performed using blink rate. Video is obtained of an individual or group. The individual or group can be within a vehicle. The video is analyzed to detect a blink event based on a classifier, where the blink event is determined by identifying that eyes are closed for a frame in the video. A blink duration is evaluated for the blink event. Blink-rate information is determined using the blink event and one or more other blink events. The evaluating can include evaluating blinking for a group of people. The blink-rate information is compensated to determine drowsiness, based on the temporal distribution mapping of the blink-rate information. Mental states of the individual are inferred for the blink event based on the blink event, the blink duration of the individual, and the blink-rate information that was compensated. The compensating is biased based on demographic information of the individual.

Abstract: What is disclosed is a system and method for selecting a region of interest for extracting physiological parameters from a video of a subject. In one embodiment the present method involves performing the following. First, time-series signals are received which have been generated by having processing image frames of a video of a subject captured using a single band video camera with a bandpass filter with a pass band in a wavelength range of 495-565 nm and/or 800-1000 nm. The regions of interest are areas where a plethysmographic signal can be detected by the camera. Each time-series signal is associated with a different region of interest. A signal strength is then calculated for each of the time-series signals. The region associated with the time-series signal having a highest signal strength is selected. The time-series signal associated with the selected region can be processed to extract a videoplethysmographic (VPG) signal containing physiological parameters.

Abstract: What is disclosed is a system and method for determining respiration rate from a video of a subject. In one embodiment, a video is received comprising plurality of time-sequential image frames of a region of a subject's body. Features of pixels are extracted from that region from each image frame and vectors formed from these features. Each image frame has an associated feature vector. A N×M video matrix of the vectors of length N is constructed such that a total number of columns M in the video matrix correspond to a time duration over which the subject's respiration rate is to be determined. The video matrix is processed to obtain a matrix of eigenvectors where principal axes of variations due to motion associated with respiration are contained in a first few eigenvectors. One eigenvector is selected from the first few eigenvectors. A respiration rate is obtained from the selected eigenvector.

Abstract: What is disclosed is a system and method for extracting photoplethysmographic (PPG) signal (i.e., a cardiac signal) on a continuous basis from a time-series signals obtained from video images captured of a subject being monitored for cardiac function in a non-contact remote sensing environment involves the following. First, a time-series signal obtained from video images captured of a region of exposed skin where a photoplethysmographic (PPG) signal of a subject of interest can be registered. A sliding window is then used to define consecutive sequential segments of the time-series signal for processing. Each of the consecutive time-series signal segments is detrended such that low frequency variations and non-stationary components are removed. The detrended signals are processed to obtain, for each segment, a PPG signal. The PPG signal segments are then stitched together using a stitching method, as disclosed herein, to obtain a continuous PPG signal for the subject.

Abstract: What is disclosed is a video system and method that accounts for differences in imaging characteristics of differing video systems used to acquire video of respective regions of interest of a subject being monitored for a desired physiological function. In one embodiment, video is captured using N video imaging devices, where N?2, of respective regions of interest of a subject being monitored for a desired physiological function (i.e., a respiratory or cardiac function). Each video imaging device is different but has complimentary imaging characteristics. A reliability factor f is determined for each of the devices in a manner more fully disclosed herein. A time-series signal is generated from each of the videos. Each time-series signal is weighted by each respective reliability factor and combined to obtain a composite signal. A physiological signal can be then extracted from the composite signal. The processed physiological signal corresponds to the desired physiological function.

Abstract: What is disclosed is a system and method for compensating for motion induced artifacts in physiological signals extracted from a video of a subject being monitored for a physiological function in a non-contact, remote sensing environment. The present method identifies a center frequency from a physiological signal obtained from processing a prior video segment. Since a moment to moment change in pulse frequency from one video segment to a next is not very large, signals obtained from sequential video segments can be repeatedly processed and an adaptive band-pass filter repeatedly re-configured and used to filter a next video segment, and so on. Using the teachings disclosed herein, a motion-compensated continuous cardiac signal can be obtained for the subject for continuous monitoring of the subject's cardiac function via video imaging. The teachings hereof provide an effective means for compensating for movement by the subject during video acquisition. Various embodiments are disclosed.

Abstract: What is disclosed is a system and method for compensating for motion during processing of a video of a subject being monitored for physiological function assessment. In one embodiment, image frames are received. Successive batches of N video frames are processed to isolate pixels associated with a body region of the subject where a physiological signal is registered by the camera. The pixels are processed to obtain a time-series signal for each batch. A determination is made whether movement during video acquisition of this batch of image frames exceeds a threshold level. If so then a size N of the next batch of image frames is changed to: N=N+M1, where N+M1?Nmax. Otherwise, a size N of a next batch is changed to: N=N?M2, where N?M2?Nmin. Thereafter, processing repeats in a real-time continuous manner as the next batch of the N image frames is received. Various embodiments are disclosed.

Abstract: What is disclosed is a system for compensating for motion induced artifacts in a physiological signal obtained from multiple videos of a first and second region of interest a subject being monitored for a desired physiological function. At least one of the videos being of the first region and at least one of the videos being of the second region. The first region being at least one area of exposed skin where a desired signal corresponding to the physiological function can be registered by a video imaging device. The second region being an area where a movement by the subject is likely to induce motion artifacts into the signal. The videos are processed to isolate pixels associated with the first and second regions. Processed pixels of the isolated first regions to obtain a composite time-series signal. From the composite signal, a physiological signal corresponding to the physiological function is extracted.

Abstract: What is disclosed is a method for monitoring a subject for cardiac arrhythmia such as atrial fibrillation using an apparatus that can be comfortably worn by the subject around an area of exposed skin where a photoplethysmographic (PPG) signal can be registered. In one embodiment, the apparatus is a reflective or transmissive wrist-worn device with emitter/detector pairs fixed to an inner side of a band with at least one illuminator emitting source light at a specified wavelength band. The illuminator is paired to a respective photodetector comprising one or more sensors that are sensitive to a wavelength band of its paired illuminator. The photodetector measures intensity of sensed light emitted by a respective illuminator. The signal obtained by the sensors comprises a continuous PPG signal. The continuous PPG signal analyzed for peak-to-peak pulse points from which the existence of cardiac arrhythmia such as atrial fibrillation event can be determined.

Abstract: What is disclosed is a system and method for compensating for motion induce artifacts in a physiological signal obtained from a video. In one embodiment, a video of a first and second region of interest of a subject being monitored for a desired physiological function is captured by a video device. The first region is an area of exposed skin wherein a desired signal corresponding to the physiological function can be registered. The second region is an area where movement is likely to induce motion artifacts into that signal. The video is processed to isolate pixels in the image frames associated with these regions. Pixels of the first region are processed to obtain a time-series signal. A physiological signal is extracted from the time-series signal. Pixels of the second region are analyzed to identify motion. The physiological signal is processed to compensate for the identified motion.

Abstract: What is disclosed is a system and method for selecting a region of interest for extracting physiological parameters from a video of a subject. In one embodiment the present method involves performing the following. First, time-series signals are received which have been generated by having processing image frames of a video of a subject captured using a single band video camera with a bandpass filter with a pass band in a wavelength range of 495-565 nm and/or 800-1000 nm. The regions of interest are areas where a plethysmographic signal can be detected by the camera. Each time-series signal is associated with a different region of interest. A signal strength is then calculated for each of the time-series signals. The region associated with the time-series signal having a highest signal strength is selected. The time-series signal associated with the selected region can be processed to extract a videoplethysmographic (VPG) signal containing physiological parameters.

Abstract: What is disclosed is a system and method for determining a subject of interest's arterial pulse transit time from time-varying source signals generated from video images. In one embodiment, a video imaging system is used to capture a time-varying source signal of a proximal and distal region of a subject of interest. The image frames are processed to isolate localized areas of a proximal and distal region of exposed skin of the subject. A time-series signal for each of the proximal and distal regions is extracted from the source video images. A phase angle is computed with respect to frequency for each of the time-series signals to produce respective phase v/s frequency curves for each region. Slopes within a selected cardiac frequency range are extracted from each of the phase curves and a difference is computed between the two slopes to obtain an arterial pulse transit time for the subject.

Abstract: A method, non-transitory computer-readable medium, and apparatus for adaptive sampling an ego-centric video to extract features for performing an analysis are disclosed. For example, the method captures the ego-centric video, determines a spatio-temporal location of interest within the ego-centric video, applies an adaptive sampling centered around the spatio-temporal location of interest to obtain one or more spatio-temporal patches, extracts one or more features using the one or more spatio-temporal patches and performs an analysis based on the one or more features.

Abstract: What is disclosed is a system and method for estimating cardiac pulse rate from a video of a subject being monitored for cardiac function. In one embodiment, batches of overlapping image frames are continuously received and processed by isolating regions of exposed skin. Pixels of the isolated regions are processed to obtain a time-series signal per region and a physiological signal is extracted from each region's time-series signals. The physiological signal is processed to obtain a cardiac pulse rate for each region. The cardiac pulse rate for each region is compared to a last good cardiac pulse rate from a previous batch to obtain a difference. If the difference exceeds a threshold, the cardiac pulse rate is discarded. Otherwise, it is retained. Once all the regions have been processed, the retained cardiac pulse rate with a minimum difference becomes the good cardiac pulse rate for comparison on a next iteration.

Abstract: What is disclosed is a system and method for determining whether a subject is in atrial fibrillation. A video is received of a region of exposed skin of a subject. The video is acquired of a region where a videoplethysmographic (VPG) signal can be registered by at least one imaging channel of a video imaging device. For each batch of image frames, pixels associated with the region of exposed skin are isolated and processed to obtain a time-series signal. A VPG signal is extracted from the time-series signal. The power spectral density (PSD) is computed across all frequencies within the VPG signal. A pulse harmonic strength (PHS) is calculated for this VPG signal. The pulse harmonic strength is compared to a discrimination threshold, defined herein. A determination is made whether the subject in the video is in atrial fibrillation or in normal sinus rhythm.