Abstract: In one embodiment, a method includes recording, by a camera of a client device, a video of a region of a first user's skin and estimating, from the recorded video, a current ratio of ratios (RoR) for the first user corresponding to the first user's current blood-oxygen saturation. The method further includes converting the first user's determined RoR to a transformed RoR for the first user based at least in part on a baseline user's RoR determined while generating a trained SpO2 prediction model. The trained SpO2 prediction model is trained to estimate the baseline user's SpO2 value based on an input RoR value from the baseline user. The method further includes determining, by the trained SpO2 prediction model and based on the transformed RoR for the first user, the first user's current estimated blood-oxygen saturation.

Abstract: A method performed by at least one processor includes obtaining an image of a subject; preprocessing the image of the subject; inputting the preprocessed image into a machine learning model trained in accordance with a first frequency distribution corresponding to a first ground truth obtained from one or more sensors performing a vital measurement on one or more test subjects; and obtaining, from the machine learning model, an estimate of a signal corresponding to the vital measurement of the subject.

Abstract: In one embodiment, a method includes accessing a video of a user's face. The method further includes accessing, for each image frame in the video, (1) one or more facial landmarks determined by a facial landmark detection (FLD) model and (2) a corresponding determined position in the image for each facial landmark. The method further includes determining, based on the one or more facial landmarks and corresponding positions, a motion of the user's face in the captured video; extracting, from the determined motion of the user's face, a corrected motion signal of the user's face; adjusting, based on the extracted corrected motion signal of the user's face, the positions of one or more facial landmarks in the image frames; and determining, based at least in part on the adjusted positions of the facial landmarks in the sequential images of the video, one or more vital signs of the user.

Abstract: A method includes receiving, by an electronic device, sensor data during a spirometry test of a user, the sensor data comprising audio data of the user, image data of a face of the user, and distance data of a distance from the face of the user. The method also includes obtaining, by the electronic device, at least one measured parameter associated with the spirometry test that is determined using at least one of the audio data, the image data, or the distance data, wherein the at least one measured parameter is correlated with an amount of air volume exchange or an amount of exhalation force by the user during the spirometry test. The method further includes providing, by the electronic device, real time feedback during the spirometry test, the real time feedback based on the at least one measured parameter, the real time feedback indicating whether or not the user is performing the spirometry test correctly.

Abstract: A method includes capturing a video of a person's face using a camera. The method also includes determining a motion-based respiratory rate (RR) and a motion-based respiratory signal based on the video of the person's face. The method further includes determining a remote photoplethysmography (rPPG)-based RR and an rPPG-based respiratory signal based on the video of the person's face. The method also includes predicting whether the motion-based RR or the rPPG-based RR is more likely to be accurate using a trained machine learning model that receives the motion-based respiratory signal and the rPPG-based respiratory signal as input. In addition, the method includes presenting one of the motion-based RR or the rPPG-based RR based on the prediction.

Abstract: Pulmonary function estimation can include detecting one or more cough events from a time series of audio signals generated by an electronic device of a user. Based on the one or more cough events, one or more lung function metrics of the user can be determined.

Abstract: In particular embodiments, a method includes accessing, by a computing device, sensor data from a sensor of a client device corresponding to a user. The method further includes determining one or more segments of sensor data and selecting at least one segment of sensor data. The method further concludes determining, based at least on the selected segments of sensor data, one or more features corresponding to the user's pulmonary condition. The method further includes determining, based at least on the determined features, an assessment of the user's pulmonary condition.

Abstract: A method includes receiving, by an electronic device, sensor data during a spirometry test of a user, the sensor data comprising audio data of the user and distance data of a distance from a face of the user to the electronic device. The method also includes obtaining, by the electronic device, an amount of air volume exchange and at least one pulmonary health parameter that are determined based on the audio data and the distance data. The method also includes presenting an indicator on a display of the electronic device for use by the user or a medical provider, the indicator representing the amount of air volume exchange.

Abstract: In one embodiment, a method includes accessing a sequence of images of a portion of a person's skin illuminated by ambient light and determining, from at least one of the images, a plurality of microregions in the portion of the person's skin. The method further includes determining, based on a similarity between particular microregions of the plurality of microregions, one or more regions of interest of the person's skin; determining, for each of the regions of interest, a remote photoplethysmogram (rPPG) signal based on the sequence of images; and determining, based on one or more of the rPPG signals, an estimate of an oxygen saturation in the person's blood.

Abstract: In one embodiment, a method includes accessing a plurality of images of a region of interest of a person's skin and extracting, from the plurality of images, a color signal of the region of interest as a function of time. The method further includes determining, based at least on the color signal, a first quality associated with an estimated vital sign of the person, where the vital sign estimate is determined from the plurality of images.

Abstract: A method includes obtaining, by an electronic device, an audio segment comprising one or more audio events of a target subject. The method also includes extracting, by the electronic device, audio embeddings from the one or more audio events using an embedding model, the embedding model comprising a trained machine learning model. The method further includes comparing, by the electronic device, the extracted audio embeddings with a match profile of the target subject, the match profile generated during an enrollment stage. The method also includes generating, by the electronic device, a label for the audio segment based on whether or not the extracted audio embeddings match the match profile, wherein the label enables correlation of the audio segment with the target subject for monitoring a health condition of the target subject.

Abstract: Adaptive respiratory condition assessment can include capturing signals generated by sensors of a portable device positioned to sense a user's body. The sensors can generate the signals in response to sensor-detected movements of the user's body measured along multiple axes and corresponding to lung activity in the user's body during respiratory cycles. A preferred axis of measurement can be selected using signal processing performed by a signal processor embedded in the portable device. Waveforms generated from signals corresponding to signals generated in response to movements of the user's body measured along the preferred axis can be filtered for extracting biomarkers from the waveforms using the signal processor. The one or more biomarkers extracted correspond to the lung activity. Based on the biomarkers, a respiratory condition of the user can be determined. A notification based on the respiratory condition as determined can be generated and conveyed with the portable device.

Abstract: A multimodal, contactless vital sign monitoring system is configured to perform the following operations. Images are received from a video capture device. An image of a subject is identified within the images. The image of the subject is segmented into a plurality of segments. A first analysis is performed on the plurality of segments to identify a color feature. A second analysis is performed of the plurality of segments to identify a motion feature. Using a combination of the color feature and the motion feature a plurality of vital signs for the subject are determined. The first analyzing and the second analyzing are performed in parallel. The plurality of vital signs include one or more of heart rate, respiration rate, oxygen saturation, heart rate variability, and atrial fibrillation.

Abstract: Device-invariant, frequency-domain signal processing with machine learning includes retrieving with a host device a device-specific alien dataset corresponding to an alien device. The device-specific alien dataset is retrieved from a remote data storage device communicatively coupled with the host device. A plurality of frequency-domain features are extracted from the device-specific alien dataset and a machine learning model is trained using the plurality of frequency-domain features. The host device extracts frequency-domain features from signals generated by sensors operatively coupled with the host device. Real-time frequency bin adaptation of the frequency-domain features extracted by the host device is performed. Based on the frequency-domain features extracted by the host device, as adapted, an inference is performed using the machine learning model.

Abstract: Pulmonary assessment based on speech can include identifying one or more audio features and speech patterns of a user's speech. A cognitive burden associated with the user's speech can be determined.

Abstract: A method for pulmonary condition monitoring includes selecting a phrase from an utterance of a user of an electronic device, wherein the phrase matches an entry of multiple phrases. At least one speech feature that is associated with one or more pulmonary conditions within the phrase is identified. A pulmonary condition is determined based on analysis of the at least one speech feature.

Abstract: A method includes identifying, by an electronic device, one or more segments within a first audio recording that includes one or more non-speech segments and one or more speech segments. The method also includes generating, by the electronic device, one or more synthetic speech segments that include natural speech audio characteristics and that preserve one or more non-private features of the one or more speech segments. The method also includes generating, by the electronic device, an obfuscated audio recording by replacing the one or more speech segments with the one or more synthetic speech segments while maintaining the one or more non-speech segments, wherein the one or more synthetic speech segments prevent recognition of some content of the obfuscated audio recording.

Abstract: Early detection of infectious respiratory diseases includes monitoring, during a predetermined time interval, electronic communications generated by network-connected portable devices, each portable device used by one of a plurality of users. The electronic communications are processed by extracting from each electronic communication biomarkers obtained by processing signals captured by the portable devices, the signals generated in response to sensor-detected respiratory activity of the portable device users. Clusters of users are generated based on the biomarkers and are conditioned based on pre-condition data extracted from the electronic communications, the pre-condition data corresponding to users determined to have pre-existing health conditions.

Abstract: Adaptive respiratory condition assessment can include capturing signals generated by sensors of a portable device positioned to sense a user's body. The sensors can generate the signals in response to sensor-detected movements of the user's body measured along multiple axes and corresponding to lung activity in the user's body during respiratory cycles. A preferred axis of measurement can be selected using signal processing performed by a signal processor embedded in the portable device. Waveforms generated from signals corresponding to signals generated in response to movements of the user's body measured along the preferred axis can be filtered for extracting biomarkers from the waveforms using the signal processor. The one or more biomarkers extracted correspond to the lung activity. Based on the biomarkers, a respiratory condition of the user can be determined. A notification based on the respiratory condition as determined can be generated and conveyed with the portable device.

Abstract: A method includes receiving, by an electronic device, sensor data during a spirometry test of a user, the sensor data comprising audio data of the user, image data of a face of the user, and distance data of a distance from the face of the user. The method also includes obtaining, by the electronic device, at least one measured parameter associated with the spirometry test that is determined using at least one of the audio data, the image data, or the distance data, wherein the at least one measured parameter is correlated with an amount of air volume exchange or an amount of exhalation force by the user during the spirometry test. The method further includes providing, by the electronic device, real time feedback during the spirometry test, the real time feedback based on the at least one measured parameter, the real time feedback indicating whether or not the user is performing the spirometry test correctly.