Abstract: A head-mounted device includes one or more eye-tracking cameras and one or more computer-readable hardware storage devices having stored thereon computer-executable instructions, including a machine-learned artificial intelligence (AI) model. The head-mounted device is configured to cause the one or more eye-tracking cameras to take a series of images of one or more areas of skin around one or more eyes of a wearer, and use the machine-learned AI model to analyze the series of images to extract a photoplethysmography waveform. A heart rate is then detected based on the photoplethysmography waveform.

Abstract: Frames of a video frame sequence capturing one or more skin regions of a body are provided to a first neural network. The first neural network generates respective appearance representations based on the frames. An appearance representation generated based on a particular frame is indicative of a spatial distribution of a physiological signal across the particular frame. Simultaneously with providing the frames to the first neural network, the frames are also provided to a second neural network. The second neural network determines the physiological signal based on the frames. Determining the physiological signal by the second neural network includes applying the appearance representations, generated by the first neural network, to outputs of one or more layers of the second neural network to emphasize regions, in the frames, that exhibit relatively stronger presence of the physiological signal and deemphasize regions, in the frames, that exhibit relatively weaker presence of physiological signal.

Abstract: Frames of a video frame sequence capturing one or more skin regions of a body are provided to a first neural network. The first neural network generates respective appearance representations based on the frames. An appearance representation generated based on a particular frame is indicative of a spatial distribution of a physiological signal across the particular frame. Simultaneously with providing the frames to the first neural network, the frames are also provided to a second neural network. The second neural network determines the physiological signal based on the frames. Determining the physiological signal by the second neural network includes applying the appearance representations, generated by the first neural network, to outputs of one or more layers of the second neural network to emphasize regions, in the frames, that exhibit relatively stronger presence of the physiological signal and deemphasize regions, in the frames, that exhibit relatively weaker presence of physiological signal.

Abstract: Described herein are various technologies pertaining to estimating affective states of a user by way of monitoring data streams output by sensors and user activity on a computing device. Models of valence, arousal, and engagement can be learned during a training phase, and such models can be employed to compute values that are indicative of valence, arousal, and engagement of a user in near-real time. A visualization that represents estimated affective states of a user over time is generated to facilitate user reflection.