Abstract: The present concepts include a neuroergonomic service that processes multimodal physiological, digital, and/or environmental inputs from a user and predicts cognitive states of the user. Thus, the neuroergonomic service provides personalized feedback to the user about her current mental and physiological wellbeing to enable modulation of mood, stress, attention, and other cognitive measures for improved productivity and satisfaction. The neuroergonomic service utilizes machine learning models that are trained offline using sensor inputs taken from participants in a controlled environment that purposefully induce an array of cognitive states upon the participants.

Abstract: Systems and methods are provided that are directed to generating video sequences including physio-realistic avatars. In examples, an albedo for an avatar is received, a sub-surface skin color associated with the albedo is modified based on physiological data associated with physiologic characteristic, and an avatar based on the albedo and the modified sub-surface skin color is rendered. The rendered avatar may then be synthesized in a frame of video. In some examples, a video including the synthesized avatar may be used to train a machine learning model to detect a physiological characteristic. The machine learning model may receive a plurality of video segments, where one or more of the video segments includes a synthetic physio-realistic avatar generated with the physiological characteristic. The machine learning model may be trained using the plurality of video segments. The trained model may be provided to a requesting entity.

Abstract: Systems and methods are directed to animating subtle physiological processes directly on avatars and photos. That is, physiologically-grounded spatial, color space, and temporal modifications may be made to the appearance of an avatar to simulate a physiological characteristic, such as blood flow. More specifically, a frame of a video sequence and a physiological signal may be received. An attention mask may be generated based on the received physiological signal, where the attention mask includes attention weights indicative of a strength of the physiological signal for differing portions of the frame of the video sequence. Accordingly, a pixel adjustment value based on the physiological signal and the attention mask may be generated and applied to an identified pixel in the frame of the video sequence.

Abstract: Systems and methods are provided that are directed to generating video sequences including physio-realistic avatars. In examples, an albedo for an avatar is received, a sub-surface skin color associated with the albedo is modified based on physiological data associated with physiologic characteristic, and an avatar based on the albedo and the modified sub-surface skin color is rendered. The rendered avatar may then be synthesized in a frame of video. In some examples, a video including the synthesized avatar may be used to train a machine learning model to detect a physiological characteristic. The machine learning model may receive a plurality of video segments, where one or more of the video segments includes a synthetic physio-realistic avatar generated with the physiological characteristic. The machine learning model may be trained using the plurality of video segments. The trained model may be provided to a requesting entity.

Abstract: Systems and methods are directed to animating subtle physiological processes directly on avatars and photos. That is, physiologically-grounded spatial, color space, and temporal modifications may be made to the appearance of an avatar to simulate a physiological characteristic, such as blood flow. More specifically, a frame of a video sequence and a physiological signal may be received. An attention mask may be generated based on the received physiological signal, where the attention mask includes attention weights indicative of a strength of the physiological signal for differing portions of the frame of the video sequence. Accordingly, a pixel adjustment value based on the physiological signal and the attention mask may be generated and applied to an identified pixel in the frame of the video sequence.

Abstract: Various methods and apparatus relating to estimating and mitigating a stress level of a user are disclosed herein. Methods can include collecting potential stress indicator data from the user interacting with a computing device. The potential stress indicator data can include one or more of environmental data and contextual data associated with the user. Methods can include estimating the stress level of the user based on the potential stress indicator data. Methods can include performing an evaluation of whether to mitigate the stress level of the user via one or more stress mitigation interventions. Methods can include presenting the one or more stress mitigation interventions to the user via a graphical user interface (GUI) when the evaluation indicates that the stress level should be mitigated.

Abstract: Systems and methods are directed to animating subtle physiological processes directly on avatars and photos. That is, physiologically-grounded spatial, color space, and temporal modifications may be made to the appearance of an avatar to simulate a physiological characteristic, such as blood flow. More specifically, a frame of a video sequence and a physiological signal may be received. An attention mask may be generated based on the received physiological signal, where the attention mask includes attention weights indicative of a strength of the physiological signal for differing portions of the frame of the video sequence. Accordingly, a pixel adjustment value based on the physiological signal and the attention mask may be generated and applied to an identified pixel in the frame of the video sequence.

Abstract: Systems and methods for providing a user characteristic to a service provider for a virtual conference with a user are provided. In particular, a computing device may collect raw media data associated with the user during the virtual conference between the user and the service provider. During the virtual conference, the computing device may perform a first processing of the raw media data to extract intermediate user data, wherein the intermediate user data comprises one or more of a physiological signal and a behavioral signal associated the user. The computing device may further transform the raw media data into transformed media data and transmit the transformed media data with the intermediate user data to a server for second processing of the intermediate user data.

Abstract: The present disclosure relate to highlighting audience members with reactions to a presenter of an online meeting. Unlike physical, fact-to-face meeting that enables spontaneous interactions among the presenter and the audiences that are collocated with the presenter, presenting materials during an online meeting raises an issue of the present not being able to see real-time reactions or feedback by the audience members. The present disclosure addresses the issue by dynamically determining one or more audience members who indicate reactions during the online meeting or presentation and displaying faces of the one or more audience members under spotlight to the presenter. The presenter sees faces of the audience members with reactions during the online presentation and responds to the audience members and keep the audience engaged. The spotlight audience server analyzes video frames and determines types of reactions of the audience members.

Abstract: Systems and methods are provided that are directed to generating video sequences including physio-realistic avatars. In examples, an albedo for an avatar is received, a sub-surface skin color associated with the albedo is modified based on physiological data associated with physiologic characteristic, and an avatar based on the albedo and the modified sub-surface skin color is rendered. The rendered avatar may then be synthesized in a frame of video. In some examples, a video including the synthesized avatar may be used to train a machine learning model to detect a physiological characteristic. The machine learning model may receive a plurality of video segments, where one or more of the video segments includes a synthetic physio-realistic avatar generated with the physiological characteristic. The machine learning model may be trained using the plurality of video segments. The trained model may be provided to a requesting entity.

Abstract: Systems and methods are directed to animating subtle physiological processes directly on avatars and photos. That is, physiologically-grounded spatial, color space, and temporal modifications may be made to the appearance of an avatar to simulate a physiological characteristic, such as blood flow. More specifically, a frame of a video sequence and a physiological signal may be received. An attention mask may be generated based on the received physiological signal, where the attention mask includes attention weights indicative of a strength of the physiological signal for differing portions of the frame of the video sequence. Accordingly, a pixel adjustment value based on the physiological signal and the attention mask may be generated and applied to an identified pixel in the frame of the video sequence.

Abstract: A conversational agent that is implemented as a voice-only agent or embodied with a face may match the speech and facial expressions of a user. Linguistic style-matching by the conversational agent may be implemented by identifying prosodic characteristics of the user's speech and synthesizing speech for the virtual agent with the same or similar characteristics. The facial expressions of the user can be identified and mimicked by the face of an embodied conversational agent. Utterances by the virtual agent may be based on a combination of predetermined scripted responses and open-ended responses generated by machine learning techniques. A conversational agent that aligns with the conversational style and facial expressions of the user may be perceived as more trustworthy, easier to understand, and create a more natural human-machine interaction.

Abstract: Method and system for measuring physiological parameters. The method includes capturing a sequence of images of a human face and identifying the location of the face in a frame of the video and establishing a region of interest including the face. Pixels are separated in the region of interest in a frame into at least two channel values forming raw traces over time. The raw traces are decomposed into at least two independent source signals. At least one of the source signals is processed to obtain a physiological parameter.