Abstract: A cyber-physical system for conducting a telehealth session. In embodiments, the system includes a hardware control box that enables patients (e.g., elderly or cognitively-impaired patients) to easily participate in the telehealth session. In some embodiments, the system analyzes sensor data (e.g., thermal images, eye tracking data, etc.) and calculates state variables that form a mathematical representation (a “digital twin”) indicative of the physical, emotive, cognitive, or social state of the patient. In some of those embodiments, the system calculates Myasthenia Gravis core examination metrics by performing computer vision analysis on patient video data and/or audio analysis on patient audio data. In some embodiments, the system controls a pan-tile-zoom camera to zoom in on each region of interest that is relevant to the examination.

Abstract: Due to the precautions put in place during the COVID-19 pandemic, utilization of telemedicine has increased quickly for patient care and clinical trials. Unfortunately, teleconsultation is closer to a video conference than a medical consultation with the current solutions setting the patient and doctor into a discussion that relies entirely on a two-dimensional view of each other. A telehealth platform is augmented by a digital twin of the patient that assists with diagnostic testing of ocular manifestations of myasthenia gravis. A hybrid algorithm combines deep learning with computer vision to give quantitative metrics of ptosis and ocular muscle fatigue leading to eyelid droop and diplopia. The system works both on a fixed image and video in real time allowing capture of the dynamic muscular weakness during the examination.