Abstract: A mechanism for interpreting spatiotemporal data from augmented and virtual reality devices into fitness related metrics and recommendations. Spatiotemporal data can be compared to the physical data of the end user to interpret energy expense such as estimated calorie burn. A virtual reality or augmented reality device can report positional data over time as the user engages with a corresponding environment. The positional data is compared to a biometric profile of the end user to understand how their physical body is exerting itself. The spatiotemporal data is a series of coordinate data expressed over time. The biometric data is the physical profile of the end user. This data is used to understand and quantify physical movement while using a virtual reality or augmented reality device. The energy expense determination mechanism can be used to provide a virtual coach for activities and workouts, games involving physical movement, and other applications.