Abstract: Numerous examples are disclosed of a system and method for universal tracking of physical objects in a physical environment and generating accurate virtual counterparts within an extended reality (XR) environment. In one example, a magnetic tracking system comprises a first XR device to generate a XR environment for a first user, an electromagnetic field (EMF) source to generate EMF, a first tracked object, and a first magnetic sensor physically coupled to the first tracked object to receive the EMF and generate a first sequence of tracking data, wherein the XR environment generates a first virtual counterpart to the first tracked object.