Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, directed to an interactive procedural guidance system are disclosed. In one aspect, a method includes the actions of receiving, by an interactive procedural system, first sensor data that reflects first characteristics of an environment where a user wearing an augmented, mixed, or extended reality headset is located. The actions further include determining a location of an object within a field of view of the AR/MR/XR headset. The actions further include identifying a type of the object. The actions further include determining a skin that is configured to be displayed to the user through the headset over the object. The actions further include receiving second sensor data that reflects second characteristics of the environment. The actions further include determining that an event has occurred with respect to the object. The actions further include updating the skin.