Abstract: Interesting features of an object within a physical environment are identified from a video of the physical environment. A model of the object is obtained, the model includes dimensions of the object. A first array is maintained with the pixel coordinates of the identified features within the video. A second array is maintained with the dimensions of the features obtained from the model. A processing framework is provided for using the first and second arrays and estimating, updating, and tracking the anchor pose in the physical environment within the video along with any AR generated objects placed in the video. The framework provided to an Augmented Reality (AR) app to initiate an AR session and to maintain the session.