Abstract: A video image based tracking method and apparatus for identifying and tracking encoded color surface. Encoded color patterns are integrated with tangible objects such as tangible toys or tangible input objects so that they can be used as a tangible input device for a computing system to control virtual multimedia objects and trigger game interactions. The color pattern within the field of sight of a video image capturing device is identified and tracked and by a computing system. Method to configure the color pattern is included. A great number of encoded patterns can be created. Method to identify the encoded color pattern is included. Method to estimate the 6-DOF (degree-of-freedoms) orientation and coordinates information of the tangible object with encoded color pattern is included. Multiple encoded color patterns can be identified and tracked simultaneously.