Abstract: Disclosed are a method and apparatus for manipulating an object in virtual or augmented reality based on a hand motion capture apparatus providing haptic feedback. The method of manipulating an object in virtual or augmented reality based on a hand motion capture apparatus providing haptic feedback includes receiving a value of a sensor at a specific position in a finger from the hand motion capture apparatus, estimating a motion of the finger based on the value of the sensor and adjusting a motion of a virtual hand, detecting contact of the adjusted virtual hand with a virtual object, and upon detecting the contact with the virtual object, providing feedback to the user using the hand motion capture apparatus, wherein the virtual hand is modeled for each user.