Abstract: The technology described herein is generally directed towards a free hand, barehanded technique to provide user input to a device, such as to move a cursor on a user interface. Frames of images are captured, and each frame is processed to determine a fingertip position. The processing includes an image segmentation phase that provides a binary representation of the image, a disjoint union arborescence graph construction phase that operates on the binary representation of the image to construct set of arborescence graphs, and a fingertip location estimation phase that selects a graph from among the set of arborescence graphs and uses the root node to estimate the fingertip location. Also described is determining a hand orientation from the set of arborescence graphs.