Abstract: Part and state detection for gesture recognition is useful for human-computer interaction, computer gaming, and other applications where gestures are recognized in real time. In various embodiments a decision forest classifier is used to label image elements of an input image with both part and state labels where part labels identify components of a deformable object, such as finger tips, palm, wrist, lips, laptop lid and where state labels identify configurations of a deformable object such as open, closed, up, down, spread, clenched. In various embodiments the part labels are used to calculate a center of mass of the body parts and the part labels, centers of mass and state labels are used to recognize gestures in real time or near real-time.

Abstract: Methods and systems for detecting the location of a keyboard on a desktop. The method includes receiving an image of the desktop with the keyboard situated thereon and analyzing the image of the desktop to identify an area of the image corresponding to the keyboard. In one example, the image of the desktop is a depth image and analyzing the image of the desktop includes identifying an image element of the depth image that forms part of the keyboard, identifying first and second corners of the keyboard from the identified image element and determining the area of the image corresponding to the keyboard based on the first and second corners.

Abstract: Methods and system for controlling a computing-based device using both input received from a traditional input device (e.g. keyboard) and hand gestures made on or near a reference object (e.g. keyboard). In some examples, the hand gestures may comprise one or more hand touch gestures and/or one or more hand air gestures.

Abstract: A synthetic gesture trace generator is described. In an embodiment, a synthetic gesture trace is generated using a gesture synthesizer which may be implemented in software. The synthesizer receives a number of inputs, including parameters associated with a touch sensor to be used in the synthesis and a gesture defined in terms of gesture components. The synthesizer breaks each gesture component into a series of time-stamped contact co-ordinates at the frame rate of the sensor, with each time-stamped contact co-ordinate detailing the position of any touch events at a particular time. Sensor images are then generated from the time-stamped contact co-ordinates using a contact-to-sensor transformation function. Where there are multiple simultaneous contacts, there may be multiple sensor images generated having the same time-stamp and these are combined to form a single sensor image for each time-stamp. This sequence of sensor images is formatted to create the synthetic gesture trace.