Abstract: A tracking architecture is provided that enables data for gestures and head positions to be provided to both native and non-native clients on a computing device. A pipeline component can obtain the raw image data and sensor data and synchronize that data to be processed to determine, for example, location and/or motion data that may correspond to device input. The data can be processed by separate components, such as an event publisher and an event provider, each capable of filtering the location, motion, and/or raw sensor data to generate a set of event data. The event data then can be published to registered listeners or provided in response to polling requests. Head coordinates, gesture data, and other such information can be passed through one or more interface layers enabling the data to be processed by a non-native client on the device.

Abstract: A device configured with a user interface (UI) that changes based on a position of a user determines the position of the user through multiple data sources including camera based head tracking and output from motion sensors such as a gyroscope. Each data source may output its own estimated head position. The device may apply a reliability weight to the head position determined by each data source. A composite head position is then determined from the weighted position. The composite position is then used to render the UI.

Abstract: Object tracking, such as may involve face tracking, can utilize different detection templates that can be trained using different data. A computing device can determine state information, such as the orientation of the device, an active illumination, or an active camera to select an appropriate template for detecting an object, such as a face, in a captured image. Information about the object, such as the age range or gender of a person, can also be used, if available, to select an appropriate template. In some embodiments instances of templates can be used to process various orientations, while in other embodiments specific orientations, such as upside down orientations, may not be processed for reasons such as rate of inaccuracies or infrequency of use for the corresponding additional resource overhead.

Abstract: A tracking architecture is provided that enables data for gestures and head positions to be provided to both native and non-native clients on a computing device. A pipeline component can obtain the raw image data and sensor data and synchronize that data to be processed to determine, for example, location and/or motion data that may correspond to device input. The data can be processed by separate components, such as an event publisher and an event provider, each capable of filtering the location, motion, and/or raw sensor data to generate a set of event data. The event data then can be published to registered listeners or provided in response to polling requests. Head coordinates, gesture data, and other such information can be passed through one or more interface layers enabling the data to be processed by a non-native client on the device.