Abstract: A system enables a user to query based on a user's gaze by receiving a query from the user and capturing, via an eye tracking system on a headset, the user's gaze location near an object in a local area. The system captures one or more images of the local area with the object and formats the images based in part on a region of interest in the one or more images that includes the object. The system generates a formatted query based in part on the query. The formatted query is provided to a search engine. Information describing the object determined from the one or more formatted images and information describing the query are used by the search engine to determine an answer to the query about the object. The system presents the answer to the query about the object.

Abstract: A differential camera system for object tracking. The system includes a co-aligned light source camera assembly (LSCA) and a controller. The co-aligned LSCA includes a light source and a differential camera sensor. The light source is configured to emit light along an optical path that is directed towards an eye box including an eye of a user. The differential camera sensor is configured to detect a change in brightness of the eye caused in part by the emitted light, asynchronously output data samples corresponding to the detected change in brightness, wherein the optical path is substantially co-aligned with an optical path of the differential camera sensor. The controller is configured to identify a pupil of the eye based on data samples output from the differential camera sensor resulting from the emitted light, and determine a gaze location of the user based in part on the identified pupil.

Abstract: A tracking system for object detection and tracking. The system may include a plurality of light sources, a differential camera, and a controller. The plurality of light sources is positioned at different locations on a device and are configured to emit pulses of light that illuminate an object. The differential camera has an optical axis, and at least some of the plurality of light sources are off-axis relative to the optical axis. The differential camera is configured to detect a change in brightness of the object caused in part by one or more of the pulses of light, and asynchronously output data samples corresponding to the detected change in brightness. The controller is configured to track the object based in part on the data samples output by the differential camera.

Abstract: Embodiments relate to an eye tracking system. A headset of the system includes an eye tracking sensor that captures eye tracking data indicating positions and movements of a user's eye. A controller (e.g., in the headset) of the tracking system analyzes eye tracking data from the sensors to determine eye tracking feature values of the eye during a time period. The controller determines an activity of the user during the time period based on the eye tracking feature values. The controller updates an activity history of the user with the determined activity.