Abstract: A head-mounted device may include one or more sensors for capturing images of a real-world environment and for determining where the head-mounted device is facing in the real-world environment. The sensors can be used to capture images for creating a lighting map that indicates where one or more light sources are located within the environment and for creating a physical map representing a geometry of the environment. The sensors can be used to predict a direction in which the head-mounted device will be facing in the real-world environment. One of the sensors can include an image sensor for capturing images that are displayed to a user of the head-mounted device. One or more settings of the image sensor can be adjusted based on the lighting map and based on the predicted direction of where the head-mounted device will be facing when capturing a future frame.

Abstract: Methods and apparatus for specular surface mapping in which a camera detects reflections of a light source from a specular surface. The detected light sources may be projected onto a celestial sphere as virtual point sources. True positive observations should be tightly clustered on the celestial sphere; thus, false positives may be identified and removed. Specular surface information may then be determined from clusters of the virtual point sources on the celestial sphere. The clusters of virtual point sources on the celestial sphere may be identified and used to identify a surface as a specular surface. The clusters may also be used to extract other information regarding the specular surface, including but not limited to distance to and extent of the specular surface.