Abstract: In one embodiment, a computing system accesses a set of 3D locations associated with features in an environment previously captured by a camera from a previous camera pose. The computing system determines a predicted camera pose using the previous camera pose and motion measurements generated using a motion sensor associated with the camera. The computing system projects the set of 3D locations toward the predicted camera pose and onto a 2D image plane associated with the camera. The computing system generates, based on the projected set of 3D locations on the 2D image plane, an activation map specifying a subset of the pixel sensors of the camera that are to be activated. The computing system instructs, using the activation map, the camera to activate the subset of pixel sensors to capture a new image of the environment. The computing system reads pixel values of the new image.

Abstract: In one embodiment, a computing system accesses a set of 3D locations associated with features in an environment previously captured by a camera from a previous camera pose. The computing system determines a predicted camera pose using the previous camera pose and motion measurements generated using a motion sensor associated with the camera. The computing system projects the set of 3D locations toward the predicted camera pose and onto a 2D image plane associated with the camera. The computing system generates, based on the projected set of 3D locations on the 2D image plane, an activation map specifying a subset of the pixel sensors of the camera that are to be activated. The computing system instructs, using the activation map, the camera to activate the subset of pixel sensors to capture a new image of the environment. The computing system reads pixel values of the new image.

Abstract: In one embodiment, a method for tracking includes capturing a first frame of the environment using a first camera, identifying, in the first frame, a first patch that corresponds to the first feature, accessing a first local memory of the first camera that stores reference patches identified in one or more previous frames captured by the first camera, and determining that none of the reference patches stored in the first local memory corresponds to the first feature. The method further includes receiving, from a second camera through a data link connecting the second camera with the first camera, a reference patch corresponding to the first feature. The reference patch is identified in a previous frame captured by the second camera and of the second camera. The method may then determine correspondence data between the first patch and the reference patch, and tracks the first feature in the environment based on the determined correspondence data.

Abstract: In one embodiment, a method for tracking includes capturing a first frame of the environment using a first camera, identifying, in the first frame, a first patch that corresponds to the first feature, accessing a first local memory of the first camera that stores reference patches identified in one or more previous frames captured by the first camera, and determining that none of the reference patches stored in the first local memory corresponds to the first feature. The method further includes receiving, from a second camera through a data link connecting the second camera with the first camera, a reference patch corresponding to the first feature. The reference patch is identified in a previous frame captured by the second camera and of the second camera. The method may then determine correspondence data between the first patch and the reference patch, and tracks the first feature in the environment based on the determined correspondence data.