Abstract: Various aspects of the subject technology relate to prediction of eye movements of a user of a head-mountable display device. Predictive foveated display systems and methods, using the predicted eye movements are also disclosed. Predictive variable focus display systems and methods using the predicted eye movements are also disclosed. Predicting eye movements may include predicting a future gaze location and/or predicting a future vergence plane for the user's eyes, based on the current motion of one or both of the user's eyes. The predicted gaze location may be used to pre-render a foveated display image frame with a high-resolution region at the predicted gaze location. The predicted vergence plane may be used to modify an image plane of a display assembly to mitigate or avoid a vergence/accommodation conflict for the user.

Abstract: An image generator is configured to generate display light. A first waveguide is configured to generate wide-field image light from a first portion of the display light. A first outcoupling element of the first waveguide extends to a boundary of the frame to provide the wide-field display to substantially all of the augmented FOV of the user. A second waveguide is configured to generate inset image light from a second portion of the display light received from the image generator.

Abstract: Various aspects of the subject technology relate to prediction of eye movements of a user of a head-mountable display device. Predictive foveated display systems and methods, using the predicted eye movements are also disclosed. Predictive variable focus display systems and methods using the predicted eye movements are also disclosed. Predicting eye movements may include predicting a future gaze location and/or predicting a future vergence plane for the user's eyes, based on the current motion of one or both of the user's eyes. The predicted gaze location may be used to pre-render a foveated display image frame with a high-resolution region at the predicted gaze location. The predicted vergence plane may be used to modify an image plane of a display assembly to mitigate or avoid a vergence/accommodation conflict for the user.

Abstract: A method for evaluating an external machine learning program while limiting access to internal training data includes providing labeled training data from a first source, receiving, by the first source, a machine learning program from a second source different from the first source, blocking, by the first source, access by the second source to the labeled training data, and training, by the first source, the machine learning program according to a supervised machine learning process using the labeled training data. The method further includes generating a first set of metrics from the supervised machine learning process that provide feedback about training of the neural network model, analyzing the first set of metrics to identify subset data therein, and, in order to permit evaluation of the neural network model, transmitting, to the second source, those metrics from the first set of metrics that do not include the subset data.

Abstract: Various aspects of the subject technology relate to prediction of eye movements of a user of a head-mountable display device. Predictive foveated display systems and methods, using the predicted eye movements are also disclosed. Predictive variable focus display systems and methods using the predicted eye movements are also disclosed. Predicting eye movements may include predicting a future gaze location and/or predicting a future vergence plane for the user's eyes, based on the current motion of one or both of the user's eyes. The predicted gaze location may be used to pre-render a foveated display image frame with a high-resolution region at the predicted gaze location. The predicted vergence plane may be used to modify an image plane of a display assembly to mitigate or avoid a vergence/accommodation conflict for the user.

Abstract: Systems and methods for performing depth estimation may comprise: an illuminator capable of illuminating a scene from at least a first position and a second position, an image sensor to capture (i) a first image of the scene while the illuminator illuminates the scene from the first position and (ii) a second image of the scene while the illuminator illuminates the scene from the second position, and an image processor to receive the first and second images from the image sensor and estimate a depth of at least one feature that appears in the first and second images. The depth is estimated based on the relative intensity of the first image and the second image, a distance between the first illumination position and the second illumination position, and a position of the at least one feature within at least one of the first and second images.

Abstract: Depth estimation may be performed by a movable illumination unit, a movable image sensing unit having a fixed position relative to the illumination unit, a memory, and one or more processors coupled to the memory. The processors read instructions from the memory to perform operations including receiving a reference image and a non-reference image from the image sensing unit and estimating a depth of a point of interest that appears in the reference and non-reference images. The reference image is captured when the image sensing unit and the illumination unit are located at a first position. The non-reference image is captured when the image sensing unit and the illumination unit are located at a second position. The first and second positions are separated by at least a translation along an optical axis of the image sensing unit. Estimating the depth of the point is based on the translation.

Abstract: Systems and methods for performing depth estimation may comprise: an illuminator capable of illuminating a scene from at least a first position and a second position, an image sensor to capture (i) a first image of the scene while the illuminator illuminates the scene from the first position and (ii) a second image of the scene while the illuminator illuminates the scene from the second position, and an image processor to receive the first and second images from the image sensor and estimate a depth of at least one feature that appears in the first and second images. The depth is estimated based on the relative intensity of the first image and the second image, a distance between the first illumination position and the second illumination position, and a position of the at least one feature within at least one of the first and second images.