Abstract: Systems and methods for training a generative neural radiance field model can include geometric regularization. Geometric regularization can involve the utilization of reference geometry data and/or an output of a surface prediction model. The geometry regularization can train the generative neural radiance field model to mitigate artifact generation by limiting a distribution considered for color value prediction and density value prediction to a range associated with a realistic geometry range.

Abstract: Example embodiments relate to estimating depth information based on iris size. A computing system may obtain an image depicting a person and determine a facial mesh for a face of the person based on features of the face. In some instances, the facial mesh includes a combination of facial landmarks and eye landmarks. As such, the computing system may estimate an iris pixel dimension of an eye based on the eye landmarks of the facial mesh and estimate a distance of the eye of the face relative to the camera based on the iris pixel dimension, a mean value iris dimension, and an intrinsic matrix of the camera. The computing system may further modify the image based on the estimated distance.

Abstract: The technology relates to methods and systems for implicit calibration for gaze tracking. This can include receiving, by a neural network module, display content that is associated with presentation on a display screen. The neural network module may also receive uncalibrated gaze information, in which the uncalibrated gaze information includes an uncalibrated gaze trajectory that is associated with a viewer gaze of the display content on the display screen. A selected function is applied by the neural network module to the uncalibrated gaze information and the display content to generate a user-specific gaze function. The user-specific gaze function has one or more personalized parameters. And the neural network module can then apply the user-specific gaze function to the uncalibrated gaze information to generate calibrated gaze information associated with the display content on the display screen. Training and testing information may alternatively be created for implicit gaze calibration.

Abstract: Example embodiments relate to estimating depth information based on iris size. A computing system may obtain an image depicting a person and determine a facial mesh for a face of the person based on features of the face. In some instances, the facial mesh includes a combination of facial landmarks and eye landmarks. As such, the computing system may estimate an iris pixel dimension of an eye based on the eye landmarks of the facial mesh and estimate a distance of the eye of the face relative to the camera based on the iris pixel dimension, a mean value iris dimension, and an intrinsic matrix of the camera. The computing system may further modify the image based on the estimated distance.

Abstract: The technology relates to methods and systems for implicit calibration for gaze tracking. This can include receiving, by a neural network module, display content that is associated with presentation on a display screen (1202). The neural network module may also receive uncalibrated gaze information, in which the uncalibrated gaze information includes an uncalibrated gaze trajectory that is associated with a viewer gaze of the display content on the display screen (1204). A selected function is applied by the neural network module to the uncalibrated gaze information and the display content to generate a user-specific gaze function (1206). The user-specific gaze function has one or more personalized parameters. And the neural network module can then apply the user-specific gaze function to the uncalibrated gaze information to generate calibrated gaze information associated with the display content on the display screen (1208).

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural radiance field (NeRF) model on unposed images. In particular, the training incorporates a geometric consistency loss to train the encoder neural network that predicts the poses of the unposed images.

Abstract: Provided are systems and methods for controlling material attributes such as roughness, metallic, albedo, and transparency in real images. This method leverages the generative prior of text-to-image models known for their photorealistic capabilities, offering an alternative to traditional rendering pipelines. As one example, the technology can be used to alter the appearance of an object in an image, making it appear more metallic or changing its roughness to create a more matte or glossy finish. This can be particularly useful in various fields where the ability to manipulate the appearance of products in images can be a powerful tool.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for characterizing a gaze position of a user in a query image. One of the methods includes obtaining a query image of a user captured by a camera of a mobile device; obtaining device characteristics data specifying (ii) characteristics of the mobile device, (ii) characteristics of the camera of the mobile device, or (iii) both; and processing a neural network input comprising (i) one or more images derived from the query image and (ii) the device characteristics data using a gaze prediction neural network, wherein the gaze prediction neural network is configured to, at run time and after the gaze prediction neural network has been trained, process the neural network input to generate a neural network output that characterizes a gaze position of the user in the query image.

Abstract: A method including determining a viewpoint, generating a first image using an image generator, the first image including an object in a first orientation based on the viewpoint, modifying the image generator based on a second orientation of the object, and generating a second image based on the first image using the modified image generator.

Abstract: Systems and methods for training a generative neural radiance field model can include geometric regularization. Geometric regularization can involve the utilization of reference geometry data and/or an output of a surface prediction model. The geometry regularization can train the generative neural radiance field model to mitigate artifact generation by limiting a distribution considered for color value prediction and density value prediction to a range associated with a realistic geometry range.

Abstract: Provided are systems and methods for creating 3D representations from one or more images of objects. It involves training a machine-learned correspondence network to convert 3D locations of pixels into a 2D canonical coordinate space. This network can map texture values from ground truth or synthetic images of the object into the 2D space, creating a texture data set. When a new synthetic image is generated from a specific pose, the 3D locations can be mapped into the 2D space, allowing texture values to be retrieved and applied to the new image. The system also enables users to edit the texture data, facilitating texture edits and transfers across objects.

Abstract: Systems and methods for learning spaces of three-dimensional shape and appearance from datasets of single-view images can be utilized for generating view renderings of a variety of different objects and/or scenes. The systems and methods can be able to learn effectively from unstructured. “in-the-wild” data, without incurring the high cost of a full-image discriminator, and while avoiding problems such as mode-dropping that are inherent to adversarial methods.

Abstract: A method includes determining a measured eye gaze position of an eye of a user. The method also includes determining a first incremental change in the measured eye gaze position by processing the measured eye gaze position by a long short-term memory (LSTM) model, and determining a first predicted eye gaze position of the eye at a first future time based on the measured eye gaze position and the first incremental change. The method additionally includes determining a second incremental change in the first predicted eye gaze position by processing the first predicted eye gaze position by the LSTM model, and determining a second predicted eye gaze position of the eye at a second future time subsequent to the first future time based on the first predicted eye gaze position and the second incremental change.

Abstract: A method includes determining a measured eye gaze position of an eye of a user. The method also includes determining a first incremental change in the measured eye gaze position by processing the measured eye gaze position by a long short-term memory (LSTM) model, and determining a first predicted eye gaze position of the eye at a first future time based on the measured eye gaze position and the first incremental change. The method additionally includes determining a second incremental change in the first predicted eye gaze position by processing the first predicted eye gaze position by the LSTM model, and determining a second predicted eye gaze position of the eye at a second future time subsequent to the first future time based on the first predicted eye gaze position and the second incremental change.

Abstract: The technology relates to methods and systems for implicit calibration for gaze tracking. This can include receiving, by a neural network module, display content that is associated with presentation on a display screen (1202). The neural network module may also receive uncalibrated gaze information, in which the uncalibrated gaze information includes an uncalibrated gaze trajectory that is associated with a viewer gaze of the display content on the display screen (1204). A selected function is applied by the neural network module to the uncalibrated gaze information and the display content to generate a user-specific gaze function (1206). The user-specific gaze function has one or more personalized parameters. And the neural network module can then apply the user-specific gaze function to the uncalibrated gaze information to generate calibrated gaze information associated with the display content on the display screen (1208).

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for characterizing a gaze position of a user in a query image. One of the methods includes obtaining a query image of a user captured by a camera of a mobile device; obtaining device characteristics data specifying (ii) characteristics of the mobile device, (ii) characteristics of the camera of the mobile device, or (iii) both; and processing a neural network input comprising (i) one or more images derived from the query image and (ii) the device characteristics data using a gaze prediction neural network, wherein the gaze prediction neural network is configured to, at run time and after the gaze prediction neural network has been trained, process the neural network input to generate a neural network output that characterizes a gaze position of the user in the query image.

Abstract: Disclosed are methods for diagnosing declarative memory loss using mouse tracking to follow the visual gaze of a subject taking a visual paired comparison test. Also disclosed are methods for diagnosing dementia such as mild cognitive impairment and Alzheimer's disease.

Abstract: Example embodiments relate to estimating depth information based on iris size. A computing system may obtain an image depicting a person and determine a facial mesh for a face of the person based on features of the face. In some instances, the facial mesh includes a combination of facial landmarks and eye landmarks. As such, the computing system may estimate an iris pixel dimension of an eye based on the eye landmarks of the facial mesh and estimate a distance of the eye of the face relative to the camera based on the iris pixel dimension, a mean value iris dimension, and an intrinsic matrix of the camera. The computing system may further modify the image based on the estimated distance.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for characterizing a gaze position of a user in a query image. One of the methods includes obtaining a query image of a user captured by a camera of a mobile device; obtaining device characteristics data specifying (ii) characteristics of the mobile device, (ii) characteristics of the camera of the mobile device, or (iii) both; and processing a neural network input comprising (i) one or more images derived from the query image and (ii) the device characteristics data using a gaze prediction neural network, wherein the gaze prediction neural network is configured to, at run time and after the gaze prediction neural network has been trained, process the neural network input to generate a neural network output that characterizes a gaze position of the user in the query image.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for characterizing a gaze position of a user in a query image. One of the methods includes obtaining a query image of a user captured by a camera of a mobile device; obtaining device characteristics data specifying (ii) characteristics of the mobile device, (ii) characteristics of the camera of the mobile device, or (iii) both; and processing a neural network input comprising (i) one or more images derived from the query image and (ii) the device characteristics data using a gaze prediction neural network, wherein the gaze prediction neural network is configured to, at run time and after the gaze prediction neural network has been trained, process the neural network input to generate a neural network output that characterizes a gaze position of the user in the query image.