Abstract: Methods and systems of frame based image segmentation are provided. For example, a method for feature object tracking between frames of video data is provided. The method comprises receiving a first frame of video data, extracting a mask feature for each of one or more objects of the first frame, adjusting the first frame by applying each initial mask and corresponding identification to a respective object of the first frame, and outputting the adjusted first frame. The method further comprises tracking the one or more objects in one or more consecutive frames. The tracking comprises extracting a masked feature for each of one or more objects in the consecutive frame, adjusting the consecutive frame by applying each initial mask and corresponding identification for the consecutive frame to the respective object of the one or more objects of the consecutive frame, and outputting the adjusted consecutive frame.

Abstract: Techniques and systems are provided for authenticating a user of a device. For example, input biometric data associated with a person can be obtained. A similarity score for the input biometric data can be determined by comparing the input biometric data to a set of templates that include reference biometric data associated with the user. The similarity score can be compared to an authentication threshold. The person is authenticated as the user when the similarity score is greater than the authentication threshold. The similarity score can also be compared to a learning threshold that is greater than the authentication threshold. A new template including features of the input biometric data is saved for the user when the similarity score is less than the learning threshold and greater than the authentication threshold.

Abstract: Systems and techniques are provided for performing scene segmentation and object tracking. For example, a method for processing one or more frames. The method may include determining first one or more features from a first frame. The first frame includes a target object. The method may include obtaining a first mask associated with the first frame. The first mask includes an indication of the target object. The method may further include generating, based on the first mask and the first one or more features, a representation of a foreground and a background of the first frame. The method may include determining second one or more features from a second frame and determining, based on the representation of the foreground and the background of the first frame and the second one or more features, a location of the target object in the second frame.

Abstract: Examples are described of segmenting an image into image regions based on depicted categories of objects, and for refining the image regions semantically. For example, a system can determine that a first image region in an image depicts a first category of object. The system can generate a color distance map of the first image region that maps color distance values to each pixel in the first image region. A color distance value quantifies a difference between a color value of a pixel in the first image region and a color value of a sample pixel in a second image region in the image. The system can process the image based on a refined variant of the first image region that is refined based on the color distance map, for instance by removing pixels from the first image region whose color distances fall below a color distance threshold.

Abstract: Systems and techniques are provided for performing semantic image segmentation using a machine learning system (e.g., including one or more cross-attention transformer layers). For instance, a process can include generating one or more input image features for a frame of image data and generating one or more input depth features for a frame of depth data. One or more fused image features can be determined, at least in part, by fusing the one or more input depth features with the one or more input image features, using a first cross-attention transformer network. One or more segmentation masks can be generated for the frame of image data based on the one or more fused image features.

Abstract: Embodiments described herein can address these and other issues by using radar machine learning to address the radio frequency (RF) to perform object identification, including facial recognition. In particular, embodiments may obtain IQ samples by transmitting and receiving a plurality of data packets with a respective plurality of transmitter antenna elements and receiver antenna elements, where each data packet of the plurality of data packets comprises one or more complementary pairs of Golay sequences. I/Q samples indicative of a channel impulse responses of an identification region obtained from the transmission and reception of the plurality of data packets may then be used to identify, with a random forest model, a physical object in the identification region.

Abstract: Techniques and systems are provided for performing object verification using radar images. For example, a first radar image and a second radar image are obtained, and features are extracted from the first radar image and the second radar image. A similarity is determined between an object represented by the first radar image and an object represented by the second radar image based on the features extracted from the first radar image and the features extracted from the second radar image. A determined similarity between these two sets of features is used to determine whether the object represented by the first radar image matches the object represented by the second radar image. Distances between the features in the two radar images can optionally also be compared and used to determine object similarity. The objects in the radar images may optionally be faces.

Abstract: Embodiments described herein can address these and other issues by using radar machine learning to address the radio frequency (RF) to perform object identification, including facial recognition. In particular, embodiments may obtain IQ samples by transmitting and receiving a plurality of data packets with a respective plurality of transmitter antenna elements and receiver antenna elements. I/Q samples indicative of a channel impulse responses of an identification region obtained from the transmission and reception of the plurality of data packets may then be used to identify, with an autoencoder, a physical object in the identification region.

Abstract: Examples are described of segmenting an image into image regions based on depicted categories of objects, and for refining the image regions semantically. For example, a system can determine that a first image region in an image depicts a first category of object. The system can generate a color distance map of the first image region that maps color distance values to each pixel in the first image region. A color distance value quantifies a difference between a color value of a pixel in the first image region and a color value of a sample pixel in a second image region in the image. The system can process the image based on a refined variant of the first image region that is refined based on the color distance map, for instance by removing pixels from the first image region whose color distances fall below a color distance threshold.

Abstract: Techniques and systems are provided for authenticating a user of a device. For example, input biometric data associated with a person can be obtained. A similarity score for the input biometric data can be determined by comparing the input biometric data to a set of templates that include reference biometric data associated with the user. The similarity score can be compared to an authentication threshold. The person is authenticated as the user when the similarity score is greater than the authentication threshold. The similarity score can also be compared to a learning threshold that is greater than the authentication threshold. A new template including features of the input biometric data is saved for the user when the similarity score is less than the learning threshold and greater than the authentication threshold.

Abstract: Methods, systems, and devices for personalized (e.g., user specific) eye openness estimation are described. A network model (e.g., a convolutional neural network) may be trained using a set of synthetic eye openness image data (e.g., synthetic face images with known degrees or percentages of eye openness) and a set of real eye openness image data (e.g., facial images of real persons that are annotated as either open eyed or closed eyed). A device may estimate, using the network model, a multi-stage eye openness level (e.g., a percentage or degree to which an eye is open) of a user based on captured real time eye openness image data. The degree of eye openness estimated by the network model may then be compared to an eye size of the user (e.g., a user specific maximum eye size), and a user specific eye openness level may be estimated based on the comparison.

Abstract: Techniques and systems are provided for authenticating a user of a device. For example, input biometric data associated with a person can be obtained. A similarity score for the input biometric data can be determined by comparing the input biometric data to a set of templates that include reference biometric data associated with the user. The similarity score can be compared to an authentication threshold. The person is authenticated as the user when the similarity score is greater than the authentication threshold. The similarity score can also be compared to a learning threshold that is greater than the authentication threshold. A new template including features of the input biometric data is saved for the user when the similarity score is less than the learning threshold and greater than the authentication threshold.

Abstract: A method performed by an electronic device is described. The method includes receiving first optical data and first depth data corresponding to a first frame. The method also includes registering the first depth data to a first canonical model. The method further includes fitting a three-dimensional (3D) morphable model to the first optical data. The method additionally includes registering the 3D morphable model to a second canonical model. The method also includes producing a 3D object reconstruction based on the registered first depth data and the registered 3D morphable model.

Abstract: A method performed by an electronic device is described. The method includes receiving first optical data and first depth data corresponding to a first frame. The method also includes registering the first depth data to a first canonical model. The method further includes fitting a three-dimensional (3D) morphable model to the first optical data. The method additionally includes registering the 3D morphable model to a second canonical model. The method also includes producing a 3D object reconstruction based on the registered first depth data and the registered 3D morphable model.

Abstract: A method performed by an electronic device is described. The method includes receiving an image. The image depicts a face. The method also includes detecting at least one facial landmark of the face in the image. The method further includes receiving a depth image of the face and determining at least one landmark depth by mapping the at least one facial landmark to the depth image. The method also includes determining a plurality of scales of depth image pixels based on the at least one landmark depth and determining a scale smoothness measure for each of the plurality of scales of depth image pixels. The method additionally includes determining facial liveness based on at least two of the scale smoothness measures. Determining the facial liveness may be based on a depth-adaptive smoothness threshold and/or may be based on a natural face size criterion.

Abstract: Methods, systems, and devices for image processing are described. The method may include identifying a face in a first image based on identifying one or more biometric features of the face, determining an angular direction of one or more pixels of the identified face, identifying an anchor point on the identified face, sorting each of one or more pixels of the identified face into one of a set of pixel bins based on a combination of the determined angular direction of the pixel and a distance between the pixel and the identified anchor point, and outputting an indication of authenticity associated with the face based on a number of pixels in each bin.

Abstract: A method performed by an electronic device is described. The method includes receiving an image. The image depicts a face. The method also includes detecting at least one facial landmark of the face in the image. The method further includes receiving a depth image of the face and determining at least one landmark depth by mapping the at least one facial landmark to the depth image. The method also includes determining a plurality of scales of depth image pixels based on the at least one landmark depth and determining a scale smoothness measure for each of the plurality of scales of depth image pixels. The method additionally includes determining facial liveness based on at least two of the scale smoothness measures. Determining the facial liveness may be based on a depth-adaptive smoothness threshold and/or may be based on a natural face size criterion.

Abstract: Systems, methods, and computer-readable media are provided media for transferring visual attributes to images. In some examples, a system can obtain a first image associated with a user; generate a second image including image data from the first image modified to add a first visual attribute transferred from one or more images or remove a second visual attribute in the image data; compare a first set of features from the first image with a second set of features from the second image; determine, based on a comparison result, whether the first image and the second image match at least partially; and update a library of user verification images to include the second image when the first image and the second image match at least partially.

Abstract: A method performed by an electronic device is described. The method includes receiving a set of frames. The set of frames describes a moving three-dimensional (3D) object. The method also includes registering the set of frames based on a canonical model. The canonical model includes geometric information and optical information. The method additionally includes fusing frame information of each frame to the canonical model based on the registration. The method further includes reconstructing the 3D object based on the canonical model.

Abstract: A method performed by an electronic device is described. The method includes obtaining a two-dimensional (2D) depth image. The method also includes extracting a 2D subset of the depth image. The 2D subset includes a center pixel and a set of neighboring pixels. The method further includes calculating a normal corresponding to the center pixel by calculating a covariance matrix based on the 2D subset.