Abstract: Aspects of hair simulation, and networks therefor are provided including aspects to train such networks. There is provided a generative model for hair simulation that is guided during training by a hair classifier model. The generative model in an embodiment is provided for use in a virtual try-on (VTO) pipeline such as for virtually trying on hair color products. Further provided is a color mapping network to process an input image and target hair color for the generative model to define the hair simulation (e.g. as an output image with simulated hair color).

Abstract: Aspects of lighting estimation, and models therefor are provided including aspects to train such models. There is provided a lighting estimation model pre-trained using synthetic data to alleviate the costs and difficulty in obtaining real portrait image and HDR environment map paired datasets. To improve model performance, the model is training utilizing a discriminator configured to predict one or more average color values of a defined percentage of highest intensity pixels of a predicted environment map and to determine a color loss associated with the predicted environment map and the one or more average color values. The trained model can be used for a wide range of downstream tasks, including being used to generate hair renderings with realistic lighting effects for virtual try on experiences.

Abstract: Real-time makeup virtual try-on (VTO) on resource-constrained platforms like mobile devices and web browsers demands a delicate balance: models must be accurate enough for realistic results yet lightweight and fast enough for smooth performance. Existing approaches often rely on separate models for facial landmark detection and occlusion-aware segmentation, increasing complexity and hindering real-time performance. There is proposed, in accordance with embodiments, a unified model that performs both tasks within a single, highly efficient architecture. Specifically designed for VTO, the model offers enhanced accuracy around critical areas like the eyes and lips. Operations can be further optimized for real-time performance by leveraging temporal information: predictions from previous video frames guide current predictions, increasing parallelism and reducing inference time.

Abstract: Methods, apparatus and techniques herein relates to determining directions in GAN latent space and obtaining disentangled controls over GAN output semantics, for example, to enable use of such to generating synthesized images such as for use to train another model or create an augmented reality The methods, apparatus and techniques herein, in accordance with embodiments, utilize the gradient directions of auxiliary networks to control semantics in GAN latent codes. It is shown that minimal amounts of labelled data with sizes as small as 60 samples can be used, which data can be obtained quickly with human supervision. It is also shown herein, in accordance with embodiments, to select important latent code channels with masks during manipulation, resulting in more disentangled controls.

Abstract: Methods, apparatus, systems achieve disentanglement in semantic editing using GANs-based models. Self-corrected (low-density) latent code samples are projected in the original latent space and the editing directions corrected though relearning based on resulting high-density and low-density regions in the amended latent space. Leveraging the original meaningful directions and semantic region-specific layers, operations interpolate the original latent codes to generate images with minority combinations of attributes, then inverts these samples back to the original latent space. In accordance with embodiments, the operations can apply to preexisting methods that learn meaningful latent directions. Attribute disentanglement is improved with small amounts of low-density region samples added.

Abstract: Vision Transformers (ViT) have shown their competitive advantages performance-wise compared to convolutional neural networks (CNNs) though they often come with high computational costs. Methods, systems and techniques herein learn instance-dependent attention patterns, utilizing a lightweight connectivity predictor module to estimate a connectivity score of each pair of tokens. Intuitively, two tokens have high connectivity scores if the features are considered relevant either spatially or semantically. As each token only attends to a small number of other tokens, the binarized connectivity masks are often very sparse by nature providing an opportunity to accelerate the network via sparse computations. Equipped with the learned unstructured attention pattern, sparse attention ViT produces a superior Pareto-optimal trade-off between FLOPs and top-1 accuracy on ImageNet compared to token sparsity (48%˜69% FLOPs reduction of MHSA; accuracy drop within 0.4%).

Abstract: A computer system transmits user input and contextual information to a large language model (LLM) and requests the LLM to confirm the user input relates to one or more beauty topics. Based on the confirmation, the system requests the LLM to provide a response to be presented to a user via a user interface (UI), which relates to the beauty topic(s) and is based on the user input and contextual information. The confirmation may include requesting the LLM to provide one or more classifications of the user input. The UI may include elements such as a skin analysis request element, a product information element, or a content selection element. For example, the skin analysis request element may be activated to obtain a digital model of a face of the user, and a product or care routine recommendation can be generated based on the digital model.

Abstract: A computer system transmits user input and contextual information to a large language model (LLM) and requests the LLM to confirm the input text relates to one or more beauty topics. Based on the confirmation, the system requests the LLM to provide a response to the input text to be presented to a user. The response relates to the beauty topic(s) and is based on the input text and contextual information. The confirmation may include requesting the LLM to provide one or more classifications of the input text, which indicate that the input text relates to the beauty topic(s). The system may request the LLM to provide a summary of the input text, generate a vector representation, and identify matches for the vector representation of user input among other vector representations in the database (e.g., for relevant products or content).

Abstract: Systems, methods and techniques provide for acne localization, counting and visualization. An image is processed using a trained model to identify objects. The model may be a deep learning (e.g. convolutional neural) network configured for object classification with a detection focus on small objects. The image may be a frontal or profile facial image, processed end to end. The model identifies and localizes different types of acne. Instances are counted and visualized such as by annotating the source image. An example annotation is an overlay identifying a type and location of each instance. Counts by acne type assist with scoring. A product and/or service may be recommended in response to the identification of the acne (e.g. the type, localization, counting and/or a score).

Abstract: Aspects of lighting estimation, and models therefor are provided including aspects to train such models. There is provided a lighting estimation model pre-trained using synthetic data to alleviate the costs and difficulty in obtaining real portrait image and HDR environment map paired datasets. To improve model performance, the model is training utilizing a discriminator configured to predict one or more average color values of a defined percentage of highest intensity pixels of a predicted environment map and to determine a color loss associated with the predicted environment map and the one or more average color values. The trained model can be used for a wide range of downstream tasks, including being used to generate hair renderings with realistic lighting effects for virtual try on experiences.

Abstract: Aspects of hair simulation, and networks therefor are provided including aspects to train such networks. There is provided a generative model for hair simulation that is guided during training by a hair classifier model. The generative model in an embodiment is provided for use in a virtual try-on (VTO) pipeline such as for virtually trying on hair color products. Further provided is a color mapping network to process an input image and target hair color for the generative model to define the hair simulation (e.g. as an output image with simulated hair color).

Abstract: Aspects of hair classification, and networks therefor are provided including aspects to train such networks. There is provided a classifier model to alleviate the impact of human bias where the modeling of the real label distribution and annotators' biases are separated by incorporating annotator confusion matrices into a baseline model. To further improve the model performance leveraging unlabeled data, the model was trained using a consistency-based semi-supervised learning framework. With the use of only 1000 labeled data, the final classifier model achieved a classification accuracy that was 20% higher than a human professional annotator. The trained model can be used for a wide range of downstream tasks, including being used as a color classifier to train generative models for hair color translation.

Abstract: Methods, apparatus, systems achieve disentanglement in semantic editing using GANs-based models. Self-corrected (low-density) latent code samples are projected in the original latent space and the editing directions corrected though relearning based on resulting high-density and low-density regions in the amended latent space. Leveraging the original meaningful directions and semantic region-specific layers, operations interpolate the original latent codes to generate images with minority combinations of attributes, then inverts these samples back to the original latent space. In accordance with embodiments, the operations can apply to preexisting methods that learn meaningful latent directions. Attribute disentanglement is improved with small amounts of low-density region samples added.

Abstract: There is provided methods, devices and techniques to process an image using a deep learning model to achieve continuous effect simulation by a unified network where a simple (effect class) estimator is embedded into a regular encoder-decoder architecture. The estimator allows learning of model-estimated class embeddings of all effect classes (e.g. progressive degrees of the effect), thus representing the continuous effect information without manual efforts in selecting proper anchor effect groups. In an embodiment, given a target age class, there is derived a personalized age embedding which considers two aspects of face aging: 1) a personalized residual age embedding at a model-estimated age of the subject, preserving the subject's aging information; and 2) exemplar-face aging basis at the target age, encoding the shared aging patterns among the entire population.

Abstract: Vision Transformers (ViT) have shown their competitive advantages performance-wise compared to convolutional neural networks (CNNs) though they often come with high computational costs. Methods, systems and techniques herein learn instance-dependent attention patterns, utilizing a lightweight connectivity predictor module to estimate a connectivity score of each pair of tokens. Intuitively, two tokens have high connectivity scores if the features are considered relevant either spatially or semantically. As each token only attends to a small number of other tokens, the binarized connectivity masks are often very sparse by nature providing an opportunity to accelerate the network via sparse computations. Equipped with the learned unstructured attention pattern, sparse attention ViT produces a superior Pareto-optimal trade-off between FLOPs and top-1 accuracy on ImageNet compared to token sparsity (48%˜69% FLOPs reduction of MHSA; accuracy drop within 0.4%).

Abstract: This application provides an inter-account interaction method performed by a computer device. The method includes: displaying a battle preparation interface of a virtual battle, the battle preparation interface comprising a first icon representing a first account associated with the computer device and at least one second icon representing an account other than the first account; in response to a trigger operation on one of the at least one second icon as a target icon, displaying a prop selection area on the battle preparation interface and at least one interactive prop owned by the first account in the prop selection area; and in response to a selection of one of the at least one interactive prop as a target interactive prop, playing a target special effect animation of transmitting a special effect resource of the target interactive prop pointing from the first icon to the target icon.

Abstract: GANs based generators are useful to perform image to image translations. GANs models have large storage sizes and resource use requirements such that they are too large to be deployed directly on mobile devices. Systems and methods define through conditioning a student GANs model having a student generator that is scaled downwardly from a teacher GANs model (and generator) using knowledge distillation. A semantic relation knowledge distillation loss is used to transfer semantic knowledge from an intermediate layer of the teacher to an intermediate layer of the student. Student generators thus defined are stored and executed by mobile devices such as smartphones and laptops to provide augmented reality experiences. Effects are simulated on images, including makeup, hair, nail and age simulation effects.

Abstract: According to one aspect, what is proposed is a method for generating a photorealistic rendering of a cosmetic product, comprising: —obtaining (10, 12) a reference image (Xref) of a real cosmetic product (PC) applied to a first person (P1) and at least one source image (Xjsource) of a second person (P2), —implementing (13) an encoding artificial neural network (E) configured to determine characterizing parameters (E(Xref)) of the cosmetic product (PC) from the reference image (Xref), and then —implementing (14) a realistic physically based rendering engine (R) configured to generate a transformed image (R (Xjsource, E(Xref))) in which a photorealistic rendering of the cosmetic product (PC) is applied to the person (P2) from said at least one source image (Xjsource) based on the characterizing parameters (E (Xref)) of the cosmetic product (PC) that are determined by the encoding artificial neural network (E).

Abstract: There is described a deep learning supervised regression based model including methods and systems for facial attribute prediction and use thereof. An example of use is an augmented and/or virtual reality interface to provide a modified image responsive to facial attribute predictions determined from the image. Facial effects matching facial attributes are selected to be applied in the interface.

Abstract: A method for generating a special effect for social networking interaction in a virtual environment of a game is performed by an electronic device. The method includes: displaying an object presentation interface of a target battle of the game when loading a virtual scene corresponding to the target battle, the object presentation interface being used for displaying a plurality of virtual objects participating in the target battle; receiving a special effect generating instruction for a first virtual object of the plurality of virtual objects, the special effect generating instruction being used for instructing to generate a special effect based on the first virtual object, and the first virtual object corresponding to a user of the electronic device triggering the special effect; and generating the special effect identifying the first virtual object in the object presentation interface.