Abstract: Systems and methods process images to determine a skin condition severity analysis and to visualize a skin analysis such as using a deep neural network (e.g. a convolutional neural network) where a problem was formulated as a regression task with integer-only labels. Auxiliary classification tasks (for example, comprising gender and ethnicity predictions) are introduced to improve performance. Scoring and other image processing techniques may be used (e.g. in assoc. with the model) to visualize results such as highlighting the analyzed image. It is demonstrated that the visualization of results, which highlight skin condition affected areas, can also provide perspicuous explanations for the model. A plurality (k) of data augmentations may be made to a source image to yield k augmented images for processing. Activation masks (e.g. heatmaps) produced from processing the k augmented images are used to define a final map to visualize the skin analysis.

Abstract: Methods, apparatus and techniques herein relates to determining directions in GAN latent space and obtaining disentangled controls over GAN output semantics, for example, to enable use of such to generating synthesized images such as for use to train another model or create an augmented reality The methods, apparatus and techniques herein, in accordance with embodiments, utilize the gradient directions of auxiliary networks to control semantics in GAN latent codes. It is shown that minimal amounts of labelled data with sizes as small as 60 samples can be used, which data can be obtained quickly with human supervision. It is also shown herein, in accordance with embodiments, to select important latent code channels with masks during manipulation, resulting in more disentangled controls.

Abstract: A system and method implement deep learning on a mobile device to provide a convolutional neural network (CNN) for real time processing of video, for example, to color hair. Images are processed using the CNN to define a respective hair matte of hair pixels. The respective object mattes may be used to determine which pixels to adjust when adjusting pixel values such as to change color, lighting, texture, etc. The CNN may comprise a (pre-trained) network for image classification adapted to produce the segmentation mask. The CNN may be trained for image segmentation (e.g. using coarse segmentation data) to minimize a mask-image gradient consistency loss. The CNN may further use skip connections between corresponding layers of an encoder stage and a decoder stage where shallower layers in the encoder, which contain high-res but weak features are combined with low resolution but powerful features from deeper decoder layers.

Abstract: There is shown and described a deep learning based system and method for skin diagnostics as well as testing metrics that show that such a deep learning based system outperforms human experts on the task of apparent skin diagnostics. Also shown and described is a system and method of monitoring a skin treatment regime using a deep learning based system and method for skin diagnostics.

Abstract: There is shown and described a deep learning based system and method for skin diagnostics as well as testing metrics that show that such a deep learning based system outperforms human experts on the task of apparent skin diagnostics. Also shown and described is a system and method of monitoring a skin treatment regime using a deep learning based system and method for skin diagnostics.

Abstract: There is shown and described a deep learning based system and method for skin diagnostics as well as testing metrics that show that such a deep learning based system outperforms human experts on the task of apparent skin diagnostics. Also shown and described is a system and method of monitoring a skin treatment regime using a deep learning based system and method for skin diagnostics.

Abstract: A method for generating a special effect for social networking interaction in a virtual environment of a game is performed by an electronic device. The method includes: displaying an object presentation interface of a target battle of the game when loading a virtual scene corresponding to the target battle, the object presentation interface being used for displaying a plurality of virtual objects participating in the target battle; receiving a special effect generating instruction for a first virtual object of the plurality of virtual objects, the special effect generating instruction being used for instructing to generate a special effect based on the first virtual object, and the first virtual object corresponding to a user of the electronic device triggering the special effect; and generating the special effect identifying the first virtual object in the object presentation interface.

Abstract: There is provided methods, devices and techniques to process an image using a deep learning model to achieve continuous effect simulation by a unified network where a simple (effect class) estimator is embedded into a regular encoder-decoder architecture. The estimator allows learning of model-estimated class embeddings of all effect classes (e.g. progressive degrees of the effect), thus representing the continuous effect information without manual efforts in selecting proper anchor effect groups. In an embodiment, given a target age class, there is derived a personalized age embedding which considers two aspects of face aging: 1) a personalized residual age embedding at a model-estimated age of the subject, preserving the subject's aging information; and 2) exemplar-face aging basis at the target age, encoding the shared aging patterns among the entire population.

Abstract: A system and method implement deep learning on a mobile device to provide a convolutional neural network (CNN) for real time processing of video, for example, to color hair. Images are processed using the CNN to define a respective hair matte of hair pixels. The respective object mattes may be used to determine which pixels to adjust when adjusting pixel values such as to change color, lighting, texture, etc. The CNN may comprise a (pre-trained) network for image classification adapted to produce the segmentation mask. The CNN may be trained for image segmentation (e.g. using coarse segmentation data) to minimize a mask-image gradient consistency loss. The CNN may further use skip connections between corresponding layers of an encoder stage and a decoder stage where shallower layers in the encoder, which contain high-res but weak features are combined with low resolution but powerful features from deeper decoder layers.

Abstract: Systems, methods and techniques provide for acne localization, counting and visualization. An image is processed using a trained model to identify objects. The model may be a deep learning (e.g. convolutional neural) network configured for object classification with a detection focus on small objects. The image may be a frontal or profile facial image, processed end to end. The model identifies and localizes different types of acne. Instances are counted and visualized such as by annotating the source image. An example annotation is an overlay identifying a type and location of each instance. Counts by acne type assist with scoring. A product and/or service may be recommended in response to the identification of the acne (e.g. the type, localization, counting and/or a score).

Abstract: GANs based generators are useful to perform image to image translations. GANs models have large storage sizes and resource use requirements such that they are too large to be deployed directly on mobile devices. Systems and methods define through conditioning a student GANs model having a student generator that is scaled downwardly from a teacher GANs model (and generator) using knowledge distillation. A semantic relation knowledge distillation loss is used to transfer semantic knowledge from an intermediate layer of the teacher to an intermediate layer of the student. Student generators thus defined are stored and executed by mobile devices such as smartphones and laptops to provide augmented reality experiences. Effects are simulated on images, including makeup, hair, nail and age simulation effects.

Abstract: A system and method implement deep learning on a mobile device to provide a convolutional neural network (CNN) for real time processing of video, for example, to color hair. Images are processed using the CNN to define a respective hair matte of hair pixels. The respective object mattes may be used to determine which pixels to adjust when adjusting pixel values such as to change color, lighting, texture, etc. The CNN may comprise a (pre-trained) network for image classification adapted to produce the segmentation mask. The CNN may be trained for image segmentation (e.g. using coarse segmentation data) to minimize a mask-image gradient consistency loss. The CNN may further use skip connections between corresponding layers of an encoder stage and a decoder stage where shallower layers in the encoder, which contain high-res but weak features are combined with low resolution but powerful features from deeper decoder layers.

Abstract: There is described a deep learning supervised regression based model including methods and systems for facial attribute prediction and use thereof. An example of use is an augmented and/or virtual reality interface to provide a modified image responsive to facial attribute predictions determined from the image.

Abstract: Systems and methods process images to determine a skin condition severity analysis and to visualize a skin analysis such as using a deep neural network (e.g. a convolutional neural network) where a problem was formulated as a regression task with integer-only labels. Auxiliary classification tasks (for example, comprising gender and ethnicity predictions) are introduced to improve performance. Scoring and other image processing techniques may be used (e.g. in assoc. with the model) to visualize results such as highlighting the analyzed image. It is demonstrated that the visualization of results, which highlight skin condition affected areas, can also provide perspicuous explanations for the model. A plurality (k) of data augmentations may be made to a source image to yield k augmented images for processing. Activation masks (e.g. heatmaps) produced from processing the k augmented images are used to define a final map to visualize the skin analysis.

Abstract: A system and method implement deep learning on a mobile device to provide a convolutional neural network (CNN) for real time processing of video, for example, to color hair. Images are processed using the CNN to define a respective hair matte of hair pixels. The respective object mattes may be used to determine which pixels to adjust when adjusting pixel values such as to change color, lighting, texture, etc. The CNN may comprise a (pre-trained) network for image classification adapted to produce the segmentation mask. The CNN may be trained for image segmentation (e.g. using coarse segmentation data) to minimize a mask-image gradient consistency loss. The CNN may further use skip connections between corresponding layers of an encoder stage and a decoder stage where shallower layers in the encoder, which contain high-res but weak features are combined with low resolution but powerful features from deeper decoder layers.

Abstract: There is shown and described a deep learning based system and method for skin diagnostics as well as testing metrics that show that such a deep learning based system outperforms human experts on the task of apparent skin diagnostics. Also shown and described is a system and method of monitoring a skin treatment regime using a deep learning based system and method for skin diagnostics.

Abstract: The four-dimensional microscope includes a sample plate, a laser device, an aperture, an extraction plate, a hexapole, a quadrupole, a time-of-flight mass analyzer, a detector, and a device for supplying a voltage to the sample plate, the aperture, the extraction plate and the hexapole and the quadrupole. By utilizing the tunneling effect of photo-induced electrons on surfaces of semiconductor materials under laser irradiation and the electron capture ionization, mass-to-charge ratios and signal intensities of the ions resulting from the capture of interfacially transferred photo-induced electrons and subsequent photo-chemical reactions are measured, and image reconstruction is performed to obtain microscopic images. By using the present invention, not only active photo-catalytic sites of the semiconductor materials are imaged but also various structures of intermediates and products of photo-chemical reactions can be determined.