Patents by Inventor Aleksei Stoliar
Aleksei Stoliar has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11900565Abstract: A data item is identified on a device. A neural network that includes an adversarial transformation subnetwork is applied to the data item to generate a modified data item. Output indicative of the modified data item is caused to be presented on the device. The neural network further comprises an encoder and a decoder. The neural network is trained in at least two stages. At least one of the encoder and the decoder is trained in a first stage and the adversarial transformation subnetwork is trained in a second stage.Type: GrantFiled: March 2, 2023Date of Patent: February 13, 2024Assignee: Snap Inc.Inventors: Sergey Demyanov, Aleksei Podkin, Aleksei Stoliar, Vadim Velicodnii, Fedor Zhdanov
-
Publication number: 20240005617Abstract: Methods and systems are disclosed for performing operations for generating a photorealistic rendering of an object. The operations include: accessing a set of albedo textures and a machine learning model associated with a real-world object, the set of albedo textures and a machine learning model having been trained based on a plurality of viewpoints of the real-world object; obtaining a three-dimensional (3D) mesh of the real-world object; receiving input that selects a new viewpoint that differs from the plurality of viewpoints of the real-world object; and generating a photorealistic rendering of the real-world object from the new viewpoint based on the 3D mesh of the real-world object, the set of albedo textures, and the machine learning model associated with the real-world object.Type: ApplicationFiled: June 30, 2022Publication date: January 4, 2024Inventors: Vladislav Shakhrai, Sergey Demyanov, Mikhail Vasilkovskii, Aleksei Stoliar
-
Publication number: 20230379491Abstract: Systems and methods herein describe a video compression system. The described systems and methods accesses a sequence of image frames from a first computing device, the sequence of image frames comprising a first image frame and a second image frame, detects a first set of keypoints for the first image frame, transmits the first image frame and the first set of keypoints to a second computing device, detects a second set of keypoints for the second image frame, transmits the second set of keypoints to the second computing device, causes an animated image to be displayed on the second computing device.Type: ApplicationFiled: August 4, 2023Publication date: November 23, 2023Inventors: Sergey Demyanov, Andrew Cheng-min Lin, Walton Lin, Aleksei Podkin, Aleksei Stoliar, Sergey Tulyakov
-
Publication number: 20230334327Abstract: A compact generative neural network can be distilled from a teacher generative neural network using a training network. The compact network can be trained on the input data and output data of the teacher network. The training network train the student network using a discrimination layer and one or more types of losses, such as perception loss and adversarial loss.Type: ApplicationFiled: June 22, 2023Publication date: October 19, 2023Inventors: Sergey Tulyakov, Sergei Korolev, Aleksei Stoliar, Maksim Gusarov, Sergei Kotcur, Christopher Yale Crutchfield, Andrew Wan
-
Patent number: 11736717Abstract: Systems and methods herein describe a video compression system. The described systems and methods accesses a sequence of image frames from a first computing device, the sequence of image frames comprising a first image frame and a second image frame, detects a first set of keypoints for the first image frame, transmits the first image frame and the first set of keypoints to a second computing device, detects a second set of keypoints for the second image frame, transmits the second set of keypoints to the second computing device, causes an animated image to be displayed on the second computing device.Type: GrantFiled: September 30, 2021Date of Patent: August 22, 2023Assignee: Snap Inc.Inventors: Sergey Demyanov, Andrew Cheng-min Lin, Walton Lin, Aleksei Podkin, Aleksei Stoliar, Sergey Tulyakov
-
Patent number: 11727280Abstract: A compact generative neural network can be distilled from a teacher generative neural network using a training network. The compact network can be trained on the input data and output data of the teacher network. The training network train the student network using a discrimination layer and one or more types of losses, such as perception loss and adversarial loss.Type: GrantFiled: March 2, 2021Date of Patent: August 15, 2023Assignee: Snap Inc.Inventors: Sergey Tulyakov, Sergei Korolev, Aleksei Stoliar, Maksim Gusarov, Sergei Kotcur, Christopher Yale Crutchfield, Andrew Wan
-
Publication number: 20230252704Abstract: Systems and methods are disclosed for generating, a source image sequence using an image sensor of the computing device, the source image sequence comprising a plurality of source images depicting a head and face, identifying driving image sequence data to modify face image feature data in the source image sequence, generating, using an image transformation neural network, a modified source image sequence comprising a plurality of modified source images depicting modified versions of the head and face, and storing the modified source image sequence on the computing device.Type: ApplicationFiled: April 19, 2023Publication date: August 10, 2023Inventors: Sergey Demyanov, Aleksei Podkin, Aliaksandr Siarohin, Aleksei Stoliar, Sergey Tulyakov
-
Publication number: 20230206398Abstract: A data item is identified on a device. A neural network that includes an adversarial transformation subnetwork is applied to the data item to generate a modified data item. Output indicative of the modified data item is caused to be presented on the device. The neural network further comprises an encoder and a decoder. The neural network is trained in at least two stages. At least one of the encoder and the decoder is trained in a first stage and the adversarial transformation subnetwork is trained in a second stage.Type: ApplicationFiled: March 2, 2023Publication date: June 29, 2023Inventors: Sergey Demyanov, Aleksei Podkin, Aleksei Stoliar, Vadim Velicodnii, Fedor Zhdanov
-
Patent number: 11657479Abstract: A mobile device can implement a neural network-based domain transfer scheme to modify an image in a first domain appearance to a second domain appearance. The domain transfer scheme can be configured to detect an object in the image, apply an effect to the image, and blend the image using color space adjustments and blending schemes to generate a realistic result image. The domain transfer scheme can further be configured to efficiently execute on the constrained device by removing operational layers based on resources available on the mobile device.Type: GrantFiled: August 18, 2021Date of Patent: May 23, 2023Assignee: Snap Inc.Inventors: Sergey Demyanov, Aleksei Podkin, Aleksei Stoliar, Vadim Velicodnii, Fedor Zhdanov
-
Publication number: 20230146865Abstract: A neural network pruning system can sparsely prune neural network models using an optimizer based approach that is agnostic to the model architecture being pruned. The neural network pruning system can prune by operating on the parameter vector of the full model and the gradient vector of the loss function with respect to the model parameters. The neural network pruning system can iteratively update parameters based on the gradients, while zeroing out as many parameters as possible based a preconfigured penalty.Type: ApplicationFiled: January 12, 2023Publication date: May 11, 2023Inventors: Enxu Yan, Sergey Tulyakov, Aleksei Podkin, Aleksei Stoliar
-
Patent number: 11645798Abstract: Systems and methods are disclosed for generating, a source image sequence using an image sensor of the computing device, the source image sequence comprising a plurality of source images depicting a head and face, identifying driving image sequence data to modify face image feature data in the source image sequence, generating, using an image transformation neural network, a modified source image sequence comprising a plurality of modified source images depicting modified versions of the head and face, and storing the modified source image sequence on the computing device.Type: GrantFiled: June 1, 2021Date of Patent: May 9, 2023Assignee: Snap Inc.Inventors: Sergey Demyanov, Aleksei Podkin, Aliaksandr Siarohin, Aleksei Stoliar, Sergey Tulyakov
-
Publication number: 20230120964Abstract: Systems and methods herein describe receiving an image via an image capture device, using a machine learning model, generating an image augmentation decision, accessing an augmented reality content item, associating the generated image augmentation decision with the augmented reality content item, modifying the received image using the augmented reality content item and the associated image augmentation decision, and causing presentation of the modified image on a graphical user interface of a computing device.Type: ApplicationFiled: October 26, 2022Publication date: April 20, 2023Inventors: Olha Rykhliuk, Jonathan ` Solichin, Aleksei Stoliar
-
Patent number: 11580400Abstract: A neural network pruning system can sparsely prune neural network models using an optimizer based approach that is agnostic to the model architecture being pruned. The neural network pruning system can prune by operating on the parameter vector of the full model and the gradient vector of the loss function with respect to the model parameters. The neural network pruning system can iteratively update parameters based on the gradients, while zeroing out as many parameters as possible based a preconfigured penalty.Type: GrantFiled: September 27, 2019Date of Patent: February 14, 2023Assignee: Snap Inc.Inventors: Enxu Yan, Sergey Tulyakov, Aleksei Podkin, Aleksei Stoliar
-
Patent number: 11558325Abstract: Systems and methods are provided for receiving a first media content item associated with a first interactive object of an interactive message, receiving a second media content item associated with a second interactive object of the interactive message, generating a third media content item based on the first media content item and second media content item, wherein the third media content item comprises combined features of the first media content item and the second media content item, and causing display of the generated third media content item.Type: GrantFiled: May 26, 2021Date of Patent: January 17, 2023Assignee: Snap Inc.Inventors: Grygoriy Kozhemiak, Oleksandr Pyshchenko, Victor Shaburov, Trevor Stephenson, Aleksei Stoliar
-
Patent number: 11521339Abstract: Systems and methods herein describe receiving an image via an image capture device, using a machine learning model, generating an image augmentation decision, accessing an augmented reality content item, associating the generated image augmentation decision with the augmented reality content item, modifying the received image using the augmented reality content item and the associated image augmentation decision, and causing presentation of the modified image on a graphical user interface of a computing device.Type: GrantFiled: June 19, 2020Date of Patent: December 6, 2022Assignee: Snap Inc.Inventors: Olha Rykhliuk, Jonathan Solichin, Aleksei Stoliar
-
Publication number: 20220292866Abstract: A landmark detection system can more accurately detect landmarks in images using a detection scheme that penalizes for dispersion parameters, such as variance or scale. The landmark detection system can be trained using both labeled and unlabeled training data in a semi-supervised approach. The landmark detection system can further implement tracking of an object across multiple images using landmark data.Type: ApplicationFiled: June 1, 2022Publication date: September 15, 2022Inventors: Sergey Tulyakov, Roman Furko, Aleksei Stoliar
-
Publication number: 20220207355Abstract: Systems and methods herein describe an image manipulation system for generating modified images using a generative adversarial network. The image manipulation system accesses a pre-trained generative adversarial network (GAN), fine-tunes the pre-trained GAN by training a portion of existing neural network layers of the pre-trained GAN and newly added layers of the pre-trained GAN on a secondary image domain, adjusts the weights of the fine-tuned GAN using the weights of the pre-trained GAN, and stores the fine-tuned GAN. An image transformation system uses the generated modified images to train a subsequent neural network, which can access a face from a client device and transform it to a domain of images used for GAN fine-tuning.Type: ApplicationFiled: May 12, 2021Publication date: June 30, 2022Inventors: Sergey Demyanov, Konstantin Gudkov, Aleksei Stoliar, Roman Ushakov, Fedor Zhdanov
-
Patent number: 11354922Abstract: A landmark detection system can more accurately detect landmarks in images using a detection scheme that penalizes for dispersion parameters, such as variance or scale. The landmark detection system can be trained using both labeled and unlabeled training data in a semi-supervised approach. The landmark detection system can further implement tracking of an object across multiple images using landmark data.Type: GrantFiled: December 30, 2020Date of Patent: June 7, 2022Assignee: Snap Inc.Inventors: Sergey Tulyakov, Roman Furko, Aleksei Stoliar
-
Publication number: 20220103860Abstract: Systems and methods herein describe a video compression system. The described systems and methods acceses a sequence of image frames from a first computing device, the sequence of image frames comprising a first image frame and a second image frame, detects a first set of keypoints for the first image frame, transmits the first image frame and the first set of keypoints to a second computing device, detects a second set of keypoints for the second image frame, transmits the second set of keypoints to the second computing device, causes an animated image to be displayed on the second computing device.Type: ApplicationFiled: September 30, 2021Publication date: March 31, 2022Inventors: Sergey Demyanov, Andrew Cheng-min Lin, Walton Lin, Aleksei Podkin, Aleksei Stoliar, Sergey Tulyakov
-
Publication number: 20210390745Abstract: Systems and methods herein describe receiving an image via an image capture device, using a machine learning model, generating an image augmentation decision, accessing an augmentation reality content item, associating the generated image augmentation decision with the augmentation reality content item, modifying the received image using the augmentation reality content item and the associated image augmentation decision, and causing presentation of the modified image on a graphical user interface of a computing device.Type: ApplicationFiled: June 19, 2020Publication date: December 16, 2021Inventors: Olha Rykhliuk, Jonathan Solichin, Aleksei Stoliar