Patents by Inventor Ratheesh Kalarot

Ratheesh Kalarot has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20220391611
    Abstract: Systems and methods for image processing are described. One or more embodiments of the present disclosure identify a latent vector representing an image of a face, identify a target attribute vector representing a target attribute for the image, generate a modified latent vector using a mapping network that converts the latent vector and the target attribute vector into a hidden representation having fewer dimensions than the latent vector, wherein the modified latent vector is generated based on the hidden representation, and generate a modified image based on the modified latent vector, wherein the modified image represents the face with the target attribute.
    Type: Application
    Filed: June 8, 2021
    Publication date: December 8, 2022
    Inventors: RATHEESH KALAROT, Siavash Khodadadeh, Baldo Faieta, Shabnam Ghadar, Saeid Motiian, Wei-An Lin, Zhe Lin
  • Publication number: 20220270310
    Abstract: The present disclosure describes systems, methods, and non-transitory computer readable media for detecting user interactions to edit a digital image from a client device and modify the digital image for the client device by using a web-based intermediary that modifies a latent vector of the digital image and an image modification neural network to generate a modified digital image from the modified latent vector. In response to user interaction to modify a digital image, for instance, the disclosed systems modify a latent vector extracted from the digital image to reflect the requested modification. The disclosed systems further use a latent vector stream renderer (as an intermediary device) to generate an image delta that indicates a difference between the digital image and the modified digital image. The disclosed systems then provide the image delta as part of a digital stream to a client device to quickly render the modified digital image.
    Type: Application
    Filed: February 23, 2021
    Publication date: August 25, 2022
    Inventors: Akhilesh Kumar, Baldo Faieta, Piotr Walczyszyn, Ratheesh Kalarot, Archie Bagnall, Shabnam Ghadar, Wei-An Lin, Cameron Smith, Christian Cantrell, Patrick Hebron, Wilson Chan, Jingwan Lu, Holger Winnemoeller, Sven Olsen
  • Publication number: 20220237830
    Abstract: Embodiments are disclosed for automatic object re-colorization in images.
    Type: Application
    Filed: January 22, 2021
    Publication date: July 28, 2022
    Inventors: Siavash KHODADADEH, Zhe LIN, Shabnam GHADAR, Saeid MOTIIAN, Richard ZHANG, Ratheesh KALAROT, Baldo FAIETA
  • Publication number: 20220222776
    Abstract: An embodiment method includes performing first convolutional filtering on a first tensor constructed using a current frame and reference frames (or digital world reference images) of the current frame in a video, to generate a first estimated image of the current frame having a higher resolution than an image of the current frame. The method also includes performing second convolutional filtering on a second tensor constructed using the first estimated image and estimated reference images of the reference frames, to generate a second estimated image of the current having a higher resolution than the image of the current frame. The estimated reference images of the reference frames are reconstructed high resolution images of the reference images.
    Type: Application
    Filed: April 28, 2020
    Publication date: July 14, 2022
    Inventors: Fatih Murat Porikli, Ratheesh Kalarot
  • Publication number: 20220164380
    Abstract: A query image is received, along with a query to initiate a search process to find other images based on the query image. The query includes a preference value associated with an attribute, the preference value indicative of a level of emphasis to be placed on the attribute during the search. A full query vector, which is within a first dimensional space and representative of the query image, is generated. The full query vector is projected to a reduced dimensional space having a dimensionality lower than the first dimensional space, to generate a query vector. An attribute direction corresponding to the attribute is identified. A plurality of candidate vectors of the reduced dimensional space is searched, based on the attribute direction, the query vector, and the preference value, to identify a target vector of the plurality of candidate vectors. A target image, representative of the target vector, is displayed.
    Type: Application
    Filed: November 25, 2020
    Publication date: May 26, 2022
    Applicant: Adobe Inc.
    Inventors: Zhe Lin, Shabnam Ghadar, Saeid Motiian, Ratheesh Kalarot, Baldo Faieta, Alireza Zaeemzadeh
  • Publication number: 20220121932
    Abstract: Systems and methods train an encoder neural network for fast and accurate projection into the latent space of a Generative Adversarial Network (GAN). The encoder is trained by providing an input training image to the encoder and producing, by the encoder, a latent space representation of the input training image. The latent space representation is provided as input to the GAN to generate a generated training image. A latent code is sampled from a latent space associated with the GAN and the sampled latent code is provided as input to the GAN. The GAN generates a synthetic training image based on the sampled latent code. The sampled latent code is provided as input to the encoder to produce a synthetic training code. The encoder is updated by minimizing a loss between the generated training image and the input training image, and the synthetic training code and the sampled latent code.
    Type: Application
    Filed: July 23, 2021
    Publication date: April 21, 2022
    Inventors: Ratheesh Kalarot, Wei-An Lin, Cameron Smith, Zhixin Shu, Baldo Faieta, Shabnam Ghadar, Jingwan Lu, Aliakbar Darabi, Jun-Yan Zhu, Niloy Mitra, Richard Zhang, Elya Shechtman
  • Publication number: 20220122232
    Abstract: Systems and methods generate a filtering function for editing an image with reduced attribute correlation. An image editing system groups training data into bins according to a distribution of a target attribute. For each bin, the system samples a subset of the training data based on a pre-determined target distribution of a set of additional attributes in the training data. The system identifies a direction in the sampled training data corresponding to the distribution of the target attribute to generate a filtering vector for modifying the target attribute in an input image, obtains a latent space representation of an input image, applies the filtering vector to the latent space representation of the input image to generate a filtered latent space representation of the input image, and provides the filtered latent space representation as input to a neural network to generate an output image with a modification to the target attribute.
    Type: Application
    Filed: September 7, 2021
    Publication date: April 21, 2022
    Inventors: Wei-An Lin, Baldo Faieta, Cameron Smith, Elya Shechtman, Jingwan Lu, Jun-Yan Zhu, Niloy Mitra, Ratheesh Kalarot, Richard Zhang, Shabnam Ghadar, Zhixin Shu
  • Publication number: 20220122306
    Abstract: Systems and methods dynamically adjust an available range for editing an attribute in an image. An image editing system computes a metric for an attribute in an input image as a function of a latent space representation of the input image and a filtering vector for editing the input image. The image editing system compares the metric to a threshold. If the metric exceeds the threshold, then the image editing system selects a first range for editing the attribute in the input image. If the metric does not exceed the threshold, a second range is selected. The image editing system causes display of a user interface for editing the input image comprising an interface element for editing the attribute within the selected range.
    Type: Application
    Filed: September 7, 2021
    Publication date: April 21, 2022
    Inventors: Wei-An Lin, Baldo Faieta, Cameron Smith, Elya Shechtman, Jingwan Lu, Jun-Yan Zhu, Niloy Mitra, Ratheesh Kalarot, Richard Zhang, Shabnam Ghadar, Zhixin Shu
  • Publication number: 20220122307
    Abstract: Systems and methods combine an input image with an edited image generated using a generator neural network to preserve detail from the original image. A computing system provides an input image to a machine learning model to generate a latent space representation of the input image. The system provides the latent space representation to a generator neural network to generate a generated image. The system generates multiple scale representations of the input image, as well as multiple scale representations of the generated image. The system generates a first combined image based on first scale representations of the images and a first value. The system generates a second combined image based on second scale representations of the images and a second value. The system blends the first combined image with the second combined image to generate an output image.
    Type: Application
    Filed: September 7, 2021
    Publication date: April 21, 2022
    Inventors: Ratheesh Kalarot, Kevin Wampler, Jingwan Lu, Jakub Fiser, Elya Shechtman, Aliakbar Darabi, Alexandru Vasile Costin
  • Publication number: 20220121876
    Abstract: Systems and methods use a non-linear latent filter neural network for editing an image. An image editing system trains a first neural network by minimizing a loss based upon a predicted attribute value for a target attribute in a training image. The image editing system obtains a latent space representation of an input image to be edited and a target attribute value for the target attribute in the input image. The image editing system provides the latent space representation and the target attribute value as input to the trained first neural network for modifying the target attribute in the input image to generate a modified latent space representation of the input image. The image editing system provides the modified latent space representation as input to a second neural network to generate an output image with a modification to the target attribute corresponding to the target attribute value.
    Type: Application
    Filed: September 7, 2021
    Publication date: April 21, 2022
    Inventors: Ratheesh Kalarot, Wei-An Lin, Baldo Faieta, Shabnam Ghadar
  • Publication number: 20220122308
    Abstract: Systems and methods seamlessly blend edited and unedited regions of an image. A computing system crops an input image around a region to be edited. The system applies an affine transformation to rotate the cropped input image. The system provides the rotated cropped input image as input to a machine learning model to generate a latent space representation of the rotated cropped input image. The system edits the latent space representation and provides the edited latent space representation to a generator neural network to generate a generated edited image. The system applies an inverse affine transformation to rotate the generated edited image and aligns an identified segment of the rotated generated edited image with an identified corresponding segment of the input image to produce an aligned rotated generated edited image. The system blends the aligned rotated generated edited image with the input image to generate an edited output image.
    Type: Application
    Filed: September 7, 2021
    Publication date: April 21, 2022
    Inventors: Ratheesh Kalarot, Kevin Wampler, Jingwan Lu, Jakub Fiser, Elya Shechtman, Aliakbar Darabi, Alexandru Vasile Costin
  • Publication number: 20220121931
    Abstract: Systems and methods train and apply a specialized encoder neural network for fast and accurate projection into the latent space of a Generative Adversarial Network (GAN). The specialized encoder neural network includes an input layer, a feature extraction layer, and a bottleneck layer positioned after the feature extraction layer. The projection process includes providing an input image to the encoder and producing, by the encoder, a latent space representation of the input image. Producing the latent space representation includes extracting a feature vector from the feature extraction layer, providing the feature vector to the bottleneck layer as input, and producing the latent space representation as output. The latent space representation produced by the encoder is provided as input to the GAN, which generates an output image based upon the latent space representation. The encoder is trained using specialized loss functions including a segmentation loss and a mean latent loss.
    Type: Application
    Filed: July 23, 2021
    Publication date: April 21, 2022
    Inventors: Ratheesh Kalarot, Wei-An Lin, Cameron Smith, Zhixin Shu, Baldo Faieta, Shabnam Ghadar, Jingwan Lu, Aliakbar Darabi, Jun-Yan Zhu, Niloy Mitra, Richard Zhang, Elya Shechtman
  • Publication number: 20220122222
    Abstract: An improved system architecture uses a Generative Adversarial Network (GAN) including a specialized generator neural network to generate multiple resolution output images. The system produces a latent space representation of an input image. The system generates a first output image at a first resolution by providing the latent space representation of the input image as input to a generator neural network comprising an input layer, an output layer, and a plurality of intermediate layers and taking the first output image from an intermediate layer, of the plurality of intermediate layers of the generator neural network. The system generates a second output image at a second resolution different from the first resolution by providing the latent space representation of the input image as input to the generator neural network and taking the second output image from the output layer of the generator neural network.
    Type: Application
    Filed: July 23, 2021
    Publication date: April 21, 2022
    Inventors: Cameron Smith, Ratheesh Kalarot, Wei-An Lin, Richard Zhang, Niloy Mitra, Elya Shechtman, Shabnam Ghadar, Zhixin Shu, Yannick Hold-Geoffrey, Nathan Carr, Jingwan Lu, Oliver Wang, Jun-Yan Zhu
  • Publication number: 20220122305
    Abstract: An improved system architecture uses a pipeline including an encoder and a Generative Adversarial Network (GAN) including a generator neural network to generate edited images with improved speed, realism, and identity preservation. The encoder produces an initial latent space representation of an input image by encoding the input image. The generator neural network generates an initial output image by processing the initial latent space representation of the input image. The system generates an optimized latent space representation of the input image using a loss minimization technique that minimizes a loss between the input image and the initial output image. The loss is based on target perceptual features extracted from the input image and initial perceptual features extracted from the initial output image. The system outputs the optimized latent space representation of the input image for downstream use.
    Type: Application
    Filed: July 23, 2021
    Publication date: April 21, 2022
    Inventors: Cameron Smith, Ratheesh Kalarot, Wei-An Lin, Richard Zhang, Niloy Mitra, Elya Shechtman, Shabnam Ghadar, Zhixin Shu, Yannick Hold-Geoffrey, Nathan Carr, Jingwan Lu, Oliver Wang, Jun-Yan Zhu
  • Publication number: 20220122221
    Abstract: An improved system architecture uses a pipeline including a Generative Adversarial Network (GAN) including a generator neural network and a discriminator neural network to generate an image. An input image in a first domain and information about a target domain are obtained. The domains correspond to image styles. An initial latent space representation of the input image is produced by encoding the input image. An initial output image is generated by processing the initial latent space representation with the generator neural network. Using the discriminator neural network, a score is computed indicating whether the initial output image is in the target domain. A loss is computed based on the computed score. The loss is minimized to compute an updated latent space representation. The updated latent space representation is processed with the generator neural network to generate an output image in the target domain.
    Type: Application
    Filed: July 23, 2021
    Publication date: April 21, 2022
    Inventors: Cameron Smith, Ratheesh Kalarot, Wei-An Lin, Richard Zhang, Niloy Mitra, Elya Shechtman, Shabnam Ghadar, Zhixin Shu, Yannick Hold-Geoffrey, Nathan Carr, Jingwan Lu, Oliver Wang, Jun-Yan Zhu
  • Publication number: 20210406302
    Abstract: Multidimensional digital content search techniques are described that support an ability of a computing device to perform search with increased granularity and flexibility over conventional techniques. In one example, a control is implemented by a computing device that defines a multidimensional (e.g., two-dimensional) continuous space. Locations in the multidimensional continuous space are usable to different search criteria through different weights applied to the criteria associated with the axes. Therefore, user interaction with this control may be used to define a location and corresponding coordinates that may act as weights to the search criteria in order to perform a search of digital content through use of a single user input.
    Type: Application
    Filed: June 24, 2020
    Publication date: December 30, 2021
    Applicant: Adobe Inc.
    Inventors: Akhilesh Kumar, Zhe Lin, Ratheesh Kalarot, Jinrong Xie, Jianming Zhang, Baldo Antonio Faieta, Alex Charles Filipkowski