Patents by Inventor Shalini De Mello

Shalini De Mello has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10373332
    Abstract: A method, computer readable medium, and system are disclosed for dynamic facial analysis. The method includes the steps of receiving video data representing a sequence of image frames including at least one head and extracting, by a neural network, spatial features comprising pitch, yaw, and roll angles of the at least one head from the video data. The method also includes the step of processing, by a recurrent neural network, the spatial features for two or more image frames in the sequence of image frames to produce head pose estimates for the at least one head.
    Type: Grant
    Filed: December 8, 2017
    Date of Patent: August 6, 2019
    Assignee: NVIDIA Corporation
    Inventors: Jinwei Gu, Xiaodong Yang, Shalini De Mello, Jan Kautz
  • Publication number: 20190213439
    Abstract: A temporal propagation network (TPN) system learns the affinity matrix for video image processing tasks. An affinity matrix is a generic matrix that defines the similarity of two points in space. The TPN system includes a guidance neural network model and a temporal propagation module and is trained for a particular computer vision task to propagate visual properties from a key-frame represented by dense data (color), to another frame that is represented by coarse data (grey-scale). The guidance neural network model generates an affinity matrix referred to as a global transformation matrix from task-specific data for the key-frame and the other frame. The temporal propagation module applies the global transformation matrix to the key-frame property data to produce propagated property data (color) for the other frame. For example, the TPN system may be used to colorize several frames of greyscale video using a single manually colorized key-frame.
    Type: Application
    Filed: March 14, 2019
    Publication date: July 11, 2019
    Inventors: Sifei Liu, Shalini De Mello, Jinwei Gu, Varun Jampani, Jan Kautz
  • Publication number: 20190180469
    Abstract: A method, computer readable medium, and system are disclosed for dynamic facial analysis. The method includes the steps of receiving video data representing a sequence of image frames including at least one head and extracting, by a neural network, spatial features comprising pitch, yaw, and roll angles of the at least one head from the video data. The method also includes the step of processing, by a recurrent neural network, the spatial features for two or more image frames in the sequence of image frames to produce head pose estimates for the at least one head.
    Type: Application
    Filed: December 8, 2017
    Publication date: June 13, 2019
    Inventors: Jinwei Gu, Xiaodong Yang, Shalini De Mello, Jan Kautz
  • Publication number: 20190095791
    Abstract: A spatial linear propagation network (SLPN) system learns the affinity matrix for vision tasks. An affinity matrix is a generic matrix that defines the similarity of two points in space. The SLPN system is trained for a particular computer vision task and refines an input map (i.e., affinity matrix) that indicates pixels the share a particular property (e.g., color, object, texture, shape, etc.). Inputs to the SLPN system are input data (e.g., pixel values for an image) and the input map corresponding to the input data to be propagated. The input data is processed to produce task-specific affinity values (guidance data). The task-specific affinity values are applied to values in the input map, with at least two weighted values from each column contributing to a value in the refined map data for the adjacent column.
    Type: Application
    Filed: September 18, 2018
    Publication date: March 28, 2019
    Inventors: Sifei Liu, Shalini De Mello, Jinwei Gu, Ming-Hsuan Yang, Jan Kautz
  • Patent number: 10157309
    Abstract: A method, computer readable medium, and system are disclosed for detecting and classifying hand gestures. The method includes the steps of receiving an unsegmented stream of data associated with a hand gesture, extracting spatio-temporal features from the unsegmented stream by a three-dimensional convolutional neural network (3DCNN), and producing a class label for the hand gesture based on the spatio-temporal features.
    Type: Grant
    Filed: January 9, 2017
    Date of Patent: December 18, 2018
    Assignee: NVIDIA CORPORATION
    Inventors: Pavlo Molchanov, Xiaodong Yang, Shalini De Mello, Kihwan Kim, Stephen Walter Tyree, Jan Kautz
  • Publication number: 20180181809
    Abstract: A method, computer readable medium, and system are disclosed for performing unconstrained appearance-based gaze estimation. The method includes the steps of identifying an image of an eye and a head orientation associated with the image of the eye, determining an orientation for the eye by analyzing, within a convolutional neural network (CNN), the image of the eye and the head orientation associated with the image of the eye, and returning the orientation of the eye.
    Type: Application
    Filed: December 27, 2017
    Publication date: June 28, 2018
    Inventors: Rajeev Ranjan, Shalini De Mello, Jan Kautz
  • Publication number: 20170206405
    Abstract: A method, computer readable medium, and system are disclosed for detecting and classifying hand gestures. The method includes the steps of receiving an unsegmented stream of data associated with a hand gesture, extracting spatio-temporal features from the unsegmented stream by a three-dimensional convolutional neural network (3DCNN), and producing a class label for the hand gesture based on the spatio-temporal features.
    Type: Application
    Filed: January 9, 2017
    Publication date: July 20, 2017
    Inventors: Pavlo Molchanov, Xiaodong Yang, Shalini De Mello, Kihwan Kim, Stephen Walter Tyree, Jan Kautz