Patents by Inventor Dipanjan Das
Dipanjan Das has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11887238Abstract: A method and system for generating 2D animated lip images synchronizing to an audio signal for an unseen subject. The system receives an audio signal and a target lip image of an unseen target subject as inputs from a user and processes these inputs to extract a plurality of high dimensional audio image features. The lip generator system is meta-trained with training dataset which consists of large variety of subjects' ethnicity and vocabulary. The meta-trained model generates realistic animation for previously unseen face and unseen audio when finetuned with only a few-shot samples for a predefined interval of time. Additionally, the method protects intrinsic features of the unseen target subject.Type: GrantFiled: August 18, 2021Date of Patent: January 30, 2024Assignee: TATA CONSULTANCY SERVICES LIMITEDInventors: Swapna Agarwal, Dipanjan Das, Brojeshwar Bhowmick
-
Patent number: 11875115Abstract: Systems and methods for automatic evaluation of the quality of NLG outputs. In some aspects of the technology, a learned evaluation model may be pretrained first using NLG model pretraining tasks, and then with further pretraining tasks using automatically generated synthetic sentence pairs. In some cases, following pretraining, the evaluation model may be further fine-tuned using a set of human-graded sentence pairs, so that it learns to approximate the grades allocated by the human evaluators.Type: GrantFiled: December 12, 2022Date of Patent: January 16, 2024Assignee: Google LLCInventors: Thibault Sellam, Dipanjan Das, Ankur Parikh
-
Publication number: 20240012999Abstract: Systems and methods for automatic evaluation of the quality of NLG outputs. In some aspects of the technology, a learned evaluation model may be pretrained first using NLG model pretraining tasks, and then with further pretraining tasks using automatically generated synthetic sentence pairs. In some cases, following pretraining, the evaluation model may be further fine-tuned using a set of human-graded sentence pairs, so that it learns to approximate the grades allocated by the human evaluators.Type: ApplicationFiled: September 25, 2023Publication date: January 11, 2024Inventors: Thibault Sellam, Dipanjan Das, Ankur Parikh
-
Publication number: 20230306209Abstract: Systems and methods for automatic evaluation of the quality of NLG outputs. In some aspects of the technology, a learned evaluation model may be pretrained first using NLG model pretraining tasks, and then with further pretraining tasks using automatically generated synthetic sentence pairs. In some cases, following pretraining, the evaluation model may be further fine-tuned using a set of human-graded sentence pairs, so that it learns to approximate the grades allocated by the human evaluators. In some cases, following fine-tuning, the learned evaluation model may be distilled into a student model.Type: ApplicationFiled: June 2, 2023Publication date: September 28, 2023Inventors: Thibault Sellam, Dipanjan Das, Ankur Parikh
-
Patent number: 11704506Abstract: Systems and methods for automatic evaluation of the quality of NLG outputs. In some aspects of the technology, a learned evaluation model may be pretrained first using NLG model pretraining tasks, and then with further pretraining tasks using automatically generated synthetic sentence pairs. In some cases, following pretraining, the evaluation model may be further fine-tuned using a set of human-graded sentence pairs, so that it learns to approximate the grades allocated by the human evaluators. In some cases, following fine-tuning, the learned evaluation model may be distilled into a student model.Type: GrantFiled: December 4, 2020Date of Patent: July 18, 2023Assignee: Google LLCInventors: Thibault Sellam, Dipanjan Das, Ankur Parikh
-
Patent number: 11670047Abstract: The embodiments herein provide a system and method for integrating objects in monocular simultaneous localization and mapping (SLAM). State of art object SLAM approach use two popular threads. In first, instance specific models are assumed to be known a priori. In second, a general model for an object such as ellipsoids and cuboids is used. However, these generic models just give the label of the object category and do not give much information about the object pose in the map. The method and system disclosed provide a SLAM framework on a real monocular sequence wherein joint optimization is performed on object localization and edges using category level shape priors and bundle adjustment. The method provides a better visualization incorporating object representations in the scene along with the 3D structure of the base SLAM system, which makes it useful for augmented reality (AR) applications.Type: GrantFiled: July 1, 2020Date of Patent: June 6, 2023Assignee: TATA CONSULTANCY SERVICES LIMITEDInventors: Dipanjan Das, Brojeshwar Bhowmick, Aniket Pokale, Krishnan Madhava Krishna, Aditya Aggarwal
-
Publication number: 20230110829Abstract: Systems and methods for automatic evaluation of the quality of NLG outputs. In some aspects of the technology, a learned evaluation model may be pretrained first using NLG model pretraining tasks, and then with further pretraining tasks using automatically generated synthetic sentence pairs. In some cases, following pretraining, the evaluation model may be further fine-tuned using a set of human-graded sentence pairs, so that it learns to approximate the grades allocated by the human evaluators.Type: ApplicationFiled: December 12, 2022Publication date: April 13, 2023Inventors: Thibault Sellam, Dipanjan Das, Ankur Parikh
-
Patent number: 11597080Abstract: Conventional tele-presence robots have their own limitations with respect to task execution, information processing and management. Embodiments of the present disclosure provide a tele-presence robot (TPR) that communicates with a master device associated with a user via an edge device for task execution wherein control command from the master device is parsed for determining instructions set and task type for execution. Based on this determination, the TPR queries for information across storage devices until a response is obtained enough to execute task. The task upon execution is validated with the master device and user. Knowledge acquired, during querying, task execution and validation of the executed task, is dynamically partitioned by the TPR across storage devices namely, on-board memory of the tele-present robot, an edge device, a cloud and a web interface respectively depending upon the task type, operating environment of the tele-presence robot, and other performance affecting parameters.Type: GrantFiled: September 9, 2020Date of Patent: March 7, 2023Assignee: TATA CONSULTANCY SERVICES LIMITEDInventors: Chayan Sarkar, Snehasis Banerjee, Pradip Pramanick, Hrishav Bakul Barua, Soumyadip Maity, Dipanjan Das, Brojeshwar Bhowmick, Ashis Sau, Abhijan Bhattacharyya, Arpan Pal, Balamuralidhar Purushothaman, Ruddra Roy Chowdhury
-
Publication number: 20230059500Abstract: A method for automatically detecting and evaluating experience data associated with an experience journey to identify and adjust an action that is taken to improve customer experience is provided. In some embodiments, the method includes generating performance data associated with the experience journey from the experience data using a machine learning model. The method further includes determining the action to be taken based on analyzing the performance data. The method further includes collecting new experience data responsive to the action having been taken and training the machine learning model using the new experience data. The method further includes updating the performance data and the action to be taken based on training the machine learning model.Type: ApplicationFiled: August 19, 2021Publication date: February 23, 2023Inventors: Dipanjan Das, Rana Saha, Kumar Priyadarshi, Himanshu Shukla, Prakash Hariharan
-
Patent number: 11551002Abstract: Systems and methods for automatic evaluation of the quality of NLG outputs. In some aspects of the technology, a learned evaluation model may be pretrained first using NLG model pretraining tasks, and then with further pretraining tasks using automatically generated synthetic sentence pairs. In some cases, following pretraining, the evaluation model may be further fine-tuned using a set of human-graded sentence pairs, so that it learns to approximate the grades allocated by the human evaluators.Type: GrantFiled: August 26, 2020Date of Patent: January 10, 2023Assignee: GOOGLE LLCInventors: Thibault Sellam, Dipanjan Das, Ankur Parikh
-
Audio-speech driven animated talking face generation using a cascaded generative adversarial network
Patent number: 11551394Abstract: Conventional state-of-the-art methods are limited in their ability to generate realistic animation from audio on any unknown faces and cannot be easily generalized to different facial characteristics and voice accents. Further, these methods fail to produce realistic facial animation for subjects which are quite different than that of distribution of facial characteristics network has seen during training. Embodiments of the present disclosure provide systems and methods that generate audio-speech driven animated talking face using a cascaded generative adversarial network (CGAN), wherein a first GAN is used to transfer lip motion from canonical face to person-specific face. A second GAN based texture generator network is conditioned on person-specific landmark to generate high-fidelity face corresponding to the motion. Texture generator GAN is made more flexible using meta learning to adapt to unknown subject's traits and orientation of face during inference.Type: GrantFiled: March 11, 2021Date of Patent: January 10, 2023Assignee: TATA CONSULTANCY SERVICES LIMITEDInventors: Sandika Biswas, Dipanjan Das, Sanjana Sinha, Brojeshwar Bhowmick -
Publication number: 20220391428Abstract: Techniques are described herein for training and/or utilizing a query canonicalization system. In various implementations, a query canonicalization system can include a classification model and a canonicalization model. A classification model can be used to determine if a search query is well-formed. Additionally or alternatively, a canonicalization model can be used to determine a well-formed variant of a search query in response to determining a search query is not well-formed. In various implementations, a canonicalization model portion of a query canonicalization system can be a sequence to sequence model.Type: ApplicationFiled: August 17, 2022Publication date: December 8, 2022Inventors: Manaal Faruqui, Dipanjan Das
-
Patent number: 11423068Abstract: Techniques are described herein for training and/or utilizing a query canonicalization system. In various implementations, a query canonicalization system can include a classification model and a canonicalization model. A classification model can be used to determine if a search query is well-formed. Additionally or alternatively, a canonicalization model can be used to determine a well-formed variant of a search query in response to determining a search query is not well-formed. In various implementations, a canonicalization model portion of a query canonicalization system can be a sequence to sequence model.Type: GrantFiled: January 18, 2019Date of Patent: August 23, 2022Assignee: GOOGLE LLCInventors: Manaal Faruqui, Dipanjan Das
-
Patent number: 11295501Abstract: Most of the prior art references that generate animations fail to determine and consider head movement data. The prior art references which consider the head movement data for generating the animations rely on a sample video to generate/determine the head movements data, which, as a result, fail to capture changing head motions throughout course of a speech given by a subject in an actual whole length video. The disclosure herein generally relates to generating facial animations, and, more particularly, to a method and system for generating the facial animations from speech signal of a subject. The system determines the head movement, lip movements, and eyeball movements, of the subject, by processing a speech signal collected as input, and uses the head movement, lip movements, and eyeball movements, to generate an animation.Type: GrantFiled: March 1, 2021Date of Patent: April 5, 2022Assignee: Tata Consultancy Services LimitedInventors: Sandika Biswas, Dipanjan Das, Sanjana Sinha, Brojeshwar Bhowmick
-
Publication number: 20220067309Abstract: Systems and methods for automatic evaluation of the quality of NLG outputs. In some aspects of the technology, a learned evaluation model may be pretrained first using NLG model pretraining tasks, and then with further pretraining tasks using automatically generated synthetic sentence pairs. In some cases, following pretraining, the evaluation model may be further fine-tuned using a set of human-graded sentence pairs, so that it learns to approximate the grades allocated by the human evaluators. In some cases, following fine-tuning, the learned evaluation model may be distilled into a student model.Type: ApplicationFiled: December 4, 2020Publication date: March 3, 2022Applicant: Google LLCInventors: Thibault Sellam, Dipanjan Das, Ankur Parikh
-
Publication number: 20220067285Abstract: Systems and methods for automatic evaluation of the quality of NLG outputs. In some aspects of the technology, a learned evaluation model may be pretrained first using NLG model pretraining tasks, and then with further pretraining tasks using automatically generated synthetic sentence pairs. In some cases, following pretraining, the evaluation model may be further fine-tuned using a set of human-graded sentence pairs, so that it learns to approximate the grades allocated by the human evaluators.Type: ApplicationFiled: August 26, 2020Publication date: March 3, 2022Inventors: Thibault Sellam, Dipanjan Das, Ankur Parikh
-
Publication number: 20220058850Abstract: This disclosure relates generally to a method and system for generating 2D animated lip images synchronizing to an audio signal for an unseen subject. Recent advances in Convolutional Neural Network (CNN) based approaches generate convincing talking heads. Personalization of such talking heads requires training the model with large number of samples of the target person which is time consuming. The lip generator system receives an audio signal and a target lip image of an unseen target subject as inputs from a user and processes these inputs to extract a plurality of high dimensional audio image features. The lip generator system is meta-trained with training dataset which consists of large variety of subjects' ethnicity and vocabulary. The meta-trained model generates realistic animation for previously unseen face and unseen audio when finetuned with only a few-shot samples for a predefined interval of time. Additionally, the method protects intrinsic features of the unseen target subject.Type: ApplicationFiled: August 18, 2021Publication date: February 24, 2022Applicant: Tata Consultancy Services LimitedInventors: Swapna AGARWAL, Dipanjan DAS, Brojeshwar BHOWMICK
-
AUDIO-SPEECH DRIVEN ANIMATED TALKING FACE GENERATION USING A CASCADED GENERATIVE ADVERSARIAL NETWORK
Publication number: 20220036617Abstract: Conventional state-of-the-art methods are limited in their ability to generate realistic animation from audio on any unknown faces and cannot be easily generalized to different facial characteristics and voice accents. Further, these methods fail to produce realistic facial animation for subjects which are quite different than that of distribution of facial characteristics network has seen during training. Embodiments of the present disclosure provide systems and methods that generate audio-speech driven animated talking face using a cascaded generative adversarial network (CGAN), wherein a first GAN is used to transfer lip motion from canonical face to person-specific face. A second GAN based texture generator network is conditioned on person-specific landmark to generate high-fidelity face corresponding to the motion. Texture generator GAN is made more flexible using meta learning to adapt to unknown subject's traits and orientation of face during inference.Type: ApplicationFiled: March 11, 2021Publication date: February 3, 2022Applicant: Tata Consultancy Services LimitedInventors: Sandika BISWAS, Dipanjan DAS, Sanjana SINHA, Brojeshwar BHOWMICK -
Publication number: 20210291363Abstract: Conventional tele-presence robots have their own limitations with respect to task execution, information processing and management. Embodiments of the present disclosure provide a tele-presence robot (TPR) that communicates with a master device associated with a user via an edge device for task execution wherein control command from the master device is parsed for determining instructions set and task type for execution. Based on this determination, the TPR queries for information across storage devices until a response is obtained enough to execute task. The task upon execution is validated with the master device and user. Knowledge acquired, during querying, task execution and validation of the executed task, is dynamically partitioned by the TPR across storage devices namely, on-board memory of the tele-present robot, an edge device, a cloud and a web interface respectively depending upon the task type, operating environment of the tele-presence robot, and other performance affecting parameters.Type: ApplicationFiled: September 9, 2020Publication date: September 23, 2021Applicant: Tata Consultancy Services LimitedInventors: Chayan Sarkar, Snehasis Banerjee, Pradip Pramanick, Hrishav Bakul Barua, Soumyadip Maity, Dipanjan Das, Brojeshwar Bhowmick, Ashis Sau, Abhijan Bhattacharyya, Arpan Pal, Balamuralidhar PURUSHOTHAMAN, Ruddra Roy Chowdhury
-
Publication number: 20210042996Abstract: The embodiments herein provide a system and method for integrating objects in monocular simultaneous localization and mapping (SLAM). State of art object SLAM approach use two popular threads. In first, instance specific models are assumed to be known a priori. In second, a general model for an object such as ellipsoids and cuboids is used. However, these generic models just give the label of the object category and do not give much information about the object pose in the map. The method and system disclosed provide a SLAM framework on a real monocular sequence wherein joint optimization is performed on object localization and edges using category level shape priors and bundle adjustment. The method provides a better visualization incorporating object representations in the scene along with the 3D structure of the base SLAM system, which makes it useful for augmented reality (AR) applications.Type: ApplicationFiled: July 1, 2020Publication date: February 11, 2021Applicant: Tata Consultancy Services LimitedInventors: Dipanjan DAS, Brojeshwar BHOWMICK, Aniket POKALE, Krishnan Madhava KRISHNA, Aditya AGGARWAL