Patents by Inventor Shashank HARINATH

Shashank HARINATH has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20250086467
    Abstract: The described method may include receiving user input indicating a configuration identifying a large language model (LLM) and a subset of documents indicated in the configuration as being available to a tenant. The method may include generating one or more vectorizations of content of the subset of documents. The method may include receiving a request to generate a generative response. The method may include generating the generative artificial intelligence (AI) prompt using the content to ground the generative AI prompt. The subset of documents may be identified based on a comparison between a vectorization of the request and the one or more vectorizations and based at least in part on a determination that a user associated with the tenant is permitted to access the subset of documents. The method may include presenting a response to the generative AI prompt, the response generated by the LLM using the generative AI prompt.
    Type: Application
    Filed: January 30, 2024
    Publication date: March 13, 2025
    Inventors: Victor Yee, Yiqiao Liu, Shashank Harinath, Fermin Ordaz, Adam Smith, Suhail Barot, Tuan Nguyen
  • Publication number: 20250086309
    Abstract: A cloud platform may include a model interface that receives from a client and at an interface for accessing a large language model, a prompt for a response from the large language model, and the client is associated with a set of configuration parameters via a cloud platform that supports the interface. The cloud platform may modify, in accordance with the set of configuration parameters, the prompt that results in a modified prompt and transmit, to the large language model, the modified prompt. The cloud platform may receive the response generated by the large language model and provide the response to a model that determines one or more probabilities that the response contains content from one or more content categories. The cloud platform may transmit the response or the one or more probabilities to the client.
    Type: Application
    Filed: January 11, 2024
    Publication date: March 13, 2025
    Inventors: Shashank Harinath, Eugene Wayne Becker, Subha Melapalayam, Eric Brochu, Claire Cheng, Mario Rodriguez, Prithvi Krisnan Padmanabhan, Kathy Baxter, Kin Fai Kan
  • Patent number: 12197317
    Abstract: Embodiments described herein provide an automated testing pipeline for providing a testing dataset for testing a trained neural network model trained using a first training dataset. A first testing dataset for the trained neural network including a first plurality of user queries is received. A dependency parser is used to filter the first plurality of user queries based on one or more action verbs. A pretrained language model is used to rank the remaining user queries based on respective relationships with queries in the first training dataset. Further, user queries that are classified as keyword matches with the queries in the first training dataset using a bag of words classifier are removed. A second testing dataset is generated using the ranked remaining user queries. Testing outputs are generated, by the trained neural network model, using the second testing dataset.
    Type: Grant
    Filed: January 18, 2023
    Date of Patent: January 14, 2025
    Assignee: Salesforce, Inc.
    Inventors: Shiva Kumar Pentyala, Shashank Harinath, Sitaram Asur, Zachary Alexander
  • Publication number: 20240411991
    Abstract: Embodiments described herein provide a training framework for generative NLP models that operate on previously learnt knowledge from pretrained large language models. Specifically, to train an NLP model to generate a response to a user utterance (e.g., “resolve login issue”), document embeddings of support IT documents encoded by a pretrained LLM are fed to an NLP decoder together with a training dialogue (e.g., a dialogue between the chat agent on how to “resolve login issue”). The NLP decoder can thus be trained by a causal language modeling loss computed based on the predicted next token and the ground-truth token from the training dialogue.
    Type: Application
    Filed: June 6, 2023
    Publication date: December 12, 2024
    Inventors: Shiva Kumar Pentyala, Prafulla Kumar Choubey, Shashank Harinath, Sitaram Asur, Chien-Sheng Jason Wu, Zachary Alexander, Caiming Xiong
  • Publication number: 20240412059
    Abstract: Embodiments described herein provide A method for training a neural network based model. The methods include receiving a training dataset with a plurality of training samples, and those samples are encoded into representations in feature space. A positive sample is determined from the raining dataset based on a relationship between the given query and the positive sample in feature space. For a given query, a positive sample from the training dataset is selected based on a relationship between the given query and the positive sample in a feature space. One or more negative samples from the training dataset that are within a reconfigurable distance to the positive sample in the feature space are selected, and a loss is computed based on the positive sample and the one or more negative samples. The neural network is trained based on the loss.
    Type: Application
    Filed: June 7, 2023
    Publication date: December 12, 2024
    Inventors: Regunathan Radhakrishnan, Zachary Alexander, Sitaram Asur, Shashank Harinath, Na Cheng, Shiva Kumar Pentyala
  • Publication number: 20240411992
    Abstract: Embodiments described herein provide a training framework for generative NLP models. Specifically, the training input, e.g., in the form of a sequence of tokens representing a user-agent dialogue, may be randomly masked for a few spans, which can be one or more tokens, one or more words, one or more sentences, or one or more paragraphs. These masked spans are replaced with their embeddings generated from pre-trained large language models are then used for training the NLP model.
    Type: Application
    Filed: June 15, 2023
    Publication date: December 12, 2024
    Inventors: Shiva Kumar Pentyala, Prafulla Kumar Choubey, Shashank Harinath, Sitaram Asur, Chien-Sheng Jason Wu, Zachary Alexander, Caiming Xiong
  • Publication number: 20240303473
    Abstract: Embodiments provide a generative AI creation framework to a customized generative AI stack using a foundational model (such as GPT) based on user-defined prompts, a natural language description of the task to be accomplished, and domain adaptation. In one embodiment, organization-specific knowledge may be injected into either the prompt and/or the foundational model. In this way, the customized generative AI stack thus supports a full spectrum of domain-adaptive prompts to enable a full spectrum of personalized and adaptive AI chat applications.
    Type: Application
    Filed: October 27, 2023
    Publication date: September 12, 2024
    Inventors: Na (Claire) Cheng, Jayesh Govindarajan, Zachary Alexander, Shashank Harinath, Atul Kshirsagar, Fermin Ordaz
  • Publication number: 20240303443
    Abstract: Embodiments provide a generative AI creation framework to a customized generative AI stack using a foundational model (such as GPT) based on user-defined prompts, a natural language description of the task to be accomplished, and domain adaptation. In one embodiment, organization-specific knowledge may be injected into either the prompt and/or the foundational model. In this way, the customized generative AI stack thus supports a full spectrum of domain-adaptive prompts to enable a full spectrum of personalized and adaptive AI chat applications.
    Type: Application
    Filed: October 27, 2023
    Publication date: September 12, 2024
    Inventors: Na (Claire) Cheng, Jayesh Govindarajan, Zachary Alexander, Shashank Harinath, Atul Kshirsagar, Fermin Ordaz
  • Publication number: 20240242022
    Abstract: Embodiments described herein provide a structured conversation summarization framework. A user interface may be provided which allows an agent to perform a conversation with a customer, for example regarding resolving a customer support issue. Utterances by both the agent and customer may be stored, and at the end of the conversation, the utterances may be used to generate a structured summary. The structured summary may include components such as a general summary, an issue summary, and a resolution summary. Using neural network models and heuristics, each component of the summary may be automatically generated.
    Type: Application
    Filed: January 18, 2023
    Publication date: July 18, 2024
    Inventors: Victor Yee, Chien-Sheng Wu, Na Cheng, Alexander R. Fabbri, Zachary Alexander, Nicholas Feinig, Sameer Abhinkar, Shashank Harinath, Sitaram Asur, Jacob Nathaniel Huffman, Wojciech Kryscinski, Caiming Xiong
  • Publication number: 20240241820
    Abstract: Embodiments described herein provide an automated testing pipeline for providing a testing dataset for testing a trained neural network model trained using a first training dataset. A first testing dataset for the trained neural network including a first plurality of user queries is received. A dependency parser is used to filter the first plurality of user queries based on one or more action verbs. A pretrained language model is used to rank the remaining user queries based on respective relationships with queries in the first training dataset. Further, user queries that are classified as keyword matches with the queries in the first training dataset using a bag of words classifier are removed. A second testing dataset is generated using the ranked remaining user queries. Testing outputs are generated, by the trained neural network model, using the second testing dataset.
    Type: Application
    Filed: January 18, 2023
    Publication date: July 18, 2024
    Inventors: Shiva Kumar PENTYALA, Shashank HARINATH, Sitaram ASUR, Zachary ALEXANDER
  • Patent number: 12019984
    Abstract: A method that includes receiving an input at an interactive conversation service that uses an intent classification model. The method may further include generating, using an encoder model of the intent classification model, a set of output vectors corresponding to the input, where the encoder model is configured to determine a set of metrics corresponding to intent classifications. The method may further include determining, using an outlier detection model of the intent classification model, whether the input is in-domain or out-of-domain (OOD) based on a first vector of the set of output vectors satisfying a domain threshold relative to one or more of the intent classifications. The method may further include outputting, by the intent classification model, a second vector of the set of output vectors that indicates the set of metrics corresponding to the intent classifications or an indication that the input is OOD.
    Type: Grant
    Filed: September 20, 2021
    Date of Patent: June 25, 2024
    Assignee: Salesforce, Inc.
    Inventors: Shilpa Bhagavath, Shubham Mehrotra, Abhishek Sharma, Shashank Harinath, Na Cheng, Zineb Laraki
  • Publication number: 20240143945
    Abstract: Embodiments described herein provide a cross-lingual intent classification model that predicts in multiple languages without the need of training data in all the multiple languages. For example, data requirement for training can be reduced to just one utterance per intent label. Specifically, when an utterance is fed to the intent classification model, the model checks whether the utterance is similar to any of the example utterances provided for each intent. If any such utterance(s) are found, the model returns the specified intent, otherwise, it returns out of domain (OOD).
    Type: Application
    Filed: January 30, 2023
    Publication date: May 2, 2024
    Inventors: Shubham Mehrotra, Zachary Alexander, Shilpa Bhagavath, Gurkirat Singh, Shashank Harinath, Anuprit Kale
  • Patent number: 11922303
    Abstract: Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.
    Type: Grant
    Filed: May 18, 2020
    Date of Patent: March 5, 2024
    Assignee: Salesforce, Inc.
    Inventors: Wenhao Liu, Ka Chun Au, Shashank Harinath, Bryan McCann, Govardana Sachithanandam Ramachandran, Alexis Roos, Caiming Xiong
  • Patent number: 11880659
    Abstract: Methods and systems for hierarchical natural language understanding are described. A representation of an utterance is inputted to a first machine learning model to obtain information on the first utterance. According to the information on the utterance a determination that the representation of the utterance is to be inputted to a second machine learning model that performs a dedicated natural language task is performed. In response to determining that the representation of the utterance is to be inputted to a second machine learning model, the utterance is inputted to the second machine learning model to obtain an output of the dedicated natural language task.
    Type: Grant
    Filed: January 29, 2021
    Date of Patent: January 23, 2024
    Assignee: Salesforce, Inc.
    Inventors: Shiva Kumar Pentyala, Jean-Marc Soumet, Shashank Harinath, Shilpa Bhagavath, Johnson Liu, Ankit Chadha
  • Publication number: 20230333901
    Abstract: Techniques are disclosed that pertain to facilitating the execution of machine learning (ML) models. A computer system may implement an ML model layer that permits ML models built using any of a plurality of different ML model frameworks to be submitted without a submitting entity having to define execution logic for a submitted ML model. The computer system may receive, via the ML model layer, configuration metadata for a particular ML model. The computer system may then receive a prediction request from a user to produce a prediction based on the particular ML model. The computer system may produce a prediction based on the particular ML model. As a part of producing that prediction, the computer system may select, in accordance with the received configuration metadata, one of a plurality of types of hardware resources on which to load the particular ML model.
    Type: Application
    Filed: April 19, 2022
    Publication date: October 19, 2023
    Inventors: Arpeet Kale, Shashank Harinath
  • Publication number: 20230086302
    Abstract: A method that includes receiving an input at an interactive conversation service that uses an intent classification model. The method may further include generating, using an encoder model of the intent classification model, a set of output vectors corresponding to the input, where the encoder model is configured to determine a set of metrics corresponding to intent classifications. The method may further include determining, using an outlier detection model of the intent classification model, whether the input is in-domain or out-of-domain (OOD) based on a first vector of the set of output vectors satisfying a domain threshold relative to one or more of the intent classifications. The method may further include outputting, by the intent classification model, a second vector of the set of output vectors that indicates the set of metrics corresponding to the intent classifications or an indication that the input is OOD.
    Type: Application
    Filed: September 20, 2021
    Publication date: March 23, 2023
    Inventors: Shilpa Bhagavath, Shubham Mehrotra, Abhishek Sharma, Shashank Harinath, Na Cheng, Zineb Laraki
  • Patent number: 11544465
    Abstract: Approaches to using unstructured input to update heterogeneous data stores include receiving unstructured text input, receiving a template for interpreting the unstructured text input, identifying, using an entity classifier, entities in the unstructured text input, identifying one or more potential parent entities from the identified entities based on the template, receiving a selection of a parent entity from the one or more potential parent entities, identifying one or more potential child entities from the identified entities based on the template and the selected parent entity, receiving a selection of a child entity from the one or more potential child entities, identifying an action item in the unstructured text input based on the identified entities and the template, determining, using an intent classifier, an intent of the action item, and updating a data store based on the determined intent, the identified entities, and the selected child entity.
    Type: Grant
    Filed: March 24, 2021
    Date of Patent: January 3, 2023
    Assignee: SALESFORCE.COM, INC.
    Inventors: Michael Machado, John Ball, Thomas Archie Cook, Jr., Shashank Harinath, Roojuta Lalani, Zineb Laraki, Qingqing Liu, Mike Rosenbaum, Karl Ryszard Skucha, Jean-Marc Soumet, Manju Vijayakumar
  • Patent number: 11537899
    Abstract: An embodiment proposed herein uses sparsification techniques to train the neural network with a high feature dimension that may yield desirable in-domain detection accuracy but may prune away dimensions in the output that are less important. Specifically, a sparsification vector is generated based on Gaussian distribution (or other probabilistic distribution) and is used to multiply with the higher dimension output to reduce the number of feature dimensions. The pruned output may be then used for the neural network to learn the sparsification vector. In this way, out-of-distribution detection accuracy can be improved.
    Type: Grant
    Filed: May 18, 2020
    Date of Patent: December 27, 2022
    Assignee: Salesforce.com, Inc.
    Inventors: Govardana Sachithanandam Ramachandran, Ka Chun Au, Shashank Harinath, Wenhao Liu, Alexis Roos, Caiming Xiong
  • Patent number: 11481636
    Abstract: An embodiment provided herein preprocesses the input samples to the classification neural network, e.g., by adding Gaussian noise to word/sentence representations to make the function of the neural network satisfy Lipschitz property such that a small change in the input does not cause much change to the output if the input sample is in-distribution. Method to induce properties in the feature representation of neural network such that for out-of-distribution examples the feature representation magnitude is either close to zero or the feature representation is orthogonal to all class representations. Method to generate examples that are structurally similar to in-domain and semantically out-of domain for use in out-of-domain classification training. Method to prune feature representation dimension to mitigate long tail error of unused dimension in out-of-domain classification. Using these techniques, the accuracy of both in-domain and out-of-distribution identification can be improved.
    Type: Grant
    Filed: May 18, 2020
    Date of Patent: October 25, 2022
    Assignee: Salesforce.com, Inc.
    Inventors: Govardana Sachithanandam Ramachandran, Ka Chun Au, Shashank Harinath, Wenhao Liu, Alexis Roos, Caiming Xiong
  • Patent number: 11436481
    Abstract: A method for natural language processing includes receiving, by one or more processors, an unstructured text input. An entity classifier is used to identify entities in the unstructured text input. The identifying the entities includes generating, using a plurality of sub-classifiers of a hierarchical neural network classifier of the entity classifier, a plurality of lower-level entity identifications associated with the unstructured text input. The identifying the entities further includes generating, using a combiner of the hierarchical neural network classifier, a plurality of higher-level entity identifications associated with the unstructured text input based on the plurality of lower-level entity identifications. Identified entities are provided based on the plurality of higher-level entity identifications.
    Type: Grant
    Filed: September 18, 2018
    Date of Patent: September 6, 2022
    Assignee: SALESFORCE.COM, INC.
    Inventors: Govardana Sachithanandam Ramachandran, Michael Machado, Shashank Harinath, Linwei Zhu, Yufan Xue, Abhishek Sharma, Jean-Marc Soumet, Bryan McCann