Patents by Inventor Yingbo Zhou
Yingbo Zhou has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240118937Abstract: Embodiments herein relate to prediction, based on previous usage of a cloud-based computing resource by a user of one or more users of the cloud-based computing resource, future usage of the cloud-based computing resource. Based on the predicted future usage, embodiments relate to identifying that throttling of access to the cloud-based computing resource is to occur, and notifying the user of the throttling. Other embodiments may be described and/or claimed.Type: ApplicationFiled: October 7, 2022Publication date: April 11, 2024Applicant: Salesforce, Inc.Inventors: Bo Zong, Huan Wang, Tian Lan, Ran Yao, Tony Wong, Daeki Cho, Caiming Xiong, Silvio Savarese, Yingbo Zhou
-
Publication number: 20240118995Abstract: Embodiments described herein provide regression testing using artificial intelligence. A regression testing network model for a first plurality of organizations using a common codebase is provided. The regression testing network model provides an organization finite state machine (FSM) model for each organization. A first dataset including samples of the organization FSM models based on regression testing for one or more previous releases of the common codebase prior to a first release of the common codebase is received. A training dataset is generated based on the first dataset. The regression testing network model using the training dataset. A second plurality of organizations for regression testing for the first release is determined, from the first plurality of organizations, using the trained regression testing network model.Type: ApplicationFiled: October 3, 2022Publication date: April 11, 2024Inventors: Govardana Sachithanandam Ramachandran, Yingbo Zhou, Madhuri Gore, Susan Putvin, Hari Krishna Pottabathula, Ganeswara Rao Dulam
-
Patent number: 11941346Abstract: Embodiments described herein provide methods and systems for effectively and efficiently summarizing long documents. A transformer is provided with bottom-up and top-down inference combined to effectively capture long-range dependency. In the bottom-up inference, each token only attends to nearby tokens within a window of a specified size. In the top-down inference, full self-attention is given using units with coarser granularity. The bottom-up-inferred token representations are then updated with the top-down representations, which is achieved with cross-attention between the top and token levels. Multiple levels of top-down representations with increasingly coarser granularity can be used if documents are extremely long.Type: GrantFiled: January 31, 2022Date of Patent: March 26, 2024Assignee: Salesforce, Inc.Inventors: Bo Pang, Erik Nijkamp, Yingbo Zhou, Caiming Xiong
-
Patent number: 11902221Abstract: A conversation engine performs conversations with users using chatbots customized for performing a set of tasks that can be performed using an online system. The conversation engine loads a chatbot configuration that specifies the behavior of a chatbot including the tasks that can be performed by the chatbot, the types of entities relevant to each task, and so on. The conversation may be voice based and use natural language. The conversation engine may load different chatbot configurations to implement different chatbots. The conversation engine receives a conversation engine configuration that specifies the behavior of the conversation engine across chatbots. The system may be a multi-tenant system that allows customization of the chatbots for each tenant.Type: GrantFiled: September 29, 2020Date of Patent: February 13, 2024Assignee: Salesforce, Inc.Inventors: Xinyi Yang, Tian Xie, Caiming Xiong, Wenhao Liu, Huan Wang, Kazuma Hashimoto, Jin Qu, Feihong Wu, Yingbo Zhou
-
Patent number: 11887599Abstract: A conversation engine performs conversations with users using chatbots customized for performing a set of tasks that can be performed using an online system. The conversation engine loads a chatbot configuration that specifies the behavior of a chatbot including the tasks that can be performed by the chatbot, the types of entities relevant to each task, and so on. The conversation may be voice based and use natural language. The conversation engine may load different chatbot configurations to implement different chatbots. The conversation engine receives a conversation engine configuration that specifies the behavior of the conversation engine across chatbots. The system may be a multi-tenant system that allows customization of the chatbots for each tenant.Type: GrantFiled: February 10, 2023Date of Patent: January 30, 2024Assignee: Salesforce, Inc.Inventors: Xinyi Yang, Tian Xie, Caiming Xiong, Wenhao Liu, Huan Wang, Kazuma Hashimoto, Yingbo Zhou, Xugang Ye, Jin Qu, Feihong Wu
-
Publication number: 20230419027Abstract: Embodiments described herein provide a prompt-based transfer learning method that employs shared latent space prompt tuning). Specifically, a shared latent space is assumed, among all source and target tasks, where each vector in the space captures a basis skill to do a particular task. Given an instance (from either a source task or a target task), it is first encoded into an instance representation vector and then queries the latent space, which yields a skill vector for this instance. This vector modulates a frozen model, via soft prompts which are a simple prompt transformation (the prompt generator in FIG. 3) of the basis skill vector, to generate an answer for the instance. The latent space and prompt transformation are learned end-to-end in upstream pre-training on source tasks.Type: ApplicationFiled: November 30, 2022Publication date: December 28, 2023Inventors: Bo Pang, Semih Yavuz, Caiming Xiong, Yingbo Zhou
-
Patent number: 11829721Abstract: Embodiments described herein provide dynamic blocking, a decoding algorithm which enables large-scale pretrained language models to generate high-quality paraphrases in an un-supervised setting. Specifically, in order to obtain an alternative surface form, when the language model emits a token that is present in the source sequence, the language model is prevented from generating the next token that is the same as the subsequent source token in the source sequence at the next time step. In this way, the language model is forced to generate a paraphrased sequence of the input source sequence, but with mostly different wording.Type: GrantFiled: January 28, 2021Date of Patent: November 28, 2023Assignee: salesforce.com, inc.Inventors: Tong Niu, Semih Yavuz, Yingbo Zhou, Nitish Shirish Keskar, Huan Wang, Caiming Xiong
-
Patent number: 11830477Abstract: An automatic speech recognition (ASR) system that determines a textual representation of a word from a word spoken in a natural language is provided. The ASR system uses an acoustic model, a language model, and a decoder. When the ASR system receives a spoken word, the acoustic model generates word candidates for the spoken word. The language model determines an n-gram score for each word candidate. The n-gram score includes a base score and a bias score. The bias score is based on a logarithmic probability of the word candidate, where the logarithmic probability is derived using a class-based language model where the words are clustered into non-overlapping clusters according to word statistics. The decoder decodes a textual representation of the spoken word from the word candidates and the corresponding n-gram score for each word candidate.Type: GrantFiled: August 14, 2020Date of Patent: November 28, 2023Assignee: Salesforce, Inc.Inventors: Young Mo Kang, Yingbo Zhou
-
Publication number: 20230368078Abstract: A computing device may perform training of a set of machine learning models on a first data set associated with a first domain. In some examples, the training may include, for each machine learning model of the set of machine learning models, inputting, as values for a set of parameters of the respective sets of parameters and for an iteration of a set of iterations, a moving average of the set of parameters calculated over a threshold number of previous iterations. The computing device may select a set of model states that are generated during the training of the plurality of machine learning models based on a validation performance of the set of model states performed during the training. The computing device may then generate an ensembled machine learning model by aggregating the set of machine learning models corresponding to the set of selected model states.Type: ApplicationFiled: May 16, 2022Publication date: November 16, 2023Inventors: Devansh Arpit, Huan Wang, Yingbo Zhou, Caiming Xiong
-
Publication number: 20230280985Abstract: Embodiments described herein provide a program synthesis framework that generates code programs through a multi-turn conversation between a user and a system. Specifically, the description to solve a target problem is factorized into multiple steps, each of which includes a description in natural language (prompt) to be input into the generation model as a user utterance. The model in turn synthesizes functionally correct subprograms following the current user utterance and considering descriptions and synthesized subprograms at previous steps. The subprograms generated at the multiple steps are then combined to form an output of program in response to the target problem.Type: ApplicationFiled: August 17, 2022Publication date: September 7, 2023Inventors: Hiroaki Hayashi, Yingbo Zhou, Bo Pang, Erik Nijkamp
-
Patent number: 11741142Abstract: Embodiments described herein provide document summarization systems and methods that utilize fine-tuning of pre-trained abstractive summarization models to produce summaries that more faithfully track the content of the documents. Such abstractive summarization models may be pre-trained using a corpus consisting of pairs of articles and associated summaries. For each article-summary pair, a pseudo label or control code is generated and represents a faithfulness of the summary with respect to the article. The pre-trained model is then fine-tuned based on the article-summary pairs and the corresponding control codes. The resulting fine-tuned models then provide improved faithfulness in document summarization tasks.Type: GrantFiled: January 31, 2022Date of Patent: August 29, 2023Assignee: salesforce.com, inc.Inventors: Haopeng Zheng, Semih Yavuz, Wojciech Kryscinski, Kazuma Hashimoto, Yingbo Zhou
-
Publication number: 20230229957Abstract: Methods, apparatuses, and computer-program products are disclosed. The method may include inputting one or more subcomponent training datasets into the plurality of subcomponent models of the machine learning model, the machine learning model may be configured to perform a final task, and the plurality of subcomponent models may be configured to perform sequential subtasks that result in the final task. The method may include computing one or more weights for data points of the one or more subcomponent training datasets and the one or more weights may be based on a contribution of the data points to an end-to-end error loss measurement associated with performing the final task of the machine learning model. The method may include training the plurality of subcomponent models based on the one or more weights for the data points of the one or more subcomponent training datasets.Type: ApplicationFiled: January 14, 2022Publication date: July 20, 2023Inventors: Shuyang Li, Yingbo Zhou, Semih Yavuz, Govardana Sachithanandam Ramachandran
-
Publication number: 20230186916Abstract: A conversation engine performs conversations with users using chatbots customized for performing a set of tasks that can be performed using an online system. The conversation engine loads a chatbot configuration that specifies the behavior of a chatbot including the tasks that can be performed by the chatbot, the types of entities relevant to each task, and so on. The conversation may be voice based and use natural language. The conversation engine may load different chatbot configurations to implement different chatbots. The conversation engine receives a conversation engine configuration that specifies the behavior of the conversation engine across chatbots. The system may be a multi-tenant system that allows customization of the chatbots for each tenant.Type: ApplicationFiled: February 10, 2023Publication date: June 15, 2023Inventors: Xinyi Yang, Tian Xie, Caiming Xiong, Wenhao Liu, Huan Wang, Kazuma Hashimoto, Yingbo Zhou, Xugang Ye, Jin Qu, Feihong Wu
-
Patent number: 11676022Abstract: A method for training parameters of a first domain adaptation model. The method includes evaluating a cycle consistency objective using a first task specific model associated with a first domain and a second task specific model associated with a second domain, and evaluating one or more first discriminator models to generate a first discriminator objective using the second task specific model. The one or more first discriminator models include a plurality of discriminators corresponding to a plurality of bands that corresponds domain variable ranges of the first and second domains respectively. The method further includes updating, based on the cycle consistency objective and the first discriminator objective, one or more parameters of the first domain adaptation model for adapting representations from the first domain to the second domain.Type: GrantFiled: August 30, 2021Date of Patent: June 13, 2023Assignee: salesforce.com, inc.Inventors: Ehsan Hosseini-Asl, Caiming Xiong, Yingbo Zhou, Richard Socher
-
Publication number: 20230153542Abstract: Embodiments described herein provide a cross-lingual sentence alignment framework that is trained only on rich-resource language pairs. To obtain an accurate aligner, a pretrained multi-lingual language model is used, and a classifier is trained on parallel data from rich-resource language pairs. This trained classifier may then be used for cross-lingual transfer with low-resource languages.Type: ApplicationFiled: January 21, 2022Publication date: May 18, 2023Inventors: Tong Niu, Kazuma Hashimoto, Yingbo Zhou, Caiming Xiong
-
Patent number: 11645509Abstract: Embodiments for training a neural network using sequential tasks are provided. A plurality of sequential tasks are received. For each task in the plurality of tasks a copy of the neural network that includes a plurality of layers is generated. From the copy of the neural network a task specific neural network is generated by performing an architectural search on the plurality of layers in the copy of the neural network. The architectural search identifies a plurality of candidate choices in the layers of the task specific neural network. Parameters in the task specific neural network that correspond to the plurality of candidate choices and that maximize architectural weights at each layer are identified. The parameters are retrained and merged with the neural network. The neural network trained on the plurality of sequential tasks is a trained neural network.Type: GrantFiled: October 31, 2018Date of Patent: May 9, 2023Assignee: Salesforce.com, Inc.Inventors: Yingbo Zhou, Xilai Li, Caiming Xiong
-
Patent number: 11640527Abstract: Systems and methods are provided for near-zero-cost (NZC) query framework or approach for differentially private deep learning. To protect the privacy of training data during learning, the near-zero-cost query framework transfers knowledge from an ensemble of teacher models trained on partitions of the data to a student model. Privacy guarantees may be understood intuitively and expressed rigorously in terms of differential privacy. Other features are also provided.Type: GrantFiled: October 21, 2019Date of Patent: May 2, 2023Assignee: Salesforce.com, Inc.Inventors: Lichao Sun, Jia Li, Caiming Xiong, Yingbo Zhou
-
Publication number: 20230113750Abstract: A system performs group testing on a population of items. The group testing identifies items satisfying particular criteria from a population of items, for example, defective items from the population. The group testing may be performed for software or hardware testing, for testing a human population, for training of deep learning applications, and so on. The system trains a machine learning based model, for example, a reinforcement learning based model to evaluate groups. The model may further determine system dynamics that may represent priors of items. An agent treats the population and groups of items being tested as the environment and performs actions, for example, adjusting the groups. The system also performs a non-adaptive strategy based on monte carlo simulation of tests based on a simulation results.Type: ApplicationFiled: October 11, 2021Publication date: April 13, 2023Inventors: Lav Raj Varshney, Yingbo Zhou, Caiming Xiong, Govardana Sachithanandam Ramachandran
-
Patent number: 11625436Abstract: Embodiments described herein provide a query autocompletion (QAC) framework at subword level. Specifically, the QAC framework employs a subword encoder that encodes or converts the sequence of input alphabet letters into a sequence of output subwords. The generated subword candidate sequences from the subword encoder is then for the n-gram language model to perform beam search on. For example, as user queries for search engines are in general short, e.g., ranging from 10 to 30 characters. The n-gram language model at subword level may be used for modeling such short contexts and outperforms the traditional language model in both completion accuracy and runtime speed. Furthermore, key computations are performed prior to the runtime to prepare segmentation candidates in support of the subword encoder to generate subword candidate sequences, thus eliminating significant computational overhead.Type: GrantFiled: December 11, 2020Date of Patent: April 11, 2023Assignee: salesforce.com, inc.Inventors: Young Mo Kang, Wenhao Liu, Yingbo Zhou
-
Publication number: 20230107640Abstract: Embodiments described herein provide methods and systems for effectively and efficiently summarizing long documents. A transformer is provided with bottom-up and top-down inference combined to effectively capture long-range dependency. In the bottom-up inference, each token only attends to nearby tokens within a window of a specified size. In the top-down inference, full self-attention is given using units with coarser granularity. The bottom-up-inferred token representations are then updated with the top-down representations, which is achieved with cross-attention between the top and token levels. Multiple levels of top-down representations with increasingly coarser granularity can be used if documents are extremely long.Type: ApplicationFiled: January 31, 2022Publication date: April 6, 2023Inventors: Bo Pang, Erik Nijkamp, Yingbo Zhou, Caiming Xiong