Patents by Inventor Yanshuai CAO
Yanshuai CAO has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11914955Abstract: A computer implemented method is described for conducting text sequence machine learning, the method comprising: receiving an input sequence x=[x1, x2, . . . , xn], to produce a feature vector for a series of hidden states hx=[h1, h2, . . . , hn], wherein the feature vector for the series of hidden states hx is generated by performing pooling over a temporal dimension of all hidden states output by the encoder machine learning data architecture; and extracting from the series of hidden states hx, a mean and a variance parameter, and to encapsulate the mean and the variance parameter as an approximate posterior data structure.Type: GrantFiled: May 21, 2020Date of Patent: February 27, 2024Assignee: ROYAL BANK OF CANADAInventors: Teng Long, Yanshuai Cao, Jackie C. K. Cheung
-
Patent number: 11763129Abstract: A system, electronic device and method for improved neural network training are provided. The improved system is adapted for tracking long range dependence in sequential data during training, and includes bootstrapping a lower bound on the mutual information (MI) over groups of variables (segments or sentences) and subsequently applying the bound to encourage high MI.Type: GrantFiled: March 4, 2020Date of Patent: September 19, 2023Assignee: ROYAL BANK OF CANADAInventors: Yanshuai Cao, Peng Xu
-
Patent number: 11763100Abstract: A system is provided comprising a processor and a memory storing instructions which configure the processor to process an original sentence structure through an encoder neural network to decompose the original sentence structure into an original semantics component and an original syntax component, process the original syntax component through a syntax variation autoencoder (VAE) to receive a syntax mean vector and a syntax covariance matrix, obtain a sampled syntax value from a syntax Gaussian posterior parameterized by the syntax mean vector and the syntax covariance matrix, process the original semantics component through a semantics VAE to receive a semantics mean vector and a semantics covariance matrix, obtain a sampled semantics vector from the Gaussian semantics posterior parameterized by the semantics mean vector and the semantics covariance matrix, and process the sampled syntax vector and the sampled semantics vector through a decoder neural network to compose a new sentence.Type: GrantFiled: May 22, 2020Date of Patent: September 19, 2023Assignee: ROYAL BANK OF CANADAInventors: Peng Xu, Yanshuai Cao, Jackie C. K. Cheung
-
Patent number: 11755916Abstract: An improved computer implemented method and corresponding systems and computer readable media for improving performance of a deep neural network are provided to mitigate effects related to catastrophic forgetting in neural network learning. In an embodiment, the method includes storing, in memory, logits of a set of samples from a previous set of tasks (D1); and maintaining classification information from the previous set of tasks by utilizing the logits for matching during training on a new set of tasks (D2).Type: GrantFiled: September 5, 2019Date of Patent: September 12, 2023Assignee: ROYAL BANK OF CANADAInventors: Yanshuai Cao, Ruitong Huang, Junfeng Wen
-
Patent number: 11615305Abstract: A variational hyper recurrent neural network (VHRNN) can be trained by, for each step in sequential training data: determining a prior probability distribution for a latent variable from a prior network of the VHRNN using an initial hidden state; determining a hidden state from a recurrent neural network (RNN) of the VHRNN using an observation state, the latent variable and the initial hidden state; determining an approximate posterior probability distribution for the latent variable from an encoder network of the VHRNN using the observation state and the initial hidden state; determining a generating probability distribution for the observation state from a decoder network of the VHRNN using the latent variable and the initial hidden state; and maximizing a variational lower bound of a marginal log-likelihood of the training data. The trained VHRNN can be used to generate sequential data.Type: GrantFiled: May 22, 2020Date of Patent: March 28, 2023Assignee: ROYAL BANK OF CANADAInventors: Ruizhi Deng, Yanshuai Cao, Bo Chang, Marcus Brubaker
-
Patent number: 11562244Abstract: Systems, methods, and computer readable media are described to train a compressed neural network with high robustness. The neural network is first adversarially pre-trained with both original data as well as data perturbed by adversarial attacks for some epochs, then “unimportant” weights or filters are pruned through criteria based on their magnitudes or other method (e.g., Taylor approximation of the loss function), and the pruned neural network is retrained with both clean and perturbed data for more epochs.Type: GrantFiled: February 7, 2019Date of Patent: January 24, 2023Assignee: ROYAL BANK OF CANADAInventors: Luyu Wang, Weiguang Ding, Ruitong Huang, Yanshuai Cao, Yik Chau Lui
-
Patent number: 11551041Abstract: A method for acquiring measurements for a data structure corresponding to an array of variable includes: selecting a subset of elements from the data structure; measuring a sampled value for each of the selected subset of elements; storing each of the sampled values in a K-nearest neighbour (KNN) database and labelling the sampled value as certain; generating a predicted value data structure where each predicted element is generated as the value of its nearest neighbor based on the values stored in the KNN database; for each predicted element: retrieve the predicted element's X nearest neighbours for the sampled value in the KNN database, and when a value of the X nearest neighbours is the same as the predicted element, the predicted element is labelled as certain, otherwise the predicted element is labelled the values as uncertain; and repeating until all elements are labelled as certain.Type: GrantFiled: October 31, 2018Date of Patent: January 10, 2023Assignee: ROYAL BANK OF CANADAInventors: Weiguang Ding, Ruitong Huang, Luyu Wang, Yanshuai Cao
-
Publication number: 20220129450Abstract: A computer system and method for answering a natural language question is provided. The system comprises at least one processor and a memory storing instructions which when executed by the processor configure the processor to perform the method. The method comprises receiving a natural language question, generating a SQL query based on the natural language question, generating an explanation regarding a solution to the natural language question as answered by the SQL query, and presenting the solution and the explanation.Type: ApplicationFiled: October 22, 2021Publication date: April 28, 2022Inventors: Yanshuai CAO, Peng XU, Keyi TANG, Wei YANG, Wenjie ZI, Teng LONG, Jackie Chit Kit CHEUNG, Chenyang HUANG, Lili MOU, Hamidreza SHAHIDI, Ákos KÁDÁR
-
Patent number: 11270072Abstract: Systems and methods of automatically generating a coherence score for text data is provided. The approach includes receiving a plurality of string tokens representing decomposed portions of the target text data object. A trained neural network is provided that has been trained against a plurality of corpuses of training text across a plurality of topics. The string tokens are arranged to extract string tokens representing adjacent sentence pairs of the target text data object. For each adjacent sentence pair, the neural network generates a local coherence score representing a coherence level of the adjacent sentence pair of the target text data object, which are then aggregated for each adjacent sentence pair of the target text data object to generate a global coherence score for the target text data object.Type: GrantFiled: October 31, 2019Date of Patent: March 8, 2022Assignee: ROYAL BANK OF CANADAInventors: Yanshuai Cao, Peng Z. Xu, Hamidreza Saghir, Jin Sung Kang, Teng Long, Jackie C. K. Cheung
-
Publication number: 20210319338Abstract: A machine learning failure discriminator machine is described, along with corresponding systems, methods, and non-transitory computer readable media. The approach operates in relation to an iterative machine learning model and includes a phased approach to extract p-values from the iterative machine learning model based on modified versions of the training or validation data sets. The p-values are then used to identify whether various null hypotheses can be rejected, and accordingly, to generate an output data structure indicative of an estimated failure reason, if any. The output data structure may be made available on an API or on a graphical user interface.Type: ApplicationFiled: April 9, 2021Publication date: October 14, 2021Inventors: Yik Chau LUI, Yanshuai CAO
-
Patent number: 11080292Abstract: A platform, device and process is provided for visual construction of operations for data querying. In particular, embodiments described herein provides a platform, device and process for visual construction of nested operations for data querying. The visual construction is a display of one or more projected data spaces enabling a selection of data indicators on the display. The selection is conducted graphically on the visual construction and the system is configured to translate the selection to generate and conduct a query operating visually on the visualized (e.g., projected) data space. The visual data space includes distinct views of the plurality of multi-dimensionality data points mapped to reduced-dimensionality data points with a transformation function associated with each view. The selections are used to augment the multi-dimensionality data points with one or more additional dimensions to track the selections and to perform operations and visualizations.Type: GrantFiled: November 13, 2018Date of Patent: August 3, 2021Assignee: ROYAL BANK OF CANADAInventors: Yanshuai Cao, Luyu Wang
-
Patent number: 11062179Abstract: An electronic device for neural network training includes at least one processor and one or more memories configured to provide or train: a generative adversarial network (GAN) using a generator and a discriminator for: receiving a plurality of training cases; and training the generative adversarial network, based on the plurality of training cases, to classify the training cases; wherein the generator generates hard negative examples for the discriminator.Type: GrantFiled: November 2, 2018Date of Patent: July 13, 2021Assignee: ROYAL BANK OF CANADAInventors: Avishek Bose, Yanshuai Cao
-
Publication number: 20200372352Abstract: A variational hyper recurrent neural network (VHRNN) can be trained by, for each step in sequential training data: determining a prior probability distribution for a latent variable from a prior network of the VHRNN using an initial hidden state; determining a hidden state from a recurrent neural network (RNN) of the VHRNN using an observation state, the latent variable and the initial hidden state; determining an approximate posterior probability distribution for the latent variable from an encoder network of the VHRNN using the observation state and the initial hidden state; determining a generating probability distribution for the observation state from a decoder network of the VHRNN using the latent variable and the initial hidden state; and maximizing a variational lower bound of a marginal log-likelihood of the training data. The trained VHRNN can be used to generate sequential data.Type: ApplicationFiled: May 22, 2020Publication date: November 26, 2020Inventors: Ruizhi DENG, Yanshuai CAO, Bo CHANG, Marcus BRUBAKER
-
Publication number: 20200372369Abstract: Variational Autoencoders (VAEs) have been shown to be effective in modeling complex data distributions. Conventional VAEs operate with fully-observed data during training. However, learning a VAE model from partially-observed data is still a problem. A modified VAE framework is proposed that can learn from partially-observed data conditioned on the fully-observed mask. A model described in various embodiments is capable of learning a proper proposal distribution based on the missing data. The framework is evaluated for both high-dimensional multimodal data and low dimensional tabular data.Type: ApplicationFiled: May 22, 2020Publication date: November 26, 2020Inventors: Yu GONG, Jiawei HE, Thibaut DURAND, Megha NAWHAL, Yanshuai CAO, Gregory MORI, Seyed Hossein HAJIMIRSADEGHI
-
Publication number: 20200372225Abstract: A computer system and method for machine text generation is provided. The system comprises at least one processor and a memory storing instructions which when executed by the processor configure the processor to perform the method.Type: ApplicationFiled: May 22, 2020Publication date: November 26, 2020Inventors: Peng XU, Yanshuai CAO, Jackie C. K. CHEUNG
-
Publication number: 20200372214Abstract: A computer implemented method is described for conducting text sequence machine learning, the method comprising: receiving an input sequence x=[x1, x2, . . . , xn], to produce a feature vector for a series of hidden states hx=[h1, h2, . . . , hn], wherein the feature vector for the series of hidden states hx is generated by performing pooling over a temporal dimension of all hidden states output by the encoder machine learning data architecture; and extracting from the series of hidden states hx, a mean and a variance parameter, and to encapsulate the mean and the variance parameter as an approximate posterior data structure.Type: ApplicationFiled: May 21, 2020Publication date: November 26, 2020Inventors: Teng LONG, Yanshuai CAO, Jackie C. K. CHEUNG
-
Patent number: 10819724Abstract: There is provided a neural network system for detection of domain generation algorithm generated domain names, the neural network system comprising: an input receiver configured for receiving domain names from one or more input sources; a convolutional neural network unit including one or more convolutional layers, the convolutional unit configured for receiving the input text and processing the input text through the one or more convolutional layers; a recurrent neural network unit including one or more long short term memory layers, the recurrent neural network unit configured to process the output from the convolutional neural network unit to perform pattern recognition; and a classification unit including one or more classification layers, the classification unit configured to receive output data from the recurrent neural network unit to perform a determination of whether the input text or portions of the input text are DGA-generated or benign domain names.Type: GrantFiled: April 3, 2018Date of Patent: October 27, 2020Assignee: ROYAL BANK OF CANADAInventors: Ashkan Amiri, Bryce Croll, Cory Fong, Athinthra Krishnaswamy Sethurajan, Vikash Yadav, Sylvester King Chun Chiang, Zhengyi Qin, Cathal Smyth, Yik Chau Lui, Yanshuai Cao
-
Patent number: 10802822Abstract: Systems and methods for computationally generating a set of more “stable” configuration default values that are used for traceability and improving reproducibility of machine learning approaches. Hash values are generated based on a merged/modified configuration and both configuration content and hash are stored together in one or more data structures. These data structures can be used to link back to the actual values used in experiments.Type: GrantFiled: August 21, 2018Date of Patent: October 13, 2020Assignee: ROYAL BANK OF CANADAInventors: Weiguang Ding, Yanshuai Cao
-
Publication number: 20200285964Abstract: A system, electronic device and method for improved neural network training are provided. The improved system is adapted for tracking long range dependence in sequential data during training, and includes bootstrapping a lower bound on the mutual information (MI) over groups of variables (segments or sentences) and subsequently applying the bound to encourage high MI.Type: ApplicationFiled: March 4, 2020Publication date: September 10, 2020Inventors: Yanshuai CAO, Peng XU
-
Patent number: 10739955Abstract: An interactive data visualization system is provided that utilizes unsupervised learning process, to automatically choose the hyperparameters for generating insights, which are then used for visualizing the data using interactive plots that update dynamically in response to input control commands.Type: GrantFiled: May 8, 2018Date of Patent: August 11, 2020Assignee: ROYAL BANK OF CANADAInventors: Luyu Wang, Yanshuai Cao