Patents Examined by Alexey Shmatov
  • Patent number: 11410050
    Abstract: Various systems and methods are described herein for improving the aggressive development of machine learning systems. In machine learning, there is always a trade-off between allowing a machine learning system to learn as much as it can from training data and overfitting on the training data. This trade-off is important because overfitting usually causes performance on new data to be worse. However, various systems and methods can be utilized to separate the process of detailed learning and knowledge acquisition and the process of imposing restrictions and smoothing estimates, thereby allowing machine learning systems to aggressively learn from training data, while mitigating the effects of overfitting on the training data.
    Type: Grant
    Filed: June 15, 2020
    Date of Patent: August 9, 2022
    Assignee: D5AI LLC
    Inventor: James K. Baker
  • Patent number: 11410073
    Abstract: A device may generate an objective function for determining weights for potential features corresponding to training data. The objective function may be generated using a robust loss function such that the objective function is at least continuously twice differentiable. The objective function may comprise a neighborhood component analysis objective function that includes the robust loss function. The device may determine the weights for the potential features using the objective function. The determining may comprise optimizing a value of the objective function for each potential feature. The weights may represent predictive powers of corresponding potential features. The device may provide the weights for the potential features.
    Type: Grant
    Filed: May 31, 2017
    Date of Patent: August 9, 2022
    Assignee: The MathWorks, Inc.
    Inventors: Ilya V. Narsky, Gautam V. Pendse
  • Patent number: 11403531
    Abstract: The disclosure provides an approach for learning latent representations of data using factorized variational autoencoders (FVAEs). The FVAE framework builds a hierarchical Bayesian matrix factorization model on top of a variational autoencoder (VAE) by learning a VAE that has a factorized representation so as to compress the embedding space and enhance generalization and interpretability. In one embodiment, an FVAE application takes as input training data comprising observations of objects, and the FVAE application learns a latent representation of such data. In order to learn the latent representation, the FVAE application is configured to use a probabilistic VAE to jointly learn a latent representation of each of the objects and a corresponding factorization across time and identity.
    Type: Grant
    Filed: July 19, 2017
    Date of Patent: August 2, 2022
    Assignee: Disney Enterprises, Inc.
    Inventors: G. Peter K. Carr, Zhiwei Deng, Rajitha D. B Navarathna, Yisong Yue, Stephan Marcel Mandt
  • Patent number: 11392825
    Abstract: A system and method to reduce weight storage bits for a deep-learning network includes a quantizing module and a cluster-number reduction module. The quantizing module quantizes neural weights of each quantization layer of the deep-learning network. The cluster-number reduction module reduces the predetermined number of clusters for a layer having a clustering error that is a minimum of the clustering errors of the plurality of quantization layers. The quantizing module requantizes the layer based on the reduced predetermined number of clusters for the layer and the cluster-number reduction module further determines another layer having a clustering error that is a minimum of the clustering errors of the plurality of quantized layers and reduces the predetermined number of clusters for the another layer until a recognition performance of the deep-learning network has been reduced by a predetermined threshold.
    Type: Grant
    Filed: March 20, 2017
    Date of Patent: July 19, 2022
    Inventors: Zhengping Ji, John Wakefield Brothers
  • Patent number: 11386343
    Abstract: Real time detection of cyber threats using behavioral analytics is disclosed. An example method includes obtaining, in real time, attributes for an entity within a population of entities, the attributes being indicative of entity behavior; building an entity probability model using the attributes and associated values collected over a period of time; and establishing a control portion of the entity probability model associated with a portion of the period of time. The example method includes comparing any of the entity attribute values and the entity probability model for other portions of the period of time to the control portion to identify one or more anomalous differences, and executing a remediation action based thereon. Some embodiments include determining a set comprising the anomalous differences and additional anomalous differences for the entity or the entity's peer group, and calculating the set's overall probability to determine if the entity is malicious.
    Type: Grant
    Filed: May 9, 2017
    Date of Patent: July 12, 2022
    Assignee: Elasticsearch B.V.
    Inventors: Stephen Dodson, Thomas Veasey
  • Patent number: 11379743
    Abstract: A computing device determines a recommendation. (A) A first parameter matrix is updated using a first direction matrix and a first step-size parameter value that is greater than one. The first parameter matrix includes a row dimension equal to a number of users of a plurality of users included in a ratings matrix and the ratings matrix includes a missing matrix value. (B) A second parameter matrix is updated using a second direction matrix and a second step-size parameter value that is greater than one. The second parameter matrix includes a column dimension equal to a number of items of a plurality of items included in the ratings matrix. (C) An objective function value is updated based on the first parameter matrix and the second parameter matrix. (D) (A) through (C) are repeated until the first parameter matrix and the second parameter matrix satisfy a convergence test.
    Type: Grant
    Filed: July 28, 2021
    Date of Patent: July 5, 2022
    Assignee: SAS Institute Inc.
    Inventors: Xuejun Liao, Patrick Nathan Koch, Shunping Huang, Yan Xu
  • Patent number: 11370113
    Abstract: An apparatus, system and process for guiding a surgeon during a medical procedure to prevent surgical mistakes are described. The system may include a machine learning medical procedure server that generates one or more machine learning medical procedure models using, at least, medical procedure data captured during medical procedures performed at a plurality of different medical procedure systems. The system may also include a medical procedure system communicably coupled with the machine learning medical procedure server that receives a selected machine learning medical procedure model from the machine learning medical procedure server, and utilizes the selected machine learning medical procedure model during a corresponding medical procedure to control one or more operations of the medical procedure system.
    Type: Grant
    Filed: August 25, 2017
    Date of Patent: June 28, 2022
    Assignee: Verily Life Sciences LLC
    Inventors: Joëlle Barral, Martin Habbecke, Daniele Piponi, Thomas Teisseyre
  • Patent number: 11354590
    Abstract: Rule determination for black-box machine-learning models (BBMLMs) is described. These rules are determined by an interpretation system to describe operation of a BBMLM to associate inputs to the BBMLM with observed outputs of the BBMLM and without knowledge of the logic used in operation by the BBMLM to make these associations. To determine these rules, the interpretation system initially generates a proxy black-box model to imitate the behavior of the BBMLM based solely on data indicative of the inputs and observed outputs—since the logic actually used is not available to the system. The interpretation system generates rules describing the operation of the BBMLM by combining conditions—identified based on output of the proxy black-box model—using a genetic algorithm. These rules are output as if-then statements configured with an if-portion formed as a list of the conditions and a then-portion having an indication of the associated observed output.
    Type: Grant
    Filed: November 14, 2017
    Date of Patent: June 7, 2022
    Assignee: Adobe Inc.
    Inventors: Piyush Gupta, Sukriti Verma, Pratiksha Agarwal, Nikaash Puri, Balaji Krishnamurthy
  • Patent number: 11354574
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for increasing the security of neural network by discretizing neural network inputs. One of the methods includes receiving a network input for a neural network; processing the network input using a discretization layer, wherein the discretization layer is configured to generate a discretized network input comprising a respective discretized vector for each of the numeric values in the network input; and processing the discretized network input using the plurality of additional neural network layers to generate a network output for the network input.
    Type: Grant
    Filed: April 27, 2020
    Date of Patent: June 7, 2022
    Assignee: Google LLC
    Inventors: Aurko Roy, Ian Goodfellow, Jacob Buckman, Colin Abraham Raffel
  • Patent number: 11354749
    Abstract: Described herein is computing device for machine learning based risk analysis and decision automation in intelligent underwriting. In accordance with one aspect of the framework, input data of underwriting cases is preprocessed and used to train a predictive model. The predictive model may be cross-validated and tested for accuracy. The predictive model may then be applied to a current underwriting case in order to automatically identify and assess associated risks and provide a decision recommendation.
    Type: Grant
    Filed: April 27, 2017
    Date of Patent: June 7, 2022
    Assignee: SAP SE
    Inventors: Yang Jiang, Xiangyu Pang, Baiquan Liu, Dong Wang
  • Patent number: 11315018
    Abstract: A method, computer readable medium, and system are disclosed for neural network pruning. The method includes the steps of receiving first-order gradients of a cost function relative to layer parameters for a trained neural network and computing a pruning criterion for each layer parameter based on the first-order gradient corresponding to the layer parameter, where the pruning criterion indicates an importance of each neuron that is included in the trained neural network and is associated with the layer parameter. The method includes the additional steps of identifying at least one neuron having a lowest importance and removing the at least one neuron from the trained neural network to produce a pruned neural network.
    Type: Grant
    Filed: October 17, 2017
    Date of Patent: April 26, 2022
    Assignee: NVIDIA Corporation
    Inventors: Pavlo Molchanov, Stephen Walter Tyree, Tero Tapani Karras, Timo Oskari Aila, Jan Kautz
  • Patent number: 11314546
    Abstract: A technique for executing a containerized stateful application that is deployed on a stateless computing platform is disclosed. The technique involves deploying a containerized stateful application on a stateless computing platform and executing the stateful application on the stateless computing platform. The technique also involves during execution of the stateful application, evaluating, in an application virtualization layer, events that are generated during execution of the stateful application to identify events that may trigger a change in state of the stateful application and during execution of the stateful application, updating a set of storage objects in response to the evaluations, and during execution of the stateful application, comparing events that are generated by the stateful application to the set of storage objects and redirecting a storage object that corresponds to an event to a persistent data store if the storage object matches a storage object in the set of storage objects.
    Type: Grant
    Filed: November 18, 2016
    Date of Patent: April 26, 2022
    Assignee: DATA ACCELERATOR LTD
    Inventors: Priya Saxena, Matthew Philip Clothier
  • Patent number: 11308419
    Abstract: A method including: generating, from a text corpus, a lexicon of unigrams and bigrams comprising an embedding for each of said unigrams and bigrams; training a machine learning classifier on a training set comprising a subset of said lexicon, wherein each of said unigrams and bigrams in said subset has a sentiment label; applying said machine learning classifier to said lexicon, to (i) predict a sentiment of each of said unigrams and bigrams, and (ii) update said lexicon with the predicted sentiments; and performing statistical analysis on said updated lexicon, to extract one or more sentiment composition lexicons, wherein each of said one or more sentiment composition lexicons is associated with a sentiment composition class.
    Type: Grant
    Filed: November 15, 2018
    Date of Patent: April 19, 2022
    Assignee: International Business Machines Corporation
    Inventors: Ranit Aharonov, Roy Bar-Haim, Alon Halfon, Charles Arthur Jochim, Amir Menczel, Noam Slonim, Orith Toledo-Ronen
  • Patent number: 11308387
    Abstract: A circuit implementing a spiking neural network that includes a learning component that can learn from temporal correlations in the spikes regardless of correlations in the rates. In some embodiments, the learning component comprises a rate-discounting component. In some embodiments, the learning rule computes a rate-normalized covariance (normcov) matrix, detects clusters in this matrix, and sets the synaptic weights according to these clusters. In some embodiments, a synapse with a long-term plasticity rule has an efficacy that is composed by a weight and a fatiguing component. In some embodiments, A Hebbian plasticity component modifies the weight component and a short-term fatigue plasticity component modifies the fatiguing component. The fatigue component increases with increases in the presynaptic spike rate. In some embodiments, the fatigue component increases are implemented in a spike-based manner.
    Type: Grant
    Filed: May 9, 2017
    Date of Patent: April 19, 2022
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Wabe W. Koelmans, Timoleon Moraitis, Abu Sebastian, Tomas Tuma
  • Patent number: 11301751
    Abstract: A method for executing a binarized neural network (BNN) using a switching chip includes describing an artificial neural network application in a binarized form to provide the BNN; configuring a parser of the switching chip to encode an input vector of the BNN in a packet header; configuring a plurality of match-action tables (MATs) of the switching chip to execute, on the input vector encoded in the packet header, one or more of the operations including XNOR, bit counting, and sign operations such that the plurality of MATs are configured to: implement a bitwise XNOR operation between the input vector and a weights matrix to produce a plurality of first stage vectors, implement an algorithm for counting a number of bits set to 1 in the plurality of first stage vectors to produce a plurality of second stage vectors, and implement a sign operation on the second stage vectors.
    Type: Grant
    Filed: October 4, 2017
    Date of Patent: April 12, 2022
    Assignee: NEC CORPORATION
    Inventors: Roberto Bifulco, Giuseppe Siracusano
  • Patent number: 11301755
    Abstract: The disclosure provides a method for predicting a traffic matrix, a computing device, and a storage medium. The method includes: establishing a dataset based on continuous historical traffic matrices; and inputting one or more historical traffic matrices in the dataset into a trained model for predicting traffic matrices, to obtain one or more predicted traffic matrices. The trained model for predicting traffic matrices is obtained by the following actions: establishing a model for predicting traffic matrices based on a correlation-modeling neural network and a temporal-modeling neural network; and training the model for predicting traffic matrices based on a set of training samples, in which the set of training samples includes sample traffic matrices and label traffic matrices corresponding to the sample traffic matrices at prediction moment samples.
    Type: Grant
    Filed: November 3, 2020
    Date of Patent: April 12, 2022
    Assignee: TSINGHUA UNIVERSITY
    Inventors: Dan Li, Kaihui Gao
  • Patent number: 11295207
    Abstract: Boltzmann machines are trained using an objective function that is evaluated by sampling quantum states that approximate a Gibbs state. Classical processing is used to produce the objective function, and the approximate Gibbs state is based on weights and biases that are refined using the sample results. In some examples, amplitude estimation is used. A combined classical/quantum computer produces suitable weights and biases for classification of shapes and other applications.
    Type: Grant
    Filed: November 28, 2015
    Date of Patent: April 5, 2022
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Nathan Wiebe, Krysta Svore, Ashish Kapoor
  • Patent number: 11288576
    Abstract: The technology disclosed predicts quality of base calling during an extended optical base calling process. The base calling process includes pre-prediction base calling process cycles and at least two times as many post-prediction base calling process cycles as pre-prediction cycles. A plurality of time series from the pre-prediction base calling process cycles is given as input to a trained convolutional neural network. The convolutional neural network determines from the pre-prediction base calling process cycles, a likely overall base calling quality expected after post-prediction base calling process cycles. When the base calling process includes a sequence of paired reads, the overall base calling quality time series of the first read is also given as an additional input to the convolutional neural network to determine the likely overall base calling quality after post-prediction cycles of the second read.
    Type: Grant
    Filed: January 5, 2018
    Date of Patent: March 29, 2022
    Assignee: Illumina, Inc.
    Inventors: Anindita Dutta, Amirali Kia
  • Patent number: 11263521
    Abstract: A device, system, product and method of controlling resistive processing units (RPUs), includes applying an input voltage signal to each node of an array of resistive processing units, and controlling a learning rate of the array of resistive processing units by varying an amplitude of the input voltage signal to the array of resistive processing units. A conductance state of the array of resistive processing units is varied according to the amplitude received at each of the resistive processing units of the array of resistive processing units. The controlling of the amplitude of input voltage signal is according to a processor of a control device.
    Type: Grant
    Filed: August 30, 2016
    Date of Patent: March 1, 2022
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Tayfun Gokmen, Yurii A. Vlasov
  • Patent number: 11263704
    Abstract: A system, a machine-readable storage medium storing instructions, and a computer-implemented method are described herein are directed to a Content Optimization Engine that determines a display probability for each content item in a set of content items. Each respective display probability corresponds to a given content item's probability of display in a specific content slot of a plurality of content slots in a social network feed of a target member account in a social network service. The Content Optimization Engine calculates a selection probability for each content item in an ordered set of the content items, based on each display probability and a set of interaction effects. The Content Optimization Engine causes display of the ordered set of content items in the target member account's social network feed based on satisfaction of the first and second targets.
    Type: Grant
    Filed: January 6, 2017
    Date of Patent: March 1, 2022
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Shaunak Chatterjee, Ankan Saha, Kinjal Basu