Patents Examined by Kakali Chaki
  • Patent number: 11373087
    Abstract: A method of generating a fixed-point type neural network by quantizing a floating-point type neural network, includes obtaining, by a device, a plurality of post-activation values by applying an activation function to a plurality of activation values that are received from a layer included in the floating-point type neural network, and deriving, by the device, a plurality of statistical characteristics for at least some of the plurality of post-activation values. The method further includes determining, by the device, a step size for the quantizing of the floating-point type neural network, based on the plurality of statistical characteristics, and determining, by the device, a final fraction length for the fixed-point type neural network, based on the step size.
    Type: Grant
    Filed: July 12, 2018
    Date of Patent: June 28, 2022
    Assignee: SAMSUNG ELECTRONICS CO., LTD.
    Inventors: Han-young Yim, Do-yun Kim, Byeoung-su Kim, Nak-woo Sung, Jong-han Lim, Sang-hyuck Ha
  • Patent number: 11366999
    Abstract: It is possible to improve estimation accuracy with regard to data in which significance is attached to a relative phase. Provided is an information processing device including an estimation unit configured to estimate a status by using a neural network. The neural network includes a first complex-valued neural network to which complex data is input, a phase difference computation layer from which phase difference for each element between a plurality of sets with regard to the complex data is output, and a second complex-valued neural network from which complex data is output on the basis of the phase difference.
    Type: Grant
    Filed: November 21, 2017
    Date of Patent: June 21, 2022
    Assignee: Oki Electric Industry Co., Ltd.
    Inventors: Kohei Yamamoto, Kurato Maeno
  • Patent number: 11354565
    Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.
    Type: Grant
    Filed: December 22, 2017
    Date of Patent: June 7, 2022
    Assignee: salesforce.com, inc.
    Inventors: Alexander Rosenberg Johansen, Bryan McCann, James Bradbury, Richard Socher
  • Patent number: 11348000
    Abstract: The present disclosure relates to a computer-implemented method for routing in an electronic design. Embodiments may include receiving, using at least one processor, global route data associated with an electronic design as an input and generating detail route data, based upon, at least in part, the global route data. Embodiments may further include transforming one or more of the detail route data and the global route data into at least one input feature and at least one output result of a deep neural network. Embodiments may also include training the deep neural network with the global route data and the detail route data and predicting an output associated with a detail route based upon, at least in part, a trained deep neural network model. Embodiments may also include generating routing information for each routing grid.
    Type: Grant
    Filed: December 13, 2016
    Date of Patent: May 31, 2022
    Assignee: Cadence Design Systems, Inc.
    Inventors: Weibin Ding, Jie Chen
  • Patent number: 11341396
    Abstract: A deep approximation neural network architecture which extrapolates data over unseen states for demand response applications in order to control distribution systems like product distribution systems of which energy distribution systems, e.g. heat or electrical power distribution, are one example. The method is a model-free control technique mainly in the form of Reinforcement Learning (RL) where a controller learns from interaction with the system to be controlled to control product distributions of which energy distribution systems, e.g. heat or electrical power distribution, are one example.
    Type: Grant
    Filed: December 26, 2016
    Date of Patent: May 24, 2022
    Assignee: VITO NV
    Inventors: Bert Claessens, Peter Vrancx
  • Patent number: 11340977
    Abstract: A computer-implemented method and computing system are provided for failure prediction of a batch of manufactured objects. The method includes classifying, by a processor sing a simulation, a set of samples with uniformly distributed parameter values, to generate sample classifications for the batch of manufactured objects. The method further includes determining, by the processor, a centroid of failing ones of the samples in the set, based on the sample classifications. The method also includes generating, by the processor, a new set of samples with a distribution around the centroid of the failing ones of the sample in the set. The method additionally includes populating, by the processor, a nearest neighbor vector space using the new set of samples. The method further includes classifying, by the processor, the new set of samples by performing a nearest neighbor search on the nearest neighbor vector space using a distance metric.
    Type: Grant
    Filed: January 11, 2017
    Date of Patent: May 24, 2022
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Emrah Acar, Gradus Janssen, Rajiv V. Joshi, Tong Li
  • Patent number: 11327825
    Abstract: A computer-implemented method and computing system are provided for failure prediction of a batch of manufactured objects. The method includes classifying, by a processor using a simulation, a set of samples with uniformly distributed parameter values, to generate sample classifications for the batch of manufactured objects. The method further includes determining, by the processor, a centroid of failing ones of the samples in the set, based on the sample classifications. The method also includes generating, by the processor, a new set of samples with a distribution around the centroid of the failing ones of the sample in the set. The method additionally includes populating, by the processor, a nearest neighbor vector space using the new set of samples. The method further includes classifying, by the processor, the new set of samples by performing a nearest neighbor search on the nearest neighbor vector space using a distance metric.
    Type: Grant
    Filed: November 6, 2017
    Date of Patent: May 10, 2022
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Emrah Acar, Gradus Janssen, Rajiv V. Joshi, Tong Li
  • Patent number: 11321616
    Abstract: A method for generating an operational rule associated with a building management system includes identifying, with a processing device, a first pattern associated with a series of operational observations corresponding to a property of the building management system, correlating a first contextual attribute with the first pattern, and deriving the operational rule at least in part based on the first pattern and the first contextual attribute.
    Type: Grant
    Filed: October 12, 2016
    Date of Patent: May 3, 2022
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Bei Chen, Joern Ploennigs, Anika Schumann, Mathieu Sinn
  • Patent number: 11308385
    Abstract: Systems, methods, devices, and other techniques are described herein for training and using neural networks to encode inputs and to process encoded inputs, e.g., to reconstruct inputs from the encoded inputs. A neural network system can include an encoder neural network, a trusted decoder neural network, and an adversary decoder neural network. The encoder neural network processes a primary neural network input and a key input to generate an encoded representation of the primary neural network input. The trusted decoder neural network processes the encoded representation and the key input to generate a first estimated reconstruction of the primary neural network input. The adversary decoder neural network processes the encoded representation without the key input to generate a second estimated reconstruction of the primary neural network input. The encoder and trusted decoder neural networks can be trained jointly, and these networks trained adversarially to the adversary decoder neural network.
    Type: Grant
    Filed: August 3, 2017
    Date of Patent: April 19, 2022
    Assignee: Google LLC
    Inventors: Martin Abadi, David Godbe Andersen
  • Patent number: 11308401
    Abstract: Systems, methods, and computer readable media directed to interactive reinforcement learning with dynamic reuse of prior knowledge are described in various embodiments. The interactive reinforcement learning is adapted for providing computer implemented systems for dynamic action selection based on confidence levels associated with demonstrator data or portions thereof.
    Type: Grant
    Filed: January 31, 2019
    Date of Patent: April 19, 2022
    Assignee: ROYAL BANK OF CANADA
    Inventors: Matthew Edmund Taylor, Zhaodong Wang
  • Patent number: 11301753
    Abstract: A neuron circuit performing synapse learning on weight values includes a first sub-circuit, a second sub-circuit, and a third sub-circuit. The first sub-circuit is configured to receive an input signal from a pre-synaptic neuron circuit and determine whether the received input signal is an active signal having an active synapse value. The second sub-circuit is configured to compare a first cumulative reception counter of active input signals with a learning threshold value based on results of the determination. The third sub-circuit is configured to perform a potentiating learning process based on a first probability value to set a synaptic weight value of at least one previously received input signal to an active value, upon the first cumulative reception counter reaching the learning threshold value, and perform a depressing learning process based on a second probability value to set each of the synaptic weight values to an inactive value.
    Type: Grant
    Filed: November 6, 2018
    Date of Patent: April 12, 2022
    Assignees: Samsung Electronics Co., Ltd., CONSEJO SUPERIOR DE INVESTIGACIONES CIENTIFICAS
    Inventors: Bernabe Linares-Barranco, Amirreza Yousefzadeh, Evangelos Stromatias, Teresa Serrano-Gotarredona
  • Patent number: 11301315
    Abstract: A method and system for improving an automated hardware apparatus failure prediction system is provided. The method includes automatically retrieving operational data associated with operation of a hardware device being monitored for potential failure. Differing time frame software windows associated with observing operational data and hardware device are determined and the operational data is analyzed. In response, an apparatus malfunction prediction software application and a component prediction software application is generated. Features associated with execution of the software applications are generated and a first group of features are added to software code of the apparatus malfunction prediction software application. A second group of features are additionally added to software code of the component prediction software application.
    Type: Grant
    Filed: June 30, 2017
    Date of Patent: April 12, 2022
    Assignee: Kyndryl, Inc.
    Inventors: Chen Ch Bi, Ea-Ee Jan, Ye Wy Wang, Xiang Zhang
  • Patent number: 11295231
    Abstract: Systems, methods, and computer-readable media are disclosed for parallel stochastic gradient descent using linear and non-linear activation functions. One method includes: receiving a set of input examples; receiving a global model; and learning a new global model based on the global model and the set of input examples by iteratively performing the following steps: computing a plurality of local models having a plurality of model parameters based on the global model and at least a portion of the set of input examples; computing, for each local model, a corresponding model combiner based on the global model and at least a portion of the set of input examples; and combining the plurality of local models into the new global model based on the current global model and the plurality of corresponding model combiners.
    Type: Grant
    Filed: May 22, 2017
    Date of Patent: April 5, 2022
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Saeed Maleki, Madanlal S. Musuvathi, Todd D. Mytkowicz
  • Patent number: 11295208
    Abstract: Embodiments of the present invention provide a computer-implemented method for adaptive residual gradient compression for training of a deep learning neural network (DNN). The method includes obtaining, by a first learner, a current gradient vector for a neural network layer of the DNN, in which the current gradient vector includes gradient weights of parameters of the neural network layer that are calculated from a mini-batch of training data. A current residue vector is generated that includes residual gradient weights for the mini-batch. A compressed current residue vector is generated based on dividing the residual gradient weights of the current residue vector into a plurality of bins of a uniform size and quantizing a subset of the residual gradient weights of one or more bins of the plurality of bins. The compressed current residue vector is then transmitted to a second learner of the plurality of learners or to a parameter server.
    Type: Grant
    Filed: December 4, 2017
    Date of Patent: April 5, 2022
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Ankur Agrawal, Daniel Brand, Chia-Yu Chen, Jungwook Choi, Kailash Gopalakrishnan
  • Patent number: 11288568
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for computing Q values for actions to be performed by an agent interacting with an environment from a continuous action space of actions. In one aspect, a system includes a value subnetwork configured to receive an observation characterizing a current state of the environment and process the observation to generate a value estimate; a policy subnetwork configured to receive the observation and process the observation to generate an ideal point in the continuous action space; and a subsystem configured to receive a particular point in the continuous action space representing a particular action; generate an advantage estimate for the particular action; and generate a Q value for the particular action that is an estimate of an expected return resulting from the agent performing the particular action when the environment is in the current state.
    Type: Grant
    Filed: February 9, 2017
    Date of Patent: March 29, 2022
    Assignee: Google LLC
    Inventors: Shixiang Gu, Timothy Paul Lillicrap, Ilya Sutskever, Sergey Vladimir Levine
  • Patent number: 11281962
    Abstract: In certain embodiments, content items may be obtained, where each of the content items may include multiple data types. Machine learning models may be caused to be trained based on the content items to map data in a vector space by providing at least a first portion of each of the content items as input to at least one of the machine learning models and providing at least a second portion of each of the content items as input to at least another one of the machine learning models. A search request for results may be obtained, where the search request includes search parameters. One or more locations within the vector space may be predicted (e.g., by one or more of the machine learning models) based on the search parameters. Information (indicating content items mapped to or proximate the predicted locations) may be provided as a request response.
    Type: Grant
    Filed: September 27, 2017
    Date of Patent: March 22, 2022
    Assignee: Clarifai, Inc.
    Inventors: Matthew Zeiler, David Eigen, Ryan Compton, Christopher Fox
  • Patent number: 11270203
    Abstract: There is provided is a method and an apparatus for training a neural network capable of improving the performance of the neural network by performing intelligent normalization according to a target task of the neural network. The method according to some embodiments of the present disclosure includes transforming the output data into first normalized data using a first normalization technique, transforming the output data into second normalized data using a second normalization technique and generating target normalized data by aggregating the first normalized data and the second normalized data based on a learnable parameter. At this time, a rate at which the first normalization data is applied in the target normalization data is adjusted by the learnable parameter so that the intelligent normalization according to the target task can be performed, and the performance of the neural network can be improved.
    Type: Grant
    Filed: June 12, 2019
    Date of Patent: March 8, 2022
    Assignee: LUNIT INC.
    Inventors: Hyeon Seob Nam, Hyo Eun Kim
  • Patent number: 11270187
    Abstract: A method is provided. The method includes selecting a neural network model, wherein the neural network model includes a plurality of layers, and wherein each of the plurality of layers includes weights and activations; modifying the neural network model by inserting a plurality of quantization layers within the neural network model; associating a cost function with the modified neural network model, wherein the cost function includes a first coefficient corresponding to a first regularization term, and wherein an initial value of the first coefficient is pre-defined; and training the modified neural network model to generate quantized weights for a layer by increasing the first coefficient until all weights are quantized and the first coefficient satisfies a pre-defined threshold, further including optimizing a weight scaling factor for the quantized weights and an activation scaling factor for quantized activations, and wherein the quantized weights are quantized using the optimized weight scaling factor.
    Type: Grant
    Filed: March 7, 2018
    Date of Patent: March 8, 2022
    Inventors: Yoo Jin Choi, Mostafa El-Khamy, Jungwon Lee
  • Patent number: 11263515
    Abstract: A heterogeneous processor architecture for integrating a convolutional neural network (CNN) and a recurrent neural network (RNN) into a single high-performance, low-power chip in a neural network processor architecture, the heterogeneous processor architecture includes: an on-chip integrated circuit including a CNN operator for processing the CNN, an RNN operator for processing the RNN, an operation controller for performing control, a memory for storing data which is to be used by the operators, an interface for externally exchanging data, and a data bus through which data moves between constituent elements, wherein a fully-connected layer constituting the CNN performs data processing by sharing the RNN operator.
    Type: Grant
    Filed: February 5, 2018
    Date of Patent: March 1, 2022
    Assignees: UX FACTORY CO., LTD.
    Inventors: Dongjoo Shin, Jinmook Lee, Jinsu Lee, Ju Hyoung Lee
  • Patent number: 11244230
    Abstract: IoT Big Data information management and control systems and methods for distributed performance monitoring and critical network fault detection comprising a combination of capabilities including: IoT data collection sensor stations receiving sensor input signals and also connected to monitor units providing communication with other monitor units and/or cloud computing resources via IoT telecommunication links, and wherein a first data collection sensor station has expert predesignated other network elements comprising other data collection sensor stations, monitor units, and/or telecommunications equipment for performance and/or fault monitoring based on criticality to said first data collection sensor station operations, thereby extending monitoring and control operations to include distributed interdependent or critical operations being monitored and analyzed throughout the IoT network, and wherein performance and/or fault monitoring signals received by said first data collection sensor station are analyzed
    Type: Grant
    Filed: February 24, 2021
    Date of Patent: February 8, 2022
    Inventor: Robert D. Pedersen