Learning Method Patents (Class 706/25)
  • Patent number: 10726328
    Abstract: A method for implementing a convolutional neural network (CNN) accelerator on a target includes identifying characteristics and parameters for the CNN accelerator. Resources on the target are identified. A design for the CNN accelerator is generated in response to the characteristics and parameters of the CNN accelerator and the resources on the target.
    Type: Grant
    Filed: October 9, 2015
    Date of Patent: July 28, 2020
    Assignee: Altera Corporation
    Inventors: Andrew Chaang Ling, Gordon Raymond Chiu, Utku Aydonat
  • Patent number: 10721254
    Abstract: Systems and methods for threat detection in a network are provided. The system obtains recoils for entities that access a network. The records include attributes associated with the entities. The system identifies features for each of the entities based on the attributes. The system generates a feature set for each of the entities. The feature set is generated from the features identified based on the attributes of each of the entities. The system forms clusters of entities based on the feature set for each of the entities. The system classifies each of the clusters with a threat severity score calculated based on scores associated with entities forming each of the clusters. The system determines to generate an alert for an entity in a cluster response to the threat severity score of the cluster being greater than a threshold.
    Type: Grant
    Filed: March 1, 2018
    Date of Patent: July 21, 2020
    Assignee: Crypteia Networks S.A.
    Inventors: Ilias Kotinas, Theocharis Tsigkritis, Giorgos Gkroumas
  • Patent number: 10720235
    Abstract: A method for improving food-related personalized for a user including determining food-related preferences associated with a plurality of users to generate a user food preferences database; collecting dietary inputs from a subject matter expert (SME) at an SME interface associated with the user food preferences database; determining personalized food parameters for the user based on the user food-related preferences and the dietary inputs; receiving feedback associated with the personalized food parameters from the user; and updating the user food preferences database based on the feedback.
    Type: Grant
    Filed: January 25, 2019
    Date of Patent: July 21, 2020
    Assignee: KRAFT FOODS GROUP BRANDS LLC
    Inventors: Tjarko Leifer, Erik Andrejko, Sivan Aldor-Noiman
  • Patent number: 10720217
    Abstract: A memory device includes a plurality of memory cells and a controller. The controller is configured to program each of the memory cells to one of a plurality of program states, and to read the memory cells using a read operation of applied voltages to the memory cells. During the read operation, separations between adjacent ones of the program states vary based on frequencies of use of the program states in the plurality of memory cells.
    Type: Grant
    Filed: April 11, 2019
    Date of Patent: July 21, 2020
    Assignee: Silicon Storage Technology, Inc.
    Inventors: Hieu Van Tran, Steven Lemke, Vipin Tiwari, Nhan Do, Mark Reiten
  • Patent number: 10706327
    Abstract: There is provided with an information processing apparatus. A processing unit obtains output data by inputting training data to a recognition unit. A determination unit determines an error in a discrimination result for the training data obtained by inputting the output data to a plurality of discriminators. A first training unit trains the recognition unit based on the error in the discrimination result.
    Type: Grant
    Filed: July 27, 2017
    Date of Patent: July 7, 2020
    Assignee: CANON KABUSHIKI KAISHA
    Inventor: Yuki Saito
  • Patent number: 10706352
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection neural network. One of the methods includes maintaining a replay memory that stores trajectories generated as a result of interaction of an agent with an environment; and training an action selection neural network having policy parameters on the trajectories in the replay memory, wherein training the action selection neural network comprises: sampling a trajectory from the replay memory; and adjusting current values of the policy parameters by training the action selection neural network on the trajectory using an off-policy actor critic reinforcement learning technique.
    Type: Grant
    Filed: May 3, 2019
    Date of Patent: July 7, 2020
    Assignee: DeepMind Technologies Limited
    Inventors: Ziyu Wang, Nicolas Manfred Otto Heess, Victor Constant Bapst
  • Patent number: 10691133
    Abstract: Methods and systems that allow neural network systems to maintain or increase operational accuracy while being able to operate in various settings. A set of training data is collected over each of at least two different settings. Each setting has a set of characteristics. Examples of setting characteristic types can be time, geographical location, and/or weather condition. Each set of training data is used to train a neural network resulting in a set of coefficients. For each setting, the setting characteristics are associated with the corresponding neural network having the resulting coefficients and neural network structure. A neural network, having the coefficients and neural network structure resulted after training using the training data collected over a setting, would yield optimal results when operated in/under the setting. A database management system can store information relating to, for example, the setting characteristics, neural network coefficients, and/or neural network structures.
    Type: Grant
    Filed: January 3, 2020
    Date of Patent: June 23, 2020
    Assignee: Apex Artificial Intelligence Industries, Inc.
    Inventor: Kenneth Austin Abeloe
  • Patent number: 10685278
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing long-short term memory cells with saturating gating functions. One of the systems includes a first Long Short-Term Memory (LSTM) cell, wherein the first LSTM cell is configured to, for each of the plurality of time steps, generate a new cell state and a new cell output by applying a plurality of gates to a current cell input, a current cell state, and a current cell output, each of the plurality of gates being configured to, for each of the plurality of time steps: receive a gate input vector, generate a respective intermediate gate output vector from the gate input, and apply a respective gating function to each component of the respective intermediate gate output vector, wherein the respective gating function for at least one of the plurality of gates is a saturating gating function.
    Type: Grant
    Filed: December 30, 2019
    Date of Patent: June 16, 2020
    Assignee: Google LLC
    Inventors: Sergey Ioffe, Raymond Wensley Smith
  • Patent number: 10679118
    Abstract: A spiking neural network (SNN) is defined that includes artificial neurons interconnected by artificial synapses, the SNN defined to correspond to one or more numerical matrices in an equation such that weight values of the synapses correspond to values in the numerical matrices. An input vector is provided to the SNN to correspond to a numerical vector in the equation. A steady state spiking rate is determined for at least a portion of the neurons in the SNN and an approximate result of a matrix inverse problem corresponding to the equation is determined based on values of the steady state spiking rates.
    Type: Grant
    Filed: December 20, 2016
    Date of Patent: June 9, 2020
    Assignee: Intel Corporation
    Inventors: Tsung-Han Lin, Narayan Srinivasa
  • Patent number: 10679120
    Abstract: Embodiments of the present invention relate to providing power minimization in a multi-core neurosynaptic network. In one embodiment of the present invention, a method of and computer program product for power-driven synaptic network synthesis is provided. Power consumption of a neurosynaptic network is modeled as wire length. The neurosynaptic network comprises a plurality of neurosynaptic cores. An arrangement of the synaptic cores is determined by minimizing the wire length.
    Type: Grant
    Filed: November 10, 2014
    Date of Patent: June 9, 2020
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Charles J. Alpert, Pallab Datta, Myron D. Flickner, Zhuo Li, Dharmendra S. Modha, Gi-Joon Nam
  • Patent number: 10671922
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing a neural network. In one aspect, the neural network includes a batch renormalization layer between a first neural network layer and a second neural network layer. The first neural network layer generates first layer outputs having multiple components. The batch renormalization layer is configured to, during training of the neural network on a current batch of training examples, obtain respective current moving normalization statistics for each of the multiple components and determine respective affine transform parameters for each of the multiple components from the current moving normalization statistics. The batch renormalization layer receives a respective first layer output for each training example in the current batch and applies the affine transform to each component of a normalized layer output to generate a renormalized layer output for the training example.
    Type: Grant
    Filed: July 1, 2019
    Date of Patent: June 2, 2020
    Assignee: Google LLC
    Inventor: Sergey Ioffe
  • Patent number: 10664725
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-efficient reinforcement learning. One of the systems is a system for training an actor neural network used to select actions to be performed by an agent that interacts with an environment by receiving observations characterizing states of the environment and, in response to each observation, performing an action selected from a continuous space of possible actions, wherein the actor neural network maps observations to next actions in accordance with values of parameters of the actor neural network, and wherein the system comprises: a plurality of workers, wherein each worker is configured to operate independently of each other worker, wherein each worker is associated with a respective agent replica that interacts with a respective replica of the environment during the training of the actor neural network.
    Type: Grant
    Filed: July 31, 2019
    Date of Patent: May 26, 2020
    Assignee: DeepMind Technologies Limited
    Inventors: Martin Riedmiller, Roland Hafner, Mel Vecerik, Timothy Paul Lillicrap, Thomas Lampe, Ivaylo Popov, Gabriel Barth-Maron, Nicolas Manfred Otto Heess
  • Patent number: 10656605
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a target sequence from a source sequence. In one aspect, the system includes a recurrent neural network configured to, at each time step, receive am input for the time step and process the input to generate a progress score and a set of output scores; and a subsystem configured to, at each time step, generate the recurrent neural network input and provide the input to the recurrent neural network; determine, from the progress score, whether or not to emit a new output at the time step; and, in response to determining to emit a new output, select an output using the output scores and emit the selected output as the output at a next position in the output order.
    Type: Grant
    Filed: May 2, 2019
    Date of Patent: May 19, 2020
    Assignee: Google LLC
    Inventors: Chung-Cheng Chiu, Navdeep Jaitly, Ilya Sutskever, Yuping Luo
  • Patent number: 10657175
    Abstract: Methods and a computer-readable storage device are disclosed for generating a frequency representation of a query audio file. The frequency representation represents information about at least a number of frequencies within a time range containing a number of time frames of the audio content information and a level associated with each of said frequencies. At least one of area of data points in the frequency representation is selected. A fingerprint for each selected area of data points is generated by applying a trained neural network onto said selected area of data points thereby generating a vector in a metric space. A distance between at least one of the generated query fingerprints and at least one reference fingerprint is calculated using a specified distance metric. A reference audio file having associated reference fingerprints which have produced at least one associated distance satisfying a predetermined threshold is identified.
    Type: Grant
    Filed: October 26, 2018
    Date of Patent: May 19, 2020
    Assignee: Spotify AB
    Inventors: Jonathan Donier, Till Hoffmann
  • Patent number: 10650045
    Abstract: An apparatus includes a processor to: train a first neural network of a chain to generate first configuration data including first trained parameters, wherein the chain performs an analytical function generating a set of output values from a set of input values, each neural network has inputs to receive the set of input values and outputs to output a portion of the set of output values, and the neural networks are ordered from the first at the head to a last neural network at the tail, and are interconnected so that each neural network additionally receives the outputs of a preceding neural network; train, using the first configuration data, a next neural network in the chain ordering to generate next configuration data including next trained parameters; and use at least the first and next configuration data and data indicating the interconnections to instantiate the chain to perform the analytical function.
    Type: Grant
    Filed: August 30, 2019
    Date of Patent: May 12, 2020
    Assignee: SAS INSTITUTE INC.
    Inventors: Henry Gabriel Victor Bequet, Jacques Rioux, John Alejandro Izquierdo, Huina Chen, Juan Du
  • Patent number: 10652617
    Abstract: In one embodiment, a method separates subscriber features generated from subscriber interaction with a video delivery service into feature dimensions and inputs the feature dimensions into a respective prediction network. Each prediction network is trained to output a respective dimension score. The method outputs dimension scores using parameters in the plurality of prediction networks that are trained using a variance term to control a variance of the plurality of feature dimensions and using a de-correlation term to control a correlation of the plurality of feature dimensions. The dimension scores are combined into a retention prediction score and an action is performed on the video delivery service for the subscriber based on the retention score.
    Type: Grant
    Filed: April 4, 2018
    Date of Patent: May 12, 2020
    Assignee: HULU, LLC
    Inventors: Nathan Becker, Colin Zhou, Matthew Holcombe, Atul Arun Phadnis, Hang Li, Sridhar Srinivasa Subramanian, Kristen Huff
  • Patent number: 10650830
    Abstract: Processing circuitry of an information processing apparatus obtains a set of identity vectors that are calculated according to voice samples from speakers. The identity vectors are classified into speaker classes respectively corresponding to the speakers. The processing circuitry selects, from the identity vectors, first subsets of interclass neighboring identity vectors respectively corresponding to the identity vectors and second subsets of intraclass neighboring identity vectors respectively corresponding to the identity vectors. The processing circuitry determines an interclass difference based on the first subsets of interclass neighboring identity vectors and the corresponding identity vectors; and determines an intraclass difference based on the second subsets of intraclass neighboring identify vectors and the corresponding identity vectors.
    Type: Grant
    Filed: April 16, 2018
    Date of Patent: May 12, 2020
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Wei Li, Binghua Qian, Xingming Jin, Ke Li, Fuzhang Wu, Yongjian Wu, Feiyue Huang
  • Patent number: 10635944
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an object representation neural network. One of the methods includes obtaining training sets of images, each training set comprising: (i) a before image of a before scene of the environment, (ii) an after image of an after scene of the environment after the robot has removed a particular object, and (iii) an object image of the particular object, and training the object representation neural network on the batch of training data, comprising determining an update to the object representation parameters that encourages the vector embedding of the particular object in each training set to be closer to a difference between (i) the vector embedding of the after scene in the training set and (ii) the vector embedding of the before scene in the training set.
    Type: Grant
    Filed: June 17, 2019
    Date of Patent: April 28, 2020
    Assignee: Google LLC
    Inventors: Eric Victor Jang, Sergey Vladimir Levine, Coline Manon Devin
  • Patent number: 10635975
    Abstract: A disclosed machine learning method includes: calculating a first output error between a label and an output in a case where dropout in which values are replaced with 0 is executed for a last layer of a first channel among plural channels in a parallel neural network; calculating a second output error between the label and an output in a case where the dropout is not executed for the last layer of the first channel; and identifying at least one channel from the plural channels based on a difference between the first output error and the second output error to update parameters of the identified channel.
    Type: Grant
    Filed: December 19, 2016
    Date of Patent: April 28, 2020
    Assignee: FUJITSU LIMITED
    Inventor: Yuhei Umeda
  • Patent number: 10628710
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing images or features of images using an image classification system that includes a batch normalization layer. One of the systems includes a convolutional neural network configured to receive an input comprising an image or image features of the image and to generate a network output that includes respective scores for each object category in a set of object categories, the score for each object category representing a likelihood that that the image contains an image of an object belonging to the category, and the convolutional neural network comprising: a plurality of neural network layers, the plurality of neural network layers comprising a first convolutional neural network layer and a second neural network layer; and a batch normalization layer between the first convolutional neural network layer and the second neural network layer.
    Type: Grant
    Filed: December 19, 2018
    Date of Patent: April 21, 2020
    Assignee: Google LLC
    Inventors: Sergey Ioffe, Corinna Cortes
  • Patent number: 10628731
    Abstract: Systems and methods are provided for automatically scoring a constructed response. The constructed response is processed to generate a plurality of numerical vectors that is representative of the constructed response. A model is applied to the plurality of numerical vectors. The model includes an input layer configured to receive the plurality of numerical vectors, the input layer being connected to a following layer of the model via a first plurality of connections. Each of the connections has a first weight. An intermediate layer of nodes is configured to receive inputs from an immediately-preceding layer of the model via a second plurality of connections, each of the connections having a second weight. An output layer is connected to the intermediate layer via a third plurality of connections, each of the connections having a third weight. The output layer is configured to generate a score for the constructed response.
    Type: Grant
    Filed: June 27, 2019
    Date of Patent: April 21, 2020
    Assignee: Educational Testing Service
    Inventors: Derrick Higgins, Lei Chen, Michael Heilman, Klaus Zechner, Nitin Madnani
  • Patent number: 10628262
    Abstract: A first static server configured to perform at least one first node process and a second static server configured to perform at least one second node process may be instantiated. A conglomerate server may periodically analyze the at least one first node process and the at least one second node process to identify a network process state based on the at least one first node process and the at least one second node process. The conglomerate server may store the network process state in a memory. A failure may be detected in the first static server. In response to the detecting, the first static server may be reinstantiated. The reinstantiating may comprise restarting the at least one first node process according to the network process state from the memory.
    Type: Grant
    Filed: October 24, 2018
    Date of Patent: April 21, 2020
    Assignee: Capital One Services, LLC
    Inventors: Austin Walters, Jeremy Goodsitt, Fardin Abdi Taghi Abad
  • Patent number: 10621491
    Abstract: There is described a method for dynamically computing a patient dynamic risk score indicative of an adverse event occurring on a given day. The method comprises obtaining clinical documentation data, socio-demographic data, answers to remote patient monitoring questionnaires, and vital signs data. The method further comprises using, in a feedforward artificial neural network, the clinical documentation data, the socio-demographic data, the answers to remote patient monitoring questionnaires and the vital signs data to compute the patient dynamic risk score indicative of a risk indicative of the adverse event occurring on a given day.
    Type: Grant
    Filed: June 7, 2016
    Date of Patent: April 14, 2020
    Assignee: ALAYA CARE INC.
    Inventor: Jonathan Vallée
  • Patent number: 10614354
    Abstract: A method for implementing a convolutional neural network (CNN) accelerator on a target includes utilizing one or more processing elements to implement a standard convolution layer. A configuration of the CNN accelerator is modified to change a data flow between components on the CNN accelerator. The one or more processing elements is utilized to implement a fully connected layer in response to the change in the data flow.
    Type: Grant
    Filed: February 6, 2016
    Date of Patent: April 7, 2020
    Assignee: Altera Corporation
    Inventors: Utku Aydonat, Gordon Raymond Chiu, Andrew Chaang Ling
  • Patent number: 10614148
    Abstract: A reconfigurable convolution engine for performing a convolution operation on an image is disclosed. A data receiving module receives image data. A determination module determines a kernel size based on the image data, clock speed associated to the convolution engine and available on-chip resources. A generation module generates a plurality of instances based on the kernel size. A configuration module configures an adder engine comprising a plurality of adders configured to operate in a pipelined structure and in parallel with the plurality of instances. An execution module executes the convolution operation on each of the plurality of instances and the adder engine. A filtering module filters an output of the convolution operation by using a filter function configured to operate on a predefined threshold function.
    Type: Grant
    Filed: September 19, 2018
    Date of Patent: April 7, 2020
    Assignee: HCL TECHNOLOGIES LIMITED
    Inventors: Prasanna Venkatesh Balasubramaniyan, Sainarayanan Gopalakrishnan, Gunamani Rajagopal
  • Patent number: 10614358
    Abstract: Disclosed are various embodiments of memristive networks comprising a number of nodes. Memristive nanofibers are used to form conductive and memristive paths in the networks. Each memristive nanofiber may couple one or more nodes to one or more other nodes. In one case, a memristive network includes a first node, a second node, and a memristive fiber that couples the first node to the neural node. The memristive fiber comprises a conductive core and a memristive shell, where the conductive core forms a conductive path between the first node and the second node and the memristive shell forms a memristive path between the first node and the second node.
    Type: Grant
    Filed: January 4, 2019
    Date of Patent: April 7, 2020
    Assignee: University of Florida Research Foundation, Inc.
    Inventors: Juan Claudio Nino, Jack Kendall
  • Patent number: 10603014
    Abstract: An ultrasonic diagnostic apparatus includes a Doppler processing circuitry, an image generating circuitry, and a control circuitry. The Doppler processing circuitry calculates a correlation matrix using a first data string that is a set of reflected wave data generated based on reflected waves that are generated by transmitting ultrasonic waves without making time intervals of transmission pulses uniform on an identical scan line. The Doppler processing circuitry calculates a filter coefficient based on a result of principal component analysis using the correlation matrix. The Doppler processing circuitry extracts a second data string that is included in the first data string, and that is a set of reflected wave data originated from reflected waves of the ultrasonic waves that are reflected on a moving object present on the identical scan line, using the filter coefficient. The Doppler processing circuitry estimates moving object information of the moving object based on the extracted second data string.
    Type: Grant
    Filed: June 5, 2015
    Date of Patent: March 31, 2020
    Assignee: Canon Medical Systems Corporation
    Inventor: Takeshi Sato
  • Patent number: 10607668
    Abstract: The present application discloses a data processing method and apparatus. A specific embodiment of the method includes: preprocessing received to-be-processed input data; obtaining a storage address of configuration parameters of the to-be-processed input data based on a result of the preprocessing and a result obtained by linearly fitting an activation function, the configuration parameters being preset according to curve characteristics of the activation function; acquiring the configuration parameters of the to-be-processed input data according to the storage address; and processing the result of the preprocessing of the to-be-processed input data based on the configuration parameters of the to-be-processed input data and a preset circuit structure, to obtain a processing result.
    Type: Grant
    Filed: September 30, 2016
    Date of Patent: March 31, 2020
    Assignee: Beijing Baidu Netcom Science and Technology Co., Ltd.
    Inventors: Jian Ouyang, Wei Qi, Yong Wang
  • Patent number: 10592786
    Abstract: Methods and systems for generating an annotated dataset for training a deep tracking neural network, and training of the neural network using the annotated dataset. For each object in each frame of a dataset, one or more likelihood functions are calculated to correlate feature score of the object with respective feature scores each associated with one or more previously assigned target identifiers (IDs) in a selected range of frames. A target ID is assigned to the object by assigning a previously assigned target ID associated with a calculated highest likelihood or assigning a new target ID. Track management is performed according to a predefined track management scheme to assign a track type to the object. This is performed for all objects in all frames of the dataset. The resulting annotated dataset contains target IDs and track types assigned to all objects in all frames.
    Type: Grant
    Filed: August 14, 2017
    Date of Patent: March 17, 2020
    Assignee: HUAWEI TECHNOLOGIES CO., LTD.
    Inventor: Ehsan Taghavi
  • Patent number: 10586150
    Abstract: Described is a system for decoding spiking reservoirs even when the spiking reservoir has continuous synaptic plasticity. The system uses a set of training patterns to train a neural network having a spiking reservoir comprised of spiking neurons. A test pattern duration d is estimated for a set of test patterns P, and each test pattern is presented to the spiking reservoir for a duration of d/P seconds. Output spikes from the spiking reservoir are generated via readout neurons. The output spikes are measured and the measurements are used to compute firing rate codes, each firing rate code corresponding to a test pattern in the set of test patterns P. The firing rate codes are used to decode performance of the neural network by computing a discriminability index (DI) to discriminate between test patterns in the set of test patterns P.
    Type: Grant
    Filed: March 18, 2016
    Date of Patent: March 10, 2020
    Assignee: HTL Laboratories, LLC
    Inventors: Youngkwan Cho, Narayan Srinivasa
  • Patent number: 10565498
    Abstract: A data set whose records include respective pairs of entity descriptors with at least some text and a representation of a relationship such as similarity between the entities of the pair is obtained. Using the data set, a neural network model is trained to generate relationship indicators for pairs of entity descriptors. In an extensible token model of the neural network model, a text token of a first attribute of a particular entity descriptor is represented by a plurality of features including a first feature which was added to the token model as a result of a programmatic request. A particular relationship indicator corresponding to a source entity descriptor and a target entity descriptor is generated using the trained neural network model.
    Type: Grant
    Filed: February 28, 2017
    Date of Patent: February 18, 2020
    Assignee: Amazon Technologies, Inc.
    Inventor: Dmitry Vladimir Zhiyanov
  • Patent number: 10558885
    Abstract: A determination method for determining the structure of a convolutional neural network includes acquiring N filters having the weights trained using a training image group as the initial values, where N is a natural number greater than or equal to 1, and increasing the number of the filters from N to M, where M is a natural number greater than or equal to 2 and is greater than N, by adding a filter obtained by performing a transformation used in image processing fields on at least one of the N filters.
    Type: Grant
    Filed: April 12, 2017
    Date of Patent: February 11, 2020
    Assignee: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
    Inventors: Min Young Kim, Luca Rigazio, Sotaro Tsukizawa, Kazuki Kozuka
  • Patent number: 10558935
    Abstract: Technologies are generally described for methods and systems effective to determine a weight benefit associated with application of weights to training data in a machine learning environment. In an example, a device may determine a first function based on the training data, where the training data includes training inputs and training labels. The device may determine a second function based on weighted training data, which is based on application of weights to the training data. The device may determine a third function based on target data, where the target data is generated based on a target function. The target data may include target labels different from the training labels. The device may determine a fourth function based on weighted target data, which is a result of application of weights to the target data. The device may determine the weight benefit based on the first, second, third, and fourth functions.
    Type: Grant
    Filed: August 5, 2014
    Date of Patent: February 11, 2020
    Assignee: California Institute of Technology
    Inventors: Yaser Said Abu-Mostafa, Carlos Roberto Gonzalez
  • Patent number: 10552732
    Abstract: A multi-layer artificial neural network having at least one high-speed communication interface and N computational layers is provided. N is an integer larger than 1. The N computational layers are serially connected via the at least one high-speed communication interface. Each of the N computational layers respectively includes a computation circuit and a local memory. The local memory is configured to store input data and learnable parameters for the computation circuit. The computation circuit in the ith computational layer provides its computation results, via the at least one high-speed communication interface, to the local memory in the (i+1)th computational layer as the input data for the computation circuit in the (i+1)th computational layer, wherein i is an integer index ranging from 1 to (N?1).
    Type: Grant
    Filed: August 22, 2016
    Date of Patent: February 4, 2020
    Assignee: Kneron Inc.
    Inventors: Yilei Li, Yuan Du, Chun-Chen Liu, Li Du
  • Patent number: 10552731
    Abstract: Described is a neuromorphic system implemented in hardware that implements neuron membrane potential update based on the leaky integrate and fire (LIF) model. The system further models synapse weights update based on the spike time-dependent plasticity (STDP) model. The system includes an artificial neural network in which the update scheme of neuron membrane potential and synapse weight are effectively defined and implemented.
    Type: Grant
    Filed: December 28, 2015
    Date of Patent: February 4, 2020
    Assignee: International Business Machines Corporation
    Inventors: Takeo Yasuda, Kohji Hosokawa, Yutaka Nakamura, Junka Okazawa, Masatoshi Ishii
  • Patent number: 10546211
    Abstract: A method is described that includes executing a convolutional neural network layer on an image processor having an array of execution lanes and a two-dimensional shift register. The two-dimensional shift register provides local respective register space for the execution lanes. The executing of the convolutional neural network includes loading a plane of image data of a three-dimensional block of image data into the two-dimensional shift register.
    Type: Grant
    Filed: July 1, 2016
    Date of Patent: January 28, 2020
    Assignee: Google LLC
    Inventors: Ofer Shacham, David Patterson, William R. Mark, Albert Meixner, Daniel Frederic Finchelstein, Jason Rupert Redgrave
  • Patent number: 10535001
    Abstract: A method for training a deep learning algorithm using N-dimensional data sets may be provided. Each data set comprises a plurality of N?1-dimensional data sets. The method comprises selecting a batch size and assembling an equally sized training batch. The samples are selected to be evenly distributed within said respective N-dimensional data sets. The method comprises also starting from a predetermined offset number, wherein the number of samples is equal to the selected batch size number, and feeding said training batches of N?1-dimensional samples into a deep learning algorithm for the training. Upon the training resulting in a learning rate that is below a predetermined level, selecting a different offset number for at least one of said N-dimensional data sets, and going back to the step of assembling. Upon the training resulting in a learning rate that is equal or higher than said predetermined level, the method stops.
    Type: Grant
    Filed: November 6, 2017
    Date of Patent: January 14, 2020
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Umit Cakmak, Lukasz G. Cmielowski, Marek Oszajec, Wojciech Sobala
  • Patent number: 10534994
    Abstract: The present disclosure relates to a computer-implemented method for analyzing one or more hyper-parameters for a multi-layer computational structure. The method may include accessing, using at least one processor, input data for recognition. The input data may include at least one of an image, a pattern, a speech input, a natural language input, a video input, and a complex data set. The method may further include processing the input data using one or more layers of the multi-layer computational structure and performing matrix factorization of the one or more layers. The method may also include analyzing one or more hyper-parameters for the one or more layers based upon, at least in part, the matrix factorization of the one or more layers.
    Type: Grant
    Filed: November 11, 2015
    Date of Patent: January 14, 2020
    Assignee: Cadence Design Systems, Inc.
    Inventors: Piyush Kaul, Samer Lutfi Hijazi, Raul Alejandro Casas, Rishi Kumar, Xuehong Mao, Christopher Rowen
  • Patent number: 10529339
    Abstract: According to a first aspect of the present disclosure, a method for facilitating detection of one or more time series patterns is conceived, comprising building one or more artificial neural networks, wherein, for at least one time series pattern to be detected, a specific one of said artificial neural networks is built, the specific one of said artificial neural networks being configured to produce a decision output and a reliability output, wherein the reliability output is indicative of the reliability of the decision output. According to a second aspect of the present disclosure, a corresponding computer program is provided. According to a third aspect of the present disclosure, a corresponding system for facilitating the detection of one or more time series patterns is provided.
    Type: Grant
    Filed: February 28, 2018
    Date of Patent: January 7, 2020
    Assignee: NXP B.V.
    Inventor: Adrien Daniel
  • Patent number: 10521526
    Abstract: Exemplary systems, apparatus, and methods for evaluating and predicting athletic performance are described. Systems may include a receiver that gathers non-deterministic data on one or more aspects of athletic performance, a deterministic model of the athletic performance, a hybrid processor that creates a conditional probabilistic model from these elements, and a display presenting the evaluated or predicted performance. The system may include sensors affixed to an athlete or their equipment to convey position, acceleration, heart rate, respiration, biomechanical attributes, and detached sensors to record video, audio, and other ambient conditions. Apparatus may include a hybridization processor that communicates the output of conditional probabilistic models directly to athletes, coaches, and trainers using sound, light, or haptic signals, or to spectators using audiovisual enhancements to broadcasts.
    Type: Grant
    Filed: November 20, 2017
    Date of Patent: December 31, 2019
    Assignee: NFL PLAYERS, INC.
    Inventors: Peter D. Haaland, Sean C. Sansiveri, Anthony J. Falcone
  • Patent number: 10521714
    Abstract: Embodiments of the invention provide a neural core circuit comprising a synaptic interconnect network including plural electronic synapses for interconnecting one or more source electronic neurons with one or more target electronic neurons. The interconnect network further includes multiple axon paths and multiple dendrite paths. Each synapse is at a cross-point junction of the interconnect network between a dendrite path and an axon path. The core circuit further comprises a routing module maintaining routing information. The routing module routes output from a source electronic neuron to one or more selected axon paths. Each synapse provides a configurable level of signal conduction from an axon path of a source electronic neuron to a dendrite path of a target electronic neuron.
    Type: Grant
    Filed: January 21, 2016
    Date of Patent: December 31, 2019
    Assignee: International Business Machines Corporation
    Inventors: Steven K. Esser, Dharmendra S. Modha
  • Patent number: 10515313
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a plurality of different types of predictive models using training data, wherein each of the predictive models implements a different machine learning technique. One or more weights are obtained wherein each weight is associated with an answer category in the plurality of examples. A weighted accuracy is calculated for each of the predictive models using the one or more weights.
    Type: Grant
    Filed: October 29, 2014
    Date of Patent: December 24, 2019
    Assignee: Google LLC
    Inventors: Robert Kaplow, Wei-Hao Lin, Gideon S. Mann, Travis H. K. Green, Gang Fu, Robbie A. Haertel
  • Patent number: 10507179
    Abstract: The present invention relates to improved formulations of Levosimendan for pharmaceutical use, and particularly for intravenous administration as infusion or injection and of infusion concentrates. The present invention therefore relates to pharmaceutical compositions comprising Levosimendan, in which Levosimendan is present in a solubilized form. The formulations have therapeutically and commercial useful concentrations of Levosimendan. The solutions of the invention have enhanced ability at physiological pH (pH 7.4) and are particular useful as infusion or injection solutions or infusion concentrates. The composition according to the present invention can also be spray-dried or lyophilized to obtain a dried powder which is very stable and which powder forms the original solution after reconstitution in water or an aqueous solvent. Levosimendan or (?)-[[4-(1,4,5,6-tetrahydro-4-methyl-6-oxo-3-pyridazi-nyl)phenyl]hydrazono]propanedinitrile is useful in the treatment of congestive heart failure.
    Type: Grant
    Filed: November 4, 2016
    Date of Patent: December 17, 2019
    Assignee: CARINOPHARM GMBH
    Inventor: Andrea Weiland
  • Patent number: 10509996
    Abstract: The present disclosure is drawn to the reduction of parameters in fully connected layers of neural networks. For a layer whose output is defined by y=Wx, where y is the output vector, x is the input vector, and W is a matrix of connection parameters, vectors uij and vij are defined and submatrices Wi,j are computed as the outer product of uij and vij, so that Wi,j=vij?uij, and W is obtained by appending submatrices Wi,j.
    Type: Grant
    Filed: September 7, 2016
    Date of Patent: December 17, 2019
    Assignee: Huawei Technologies Co., Ltd.
    Inventors: Barnaby Dalton, Serdar Sozubek, Manuel Saldana, Vanessa Courville
  • Patent number: 10489705
    Abstract: Embodiments are directed to a computer network having pathways. The network includes a plurality of units configured to communicate through the pathways. The network is configured to identify informative looping signals in loops formed from a plurality of network pathways that connect a first one of the plurality of units to a second one of the plurality of units. The network is further configured to apply spike-timing dependent plasticity (STDP) dependent inhibitory gating to the plurality of network pathways. The network is further configured to phase shift open gates and close gates in the loop by applying STDP functions to open gate outputs and closed gates outputs. The network is further configured to make a rate and a direction of the phase shift dependent on a modulatory signal, wherein the modulatory signal is based at least in part on a change in the STDP inhibitory gating.
    Type: Grant
    Filed: January 30, 2015
    Date of Patent: November 26, 2019
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventor: James R. Kozloski
  • Patent number: 10489706
    Abstract: Embodiments are directed to a computer implemented method of implementing a network having pathways. The method includes communicating among a plurality of units through the pathways. The method further includes identifying informative looping signals in loops formed from a plurality of network pathways that connect a first one of the plurality of units to a second one of the plurality of units. The method further includes applying spike-timing dependent plasticity (STDP) dependent inhibitory gating to the plurality of network pathways. The method further includes phase shifting open gates and close gates in the loop by applying STDP functions to open gate outputs and closed gates outputs. The method further includes making a rate and a direction of the phase shift dependent on a modulatory signal, wherein the modulatory signal is based at least in part on a change in the STDP-dependent inhibitory gating.
    Type: Grant
    Filed: June 22, 2015
    Date of Patent: November 26, 2019
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventor: James R. Kozloski
  • Patent number: 10489482
    Abstract: Methods for solving systems of linear equations via utilization of a vector matrix multiplication accelerator are provided. In one aspect, a method includes receiving, from a controller and by the vector matrix multiplication accelerator, an augmented coefficient matrix. The method also comprises implementing Gaussian Elimination using the vector matrix multiplication accelerator by: monitoring, by a register in at least one swap operation, a row order of the augmented coefficient matrix when a first row is swapped with a second row of the augmented coefficient matrix, delivering, by the controller in at least one multiply operation, an analog voltage to a desired row of the augmented coefficient matrix to produce a multiplication result vector, and adding, in at least one add operation, the first row to another desired row of the augmented coefficient matrix to produce an add result vector. Systems and circuits are also provided.
    Type: Grant
    Filed: June 1, 2018
    Date of Patent: November 26, 2019
    Assignee: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP
    Inventors: Catherine Graves, John Paul Strachan
  • Patent number: 10481212
    Abstract: A secondary battery status estimation device includes a sensor unit configured to detect a terminal voltage of a secondary battery, and an internal resistance calculator configured to calculate a direct current internal resistance of the secondary battery based on the terminal voltage and the charge-discharge current detected by the sensor unit. The internal resistance calculator calculates a direct current internal resistance based on the terminal voltage and the charge-discharge current detected by the sensor unit, in a stable period that is before starting a driving source for driving a vehicle and in which the terminal voltage and the charge-discharge current of the secondary battery fall within a predetermined fluctuation range, and in a high-current output period in which electric power for starting the driving source is output from the secondary battery and the terminal voltage of the secondary battery is brought to substantially minimum.
    Type: Grant
    Filed: September 13, 2017
    Date of Patent: November 19, 2019
    Assignee: Panasonic Intellectual Property Management Co., Ltd.
    Inventors: Takuma Iida, Takeshi Chiba, Shunsuke Nitta, Kazuhiro Sugie, Hiroyuki Jimbo
  • Patent number: 10484611
    Abstract: A video tracking system includes a user interface configured to facilitate tracking of a target between video cameras. The user interface includes user controls configured to assist a user in selecting video cameras as the target moves between fields of view of the video cameras. These user controls are automatically associated with specific cameras based on a camera topology relative to a point of view of a camera whose video data is currently being viewed. The video tracking system further includes systems and methods of synchronizing video data generated using the video cameras and of automatically generating a stitched video sequence based on the user selection of video cameras. The target may be tracked in real-time or in previously recorded video and may be track forward or backwards in time.
    Type: Grant
    Filed: December 1, 2016
    Date of Patent: November 19, 2019
    Assignee: Sensormatic Electronics, LLC
    Inventors: Hu Chin, Ken Prayoon Cheng
  • Patent number: 10467528
    Abstract: Techniques herein train a multilayer perceptron, sparsify edges of a graph such as the perceptron, and store edges and vertices of the graph. Each edge has weight. A computer sparsifies perceptron edges. The computer performs a forward-backward pass on the perceptron to calculate a sparse Hessian matrix. Based on that Hessian, the computer performs quasi-Newton perceptron optimization. The computer repeats this until convergence. The computer stores edges in an array and vertices in another array. Each edge has weight and input and output indices. Each vertex has input and output indices. The computer inserts each edge into an input linked list based on its weight. Each link of the input linked list has the next input index of an edge. The computer inserts each edge into an output linked list based on its weight. Each link of the output linked list comprises the next output index of an edge.
    Type: Grant
    Filed: August 11, 2015
    Date of Patent: November 5, 2019
    Assignee: Oracle International Corporation
    Inventors: Dmitry Golovashkin, Uladzislau Sharanhovich, Vaishnavi Sashikanth