Structure Patents (Class 706/26)
  • Patent number: 11055323
    Abstract: A device includes a memory and a node engine. The memory stores first and second tables that values represented by correlithm objects. The first table provides an intermediate output value based on the difference between two input values. The second table provides an output value based on an amplification of the intermediate output value from the first table. The node engine receives a first input correlithm object and a second input correlithm object, and identifies an amplification correlithm object. The node engine determines an output correlithm object by using the first and second tables in conjunction with the first input correlithm object, the second input correlithm object, and the amplified value correlithm object. The output correlithm object is based on the amplified difference between the first input correlithm object and the second input correlithm object.
    Type: Grant
    Filed: January 30, 2020
    Date of Patent: July 6, 2021
    Assignee: Bank of America Corporation
    Inventor: Patrick N. Lawrence
  • Patent number: 11055240
    Abstract: A data processing method comprises: if detecting that a number of image data to be transferred is greater than zero wherein the number of image data is a product of a number of input image data and a number of output image data, and a first available storage space of a FIFO memory is greater than or equal to a storage space occupied by an N number of input image data, transferring the N input image data in a first memory to the first FIFO memory; if detecting that a number of weight data to be transferred is greater than zero wherein the number of weight data is a product of the number of input image data and the number of output image data, and a second available storage space of a second FIFO memory is greater than or equal to a storage space occupied by an M number of weight data, transferring the M weight data in a second memory to the second FIFO memory; when the number of input image data cached in the first FIFO memory and the number of weight data cached in the second FIFO memory are greater than or e
    Type: Grant
    Filed: October 30, 2019
    Date of Patent: July 6, 2021
    Assignee: Shenzhen Intellifusion Technologies Co., Ltd.
    Inventors: Bo Wen, Qingxin Cao, Wei Li
  • Patent number: 11003992
    Abstract: In one embodiment, a method includes establishing access to first and second different computing systems. A machine learning model is assigned for training to the first computing system, and the first computing system creates a check-point during training in response to a first predefined triggering event. The check-point may be a record of an execution state in the training of the machine learning model by the first computing system. In response to a second predefined triggering event, the training of the machine learning model on the first computing system is halted, and in response to a third predefined triggering event, the training of the machine learning model is transferred to the second computing system, which continues training the machine learning model starting from the execution state recorded by the check-point.
    Type: Grant
    Filed: October 16, 2017
    Date of Patent: May 11, 2021
    Assignee: Facebook, Inc.
    Inventors: Lukasz Wesolowski, Mohamed Fawzi Mokhtar Abd El Aziz, Aditya Rajkumar Kalro, Hongzhong Jia, Jay Parikh
  • Patent number: 10958469
    Abstract: The present disclosure provides for devices, systems, and methods which optimize throughput of bonded connections over multiple variable bandwidth logical paths by adjusting a tunnel bandwidth weighting schema during a data transfer session in response to a change in bandwidth capabilities of one or more tunnels. By making such adjustments, embodiments of the present invention are able to optimize the bandwidth potential of multiple connections being used in a session, while minimizing the adverse consequences of reduced bandwidth issues which may occur during the data transfer session.
    Type: Grant
    Filed: September 23, 2019
    Date of Patent: March 23, 2021
    Assignee: PISMO LABS TECHNOLOGY LIMITED
    Inventors: Patrick Ho Wai Sung, Kam Chiu Ng, Ho Ming Chan
  • Patent number: 10942671
    Abstract: A circuit for a multistage sequential data process includes a plurality of memory units. Each memory unit is associated with a stage of the sequential data process which, for each data set inputted to the stage, the stage provides an intermediate data set for storage in the associated memory unit for use in at least one subsequent stage of the sequential data process, where each of the plurality of memory units is sized based on relative locations of the stage providing the intermediate data set and the at least one subsequent stage in the sequential data process.
    Type: Grant
    Filed: April 25, 2016
    Date of Patent: March 9, 2021
    Assignee: Huawei Technologies Co., Ltd.
    Inventors: Vanessa Courville, Manuel Saldana, Barnaby Dalton
  • Patent number: 10926031
    Abstract: The present disclosure describes a closed-loop fluid resuscitation and/or cardiovascular drug administration system that uses continuous measurements and adaptive control architecture. The adaptive control architecture uses a function approximator to identify unknown dynamics and physiological parameters of a patient to compute appropriate infusion rates and to regulate the endpoint of resuscitation.
    Type: Grant
    Filed: November 11, 2019
    Date of Patent: February 23, 2021
    Assignee: Autonomous Healthcare, Inc.
    Inventors: Behnood Gholami, Timothy S. Phan
  • Patent number: 10922607
    Abstract: In one embodiment, a processor is to store a membrane potential of a neural unit of a neural network; and calculate, at a particular time-step of the neural network, a change to the membrane potential of the neural unit occurring over multiple time-steps that have elapsed since the last time-step at which the membrane potential was updated, wherein each of the multiple time-steps that have elapsed since the last time-step is associated with at least one input to the neural unit that affects the membrane potential of the neural unit.
    Type: Grant
    Filed: December 30, 2016
    Date of Patent: February 16, 2021
    Assignee: Intel Corporation
    Inventors: Abhronil Sengupta, Gregory K. Chen, Raghavan Kumar, Huseyin Ekin Sumbul, Phil Knag
  • Patent number: 10915342
    Abstract: A system that includes an edge node in signal communication an interior node. The edge node is configured to receive a first correlithm object from a first device outside of the network, to identify an input correlithm object from a node table with the shortest distance, to fetch a second correlithm object from the node table linked with the identified input correlithm object, and to send the second correlithm object to the interior node. The edge node is further configured to receive a third correlithm object from the interior node in response to sending the second correlithm to the interior node, to identify an input correlithm object from the node table with the shortest distance, to fetch a fourth correlithm object from the node table linked with the identified input correlithm object, and to send the fourth correlithm object to the first device.
    Type: Grant
    Filed: April 30, 2018
    Date of Patent: February 9, 2021
    Assignee: Bank of America Corporation
    Inventor: Patrick N. Lawrence
  • Patent number: 10915346
    Abstract: A system configured to emulate a correlithm object processing system includes an input node, a first output node, and a second output node. The input node receives a real-world numeric value comprising a mantissa value and an exponent value. The first output node receives the mantissa value and generates a first correlithm object associated with the mantissa value. The second output node receives the exponent value and generates a second correlithm object associated with the exponent value. A string correlithm object engine maps the first correlithm object to a first sub-string correlithm object of a string correlithm object, and maps the second correlithm object to a second sub-string correlithm object of the string correlithm object.
    Type: Grant
    Filed: July 24, 2019
    Date of Patent: February 9, 2021
    Assignee: Bank of America Corporation
    Inventor: Patrick N. Lawrence
  • Patent number: 10902315
    Abstract: The present disclosure relates to a processor for implementing artificial neural networks, for example, convolutional neural networks. The processor includes a memory controller group, an on-chip bus and a processor core, wherein the processor core further includes a register map, an instruction module, a data transferring controller, a data writing scheduling unit, a buffer module, a convolution operation unit and a hybrid computation unit. The processor of the present disclosure may be used for implementing various neural networks with increased computation efficiency.
    Type: Grant
    Filed: May 22, 2017
    Date of Patent: January 26, 2021
    Assignee: XILINX, INC.
    Inventors: Shaoxia Fang, Lingzhi Sui, Qian Yu, Junbin Wang, Yi Shan
  • Patent number: 10896052
    Abstract: A distributed node network to emulate a correlithm object processing system includes a resolution node, a first calculation node, and a second calculation node. The first and second calculation nodes determine n-dimensional distances between an input correlithm object and each of the source correlithm objects that are stored in respective portions of a correlithm object mapping table. The resolution node compares the determined n-dimensional distances from the first calculation node with the determined n-dimensional distance associated with the identified source correlithm object from the second calculation node to identify the smallest determined n-dimensional distance based on the comparison. The resolution node then identifies the target correlithm object associated with the smallest determined n-dimensional distance, and outputs the identified target correlithm object.
    Type: Grant
    Filed: March 26, 2018
    Date of Patent: January 19, 2021
    Assignee: Bank of America Corporation
    Inventor: Patrick N. Lawrence
  • Patent number: 10896369
    Abstract: The present disclosure provides an operation device, comprising: an operation module for executing a neural network operation; and a power conversion module connected to the operation module, for converting input neuron data and/or output neuron data of the neural network operation into power neuron data. The present disclosure further provides an operation method. The operation device and method according to the present disclosure reduce the cost of storage resources and computing resources and increase the operation speed.
    Type: Grant
    Filed: February 22, 2019
    Date of Patent: January 19, 2021
    Assignee: CAMBRICON TECHNOLOGIES CORPORATION LIMITED
    Inventors: Shaoli Liu, Yimin Zhuang, Qi Guo, Tianshi Chen
  • Patent number: 10878710
    Abstract: Systems and methods are disclosed herein for recommending an educational course to a user, and may comprise receiving data records associated with availability of a plurality of educational courses at one or more institutions; receiving educational course data and educational course focus data associated with the user; receiving prior user data records comprising prior user educational course data and prior user educational course focus data; determining index scores for each of the plurality of educational courses based upon a similarity between the educational course data and prior user educational course data, and based upon a similarity between the educational course focus data and prior user educational course focus data; and providing a recommended educational course from the plurality of educational courses to the user based upon the determined index scores.
    Type: Grant
    Filed: September 3, 2015
    Date of Patent: December 29, 2020
    Assignee: EAB Global, Inc.
    Inventors: Catherine Ingrid Shaw, John Daniel Nelson, II, Philip James Friesen, Kathleen Susan Ash Chew, Robert Laurence Alcorn, IV, Sarah Zauner, Brittney Lim Davidson, Christopher Lance Johnson
  • Patent number: 10860922
    Abstract: A method, computer program product, and system perform computations using a sparse convolutional neural network accelerator. A first vector comprising only non-zero weight values and first associated positions of the non-zero weight values within a 3D space is received. A second vector comprising only non-zero input activation values and second associated positions of the non-zero input activation values within a 2D space is received. The non-zero weight values are multiplied with the non-zero input activation values, within a multiplier array, to produce a third vector of products. The first associated positions are combined with the second associated positions to produce a fourth vector of positions, where each position in the fourth vector is associated with a respective product in the third vector. The products in the third vector are transmitted to adders in an accumulator array, based on the position associated with each one of the products.
    Type: Grant
    Filed: November 18, 2019
    Date of Patent: December 8, 2020
    Assignee: NVIDIA Corporation
    Inventors: William J. Dally, Angshuman Parashar, Joel Springer Emer, Stephen William Keckler, Larry Robert Dennison
  • Patent number: 10838749
    Abstract: A distributed node network to emulate a correlithm object processing system includes a resolution node, a first calculation node, and a second calculation node. The first and second calculation nodes store a plurality of first source correlithm objects and a plurality of second source correlithm objects of a mapping table, respectively. The resolution node adds the n-dimensional distance determined for each of the first source correlithm objects with the n-dimensional distance determined for a corresponding one of the second source correlithm objects such that each pair of first and second source correlithm objects has an aggregate n-dimensional distance calculation. It compares these aggregate n-dimensional distance calculations to identify the pair of first and second source correlithm objects with the smallest aggregate n-dimensional distance to an input correlithm object. It identifies the target correlithm object associated with the identified pair, and outputs that target correlithm object.
    Type: Grant
    Filed: March 26, 2018
    Date of Patent: November 17, 2020
    Assignee: Bank of America Corporation
    Inventor: Patrick N. Lawrence
  • Patent number: 10810029
    Abstract: A distributed node network to emulate a correlithm object processing system, includes a resolution node, a first calculation node, and a second calculation node. The first and second calculation nodes determine the n-dimensional distances between an input correlithm object and first and second portions of each source correlithm object of a mapping table, respectively. The resolution node adds the n-dimensional distance determined for the first portion of each source correlithm object with the n-dimensional distance determined for the second portion of each source correlithm object. It compares these determined n-dimensional distances to each other to identify the source correlithm object with the smallest n-dimensional distance to the input correlithm object. It identifies the target correlithm object that corresponds to the identified source correlithm object, and outputs that target correlithm object.
    Type: Grant
    Filed: March 26, 2018
    Date of Patent: October 20, 2020
    Assignee: Bank of America Corporation
    Inventor: Patrick N. Lawrence
  • Patent number: 10810484
    Abstract: The present technical disclosure relates to artificial neural networks, e.g., gated recurrent unit (GRU). In particular, the present technical disclosure relates to how to implement a hardware accelerator for compressed GRU based on an embedded FPGA. Specifically, it proposes an overall design processing method of matrix decoding, matrix-vector multiplication, vector accumulation and activation function. In another aspect, the present technical disclosure proposes an overall hardware design to implement and accelerate the above process.
    Type: Grant
    Filed: December 27, 2016
    Date of Patent: October 20, 2020
    Assignee: XILINX, INC.
    Inventors: Dongliang Xie, Song Han, Yi Shan
  • Patent number: 10802488
    Abstract: An apparatus having components implemented on one or more solid-state chips. The apparatus includes an input device constructed to generate an input data value (input value), and a neural network implemented on solid-state chips trained to generate an output to control the apparatus by processing the input value. The apparatus also includes another neural network implemented on solid-state chips and configured to receive the output from the neural network. The another neural network is trained to determine whether the output of the neural network corresponds to a predetermined condition and generate a control output from the output of the neural network. The apparatus includes a processor configured receive the control output from the aforementioned another neural network, and in response to the control output indicating the output of the first neural network corresponds to a predetermined condition, and control an operation of the neural network. Corresponding methods are also disclosed.
    Type: Grant
    Filed: August 14, 2019
    Date of Patent: October 13, 2020
    Assignee: Apex Artificial Intelligence Industries, Inc.
    Inventor: Kenneth A. Abeloe
  • Patent number: 10802489
    Abstract: A method of operating an apparatus using a control system that includes at least one neural network. The method includes receiving an input value captured by the apparatus, processing the input value using the at least one neural network of the control system implemented on first one or more solid-state chips, and obtaining an output from the at least one neural network resulting from processing the input value. The method may also include processing the output with another neural network implemented on solid-state chips to determine whether the output breaches a predetermined condition that is unchangeable after an initial installation onto the control system. The aforementioned another neural network is prevented from being retrained. The method may also include the step of using the output from the at least one neural network to control the apparatus unless the output breaches the predetermined condition. Similar corresponding apparatuses are described.
    Type: Grant
    Filed: August 14, 2019
    Date of Patent: October 13, 2020
    Assignee: Apex Artificial Intelligence Industries, Inc.
    Inventor: Kenneth A. Abeloe
  • Patent number: 10783433
    Abstract: Method of training a neural network, including receiving sets of digital attributes representing multidimensional regression at inputs; expanding the network by adding neurons and defining their activation functions and interconnections; any neuron of the neural network is capable of being directly connected to any other neuron; (i) when a training speed falls below predefined threshold, and accuracy does not improve, identifying a neuron with highest error value; (ii) adding a neuron directly between the identified neuron and a corresponding output; (iii) setting only a connection coefficient between the added neuron and the identified neuron to zero before it is modified by the training, while other coefficients of the added neuron are set the same as coefficients of the identified neuron, before they are modified. After at least one iteration, either (iv) finishing the training of the neural network or (v) continuing to train the network to reach a predefined depth.
    Type: Grant
    Filed: April 22, 2019
    Date of Patent: September 22, 2020
    Assignee: Bell Integrator Inc.
    Inventors: Dmitry Ivanovich Proshin, Andrey Ivanovich Korobitsyn
  • Patent number: 10762420
    Abstract: Some embodiments of the invention provide an integrated circuit (IC) with a defect-tolerant neural network. The neural network has one or more redundant neurons in some embodiments. After the IC is manufactured, a defective neuron in the neural network can be detected through a test procedure and then replaced by a redundant neuron (i.e., the redundant neuron can be assigned the operation of the defective neuron). The routing fabric of the neural network can be reconfigured so that it re-routes signals around the discarded, defective neuron. In some embodiments, the reconfigured routing fabric does not provide any signal to or forward any signal from the discarded, defective neuron, and instead provides signals to and forwards signals from the redundant neuron that takes the defective neuron's position in the neural network. In some embodiments that implement a neural network by re-purposing (i.e.
    Type: Grant
    Filed: December 31, 2017
    Date of Patent: September 1, 2020
    Assignee: XCELSIS CORPORATION
    Inventors: Steven L. Teig, Kenneth Duong
  • Patent number: 10740395
    Abstract: An apparatus includes a processor to: train a first neural network of a chain to generate first configuration data including first trained parameters, wherein the chain performs an analytical function generating a set of output values from a set of input values, each neural network has inputs to receive the set of input values and outputs to output a portion of the set of output values, and the neural networks are ordered from the first at the head to a last neural network at the tail, and are interconnected so that each neural network additionally receives the outputs of a preceding neural network; train, using the first configuration data, a next neural network in the chain ordering to generate next configuration data including next trained parameters; and use at least the first and next configuration data and data indicating the interconnections to instantiate the chain to perform the analytical function.
    Type: Grant
    Filed: December 26, 2019
    Date of Patent: August 11, 2020
    Assignee: SAS INSTITUTE INC.
    Inventors: Henry Gabriel Victor Bequet, Jacques Rioux, John Alejandro Izquierdo, Huina Chen, Juan Du
  • Patent number: 10726070
    Abstract: A method, system and computer-usable medium for providing cognitive insights comprising receiving streams of data from a plurality of data sources; processing the streams of data from the plurality of data sources, the processing the streams of data from the plurality of data sources performing data enriching to provide enriched data; generating the cognitive session graph, the cognitive session graph being associated with a session, the cognitive session graph comprising at least some enriched data; and, processing the cognitive session graph to provide a cognitive insight, the cognitive insight being related to the session.
    Type: Grant
    Filed: June 3, 2015
    Date of Patent: July 28, 2020
    Assignee: Cognitive Scale, Inc.
    Inventors: Matthew Sanchez, Dilum Ranatunga
  • Patent number: 10699779
    Abstract: A neural network device having a first plurality of synapses that includes a plurality of memory cells. Each memory cell includes a floating gate over a first portion of a channel region and a first gate over a second portion of the channel region. The memory cells are arranged in rows and columns. A plurality of first lines each electrically connect together the first gates in one of the memory cell rows, a plurality of second lines each electrically connect together the source regions in one of the memory cell rows, and a plurality of third lines each electrically connect together the drain regions in one of the memory cell columns. The first plurality of synapses receives a first plurality of inputs as electrical voltages on the plurality of third lines, and provides a first plurality of outputs as electrical currents on the plurality of second lines.
    Type: Grant
    Filed: April 11, 2019
    Date of Patent: June 30, 2020
    Assignee: Silicon Storage Technology, Inc.
    Inventors: Hieu Van Tran, Steven Lemke, Vipin Tiwari, Nhan Do, Mark Reiten
  • Patent number: 10657426
    Abstract: A system and method for pruning. A neural network includes a plurality of long short-term memory cells, each of which includes an input having a weight matrix Wc, an input gate having a weight matrix Wi, a forget gate having a weight matrix Wf, and an output gate having a weight matrix Wo. In some embodiments, after initial training, one or more of the weight matrices Wi, Wf, and Wo are pruned, and the weight matrix Wc is left unchanged. The neural network is then retrained, the pruned weights being constrained to remain zero during retraining.
    Type: Grant
    Filed: March 27, 2018
    Date of Patent: May 19, 2020
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Georgios Georgiadis, Weiran Deng
  • Patent number: 10635974
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for neural programming. One of the methods includes processing a current neural network input using a core recurrent neural network to generate a neural network output; determining, from the neural network output, whether or not to end a currently invoked program and to return to a calling program from the set of programs; determining, from the neural network output, a next program to be called; determining, from the neural network output, contents of arguments to the next program to be called; receiving a representation of a current state of the environment; and generating a next neural network input from an embedding for the next program to be called and the representation of the current state of the environment.
    Type: Grant
    Filed: November 11, 2016
    Date of Patent: April 28, 2020
    Assignee: DeepMind Technologies Limited
    Inventors: Scott Ellison Reed, Joao Ferdinando Gomes de Freitas
  • Patent number: 10627820
    Abstract: Systems and methods for automatically self-correcting or correcting in real-time one or more neural networks after detecting a triggering event, or breaching boundary conditions are provided. Such a triggering event may indicate incorrect output signal or data being generated by the one or more neural networks. In particular, machine controllers of the invention limit the operations of neural networks to be within boundary conditions. Autonomous machines of the invention can be self-corrected after a breach of a boundary condition is detected. Autonomous land vehicles of the invention are capable of determining the timing of automatic transition to the manual control from automated driving mode. The controller of the invention filters and saves input-output data sets that fall within boundary conditions for later training of neural networks. The controllers of the invention include security architectures to prevent damages from virus attacks or system malfunctions.
    Type: Grant
    Filed: April 26, 2019
    Date of Patent: April 21, 2020
    Assignee: Apex Artificial Intelligence Industries, Inc.
    Inventor: Kenneth A. Abeloe
  • Patent number: 10627508
    Abstract: A radar system includes an array antenna including antenna elements that each output a reception signal in response to one or plural arriving waves, and a signal processing circuit in which a learned neural network has been established. The signal processing circuit receives the reception signals, inputs the reception signals or a secondary signal generated from the reception signals to the neural network, performs computation by using the reception signals or secondary signal and learned data of the neural network, and outputs a signal indicating the number of arriving waves from the neural network.
    Type: Grant
    Filed: August 6, 2018
    Date of Patent: April 21, 2020
    Assignees: NIDEC CORPORATION, NATIONAL UNIVERSITY CORPORATION SHIZUOKA UNIVERSIT
    Inventors: Hiroyuki Kamo, Yoshihiko Kuwahara
  • Patent number: 10606678
    Abstract: A system for handling errors in a neural network includes a neural network processor for executing a neural network associated with use of a vehicle. The neural network processor includes an error detector configured to detect a data error associated with execution of the neural network and a neural network controller configured to receive a report of the data error from the error detector. In response to receiving the report, the neural network controller is further configured to signal that a pending result of the neural network is tainted without terminating execution of the neural network.
    Type: Grant
    Filed: November 17, 2017
    Date of Patent: March 31, 2020
    Assignee: Tesla, Inc.
    Inventors: Christopher Hsiong, Emil Talpes, Debjit Das Sarma, Peter Bannon, Kevin Hurd, Benjamin Floering
  • Patent number: 10606898
    Abstract: A system includes a robot having a module that includes a function for mapping natural language commands of varying complexities to reward functions at different levels of abstraction within a hierarchical planning framework, the function including using a deep neural network language model that learns how to map the natural language commands to reward functions at an appropriate level of the hierarchical planning framework.
    Type: Grant
    Filed: April 19, 2018
    Date of Patent: March 31, 2020
    Assignee: Brown University
    Inventors: Stefanie Tellex, Dilip Arumugam, Siddharth Karamcheti, Nakul Gopalan, Lawson L. S. Wong
  • Patent number: 10599976
    Abstract: Provided are a computer program product, a learning apparatus and a learning method. The method includes calculating, by a processor, a first propagation value that is propagated from a propagation source node to a propagation destination node in a neural network including a plurality of nodes, based on node values of the propagation source node at a plurality of time points and a weight corresponding to passage of time points based on a first attenuation coefficient. The method further includes updating, by the processor, a first update parameter, which is used for updating the first attenuation coefficient, by using the first propagation value. The method also includes updating, by the processor, the first attenuation coefficient by using the first update parameter and an error of the node value of the propagation destination node.
    Type: Grant
    Filed: November 7, 2016
    Date of Patent: March 24, 2020
    Assignee: International Business Machines Corporation
    Inventor: Takayuki Osogami
  • Patent number: 10599975
    Abstract: A source system initializes, using an initialization seed, a first parameter vector representing weights of a neural network. The source system determines a second parameter vector by performing a sequence of mutations on the first parameter vector, the mutations each being based on a perturbation seed. The source system generates, and stores to memory, an encoded representation of the second parameter vector that comprises the initialization seed and a sequence of perturbation seeds corresponding to the sequence of mutations. The source system transmits the data structure to a target system, which processes a neural network based on the data structure.
    Type: Grant
    Filed: December 14, 2018
    Date of Patent: March 24, 2020
    Assignee: Uber Technologies, Inc.
    Inventors: Felipe Petroski Such, Jeffrey Michael Clune, Kenneth Owen Stanley, Edoardo Conti, Vashisht Madhavan, Joel Anthony Lehman
  • Patent number: 10528864
    Abstract: A method, computer program product, and system perform computations using a sparse convolutional neural network accelerator. A first vector comprising only non-zero weight values and first associated positions of the non-zero weight values within a 3D space is received. A second vector comprising only non-zero input activation values and second associated positions of the non-zero input activation values within a 2D space is received. The non-zero weight values are multiplied with the non-zero input activation values, within a multiplier array, to produce a third vector of products. The first associated positions are combined with the second associated positions to produce a fourth vector of positions, where each position in the fourth vector is associated with a respective product in the third vector. The products in the third vector are transmitted to adders in an accumulator array, based on the position associated with each one of the products.
    Type: Grant
    Filed: March 14, 2017
    Date of Patent: January 7, 2020
    Assignee: NVIDIA Corporation
    Inventors: William J. Dally, Angshuman Parashar, Joel Springer Emer, Stephen William Keckler, Larry Robert Dennison
  • Patent number: 10504023
    Abstract: This document generally describes a neural network training system, including one or more computers, that trains a recurrent neural network (RNN) to receive an input, e.g., an input sequence, and to generate a sequence of outputs from the input sequence. In some implementations, training can include, for each position after an initial position in a training target sequence, selecting a preceding output of the RNN to provide as input to the RNN at the position, including determining whether to select as the preceding output (i) a true output in a preceding position in the output order or (ii) a value derived from an output of the RNN for the preceding position in an output order generated in accordance with current values of the parameters of the recurrent neural network.
    Type: Grant
    Filed: June 6, 2016
    Date of Patent: December 10, 2019
    Assignee: Google LLC
    Inventors: Samy Bengio, Oriol Vinyals, Navdeep Jaitly, Noam M. Shazeer
  • Patent number: 10489665
    Abstract: Systems and method are provided for controlling a vehicle. In one embodiment, a determination is made that a traffic control person and a traffic control sign are present within the environment of the vehicle based on sensor data, such as optical camera data. The position and orientation of the traffic control sign relative to the traffic control person is determined, e.g., via lidar sensor data, and the validity of the traffic control person and the traffic control sign is determined based on the position and orientation of the traffic control sign.
    Type: Grant
    Filed: September 7, 2017
    Date of Patent: November 26, 2019
    Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
    Inventor: Clement Creusot
  • Patent number: 10482372
    Abstract: Systems and methods for an interconnection scheme for reconfigurable neuromorphic hardware are disclosed. A neuromorphic processor may include a plurality of corelets, each corelet may include a plurality of synapse arrays and a neuron array. Each synapse array may include a plurality of synapses and a synapse array router coupled to synapse outputs in a synapse array. Each synapse may include a synapse input, synapse output; and a synapse memory. A neuron array may include a plurality of neurons, each neuron may include a neuron input and a neuron output. Each synapse array router may include a first logic to route one or more of the synapse outputs to one or more of the neuron inputs.
    Type: Grant
    Filed: December 23, 2015
    Date of Patent: November 19, 2019
    Assignee: Intel Corporation
    Inventors: Gregory K. Chen, Jae-Sun Seo
  • Patent number: 10452141
    Abstract: A method of conditioning one or more actions related to an operator controllable device is disclosed. The method includes receiving one or more action-based instructions representing one or more operator controllable device actions associated with the operator controllable device, and receiving one or more action-oriented time representations representing one or more times associated with the action-based instructions. The method also includes deriving at least one conditioned action-based instruction for a time subsequent to the times represented by the received action-oriented time representations from: the received action-based instructions, and the received action-oriented time representations. The conditioned action-based instruction, when executed, causes the operator controllable device to take one or more conditioned actions.
    Type: Grant
    Filed: September 29, 2016
    Date of Patent: October 22, 2019
    Assignee: Kindred Systems Inc.
    Inventor: James Sterling Bergstra
  • Patent number: 10387366
    Abstract: A neural network unit includes first and second memories that hold rows of respective N weight and data words and provides a row of them to N corresponding neural processing units (NPU), respectively. The N NPUs each have an accumulator and an arithmetic unit that performs a series of multiply operations on pairs of weight words and data words received from the first and second memories to generate a series of products. The arithmetic unit also performs a series of addition operations on the series of products to accumulate an accumulated value in the accumulator. Activation function units (AFU) are each shared by a corresponding plurality of the N NPUs. Each AFU, in a sequential fashion with respect to each NPU of the corresponding plurality of the N NPUs, receives the accumulated value from the NPU and performs an activation function on the accumulated value to generate a result.
    Type: Grant
    Filed: April 5, 2016
    Date of Patent: August 20, 2019
    Assignee: VIA ALLIANCE SEMICONDUCTOR CO., LTD.
    Inventors: G. Glenn Henry, Terry Parks
  • Patent number: 10380503
    Abstract: Methods, apparatus, systems, and computer program products are provided for providing distributed online learning for personal predictive models that preserves user privacy. An example method comprises receiving model updates based at least in part on synchronization metadata. The model update comprises one or more received model parameter updates. The received model parameter updates correspond to one or more model parameters. The model updates are associated with a particular apparatus. The example method further comprises determining a decay factor based at least in part on the synchronization metadata; applying the decay factor to the one or more received model parameter updates; and updating the one or more model parameters based on the decayed model parameter updates.
    Type: Grant
    Filed: April 8, 2016
    Date of Patent: August 13, 2019
    Assignee: HERE Global B.V.
    Inventor: Tadej Stajner
  • Patent number: 10365836
    Abstract: An apparatus includes: an adaptive declustered RAID array configured of data storage devices (DSDs), the DSDs comprise data chunks allocated as data, a local parity, or a global parity; and a controller configured to generate a reliability indicator reflective of a reliability status of at least a portion of the adaptive declustered RAID array for reallocating the data chunks by dynamically increasing or decreasing the data chunks allocated as the local parity, the global parity, or a combination thereof.
    Type: Grant
    Filed: January 27, 2015
    Date of Patent: July 30, 2019
    Assignee: WESTERN DIGITAL TECHNOLOGIES, INC.
    Inventors: Jun Xu, Wei Xi, Grant Mackey, Sanghoon Chu, Jie Yu
  • Patent number: 10355713
    Abstract: A device configured to emulate a correlithm object logic function gate comprises a memory and a logic engine. The memory stores a logical operator truth table that includes a plurality of input logical values, a plurality of output logical values, and a plurality of logical operators. These logical values and the logical operators are represented by correlithm objects. The logic engine receives at least one input and a context input correlithm object representing one of the plurality of logical operators. The logic engine determines a portion of the truth table to apply based at least in part upon the logical operator represented by the context input correlithm object. The logic engine further determine an output of the logic function gate based at least in part upon the determined portion of the truth table to apply and the input.
    Type: Grant
    Filed: October 13, 2017
    Date of Patent: July 16, 2019
    Assignee: Bank of America Corporation
    Inventor: Patrick N. Lawrence
  • Patent number: 10346742
    Abstract: A calculation device includes an adding unit configured to add at least one new node to a network, which has multiple nodes that output results of calculations on input data are connected and which learned a feature of data belonging to a first subclass contained in a predetermined class. The calculation device includes an accepting unit configured to accept, as input data, training data belonging to a second subclass contained in the predetermined class. The calculation device includes a calculation unit configured to calculate coupling coefficients between the new node added by the adding unit and other nodes to learn a feature of the training data belonging to the second subclass based on an output result obtained when the training data accepted by the accepting unit is input to the network.
    Type: Grant
    Filed: August 21, 2014
    Date of Patent: July 9, 2019
    Assignee: YAHOO JAPAN CORPORATION
    Inventor: Ken-ichi Iso
  • Patent number: 10343279
    Abstract: The Developmental Network incorporates a Turing Machine that injects teaching instructions directly into the skull-closed network. The Developmental Network can also autonomously learn directly from the natural world without the need for a human to encode its input and output. The neural network so configured can be used as a controller for robotic and other computer control applications where the neural network is organized into plural X-Y-Z areas receiving signals from sensors and providing signals to effectors.
    Type: Grant
    Filed: July 8, 2016
    Date of Patent: July 9, 2019
    Assignee: Board of Trustees of Michigan State University
    Inventors: Juyang Weng, Zejia Zheng, Xie He
  • Patent number: 10331983
    Abstract: An artificial intelligence inference computing device contains a printed circuit board (PCB) and a number of electronic components mounted thereon. Electronic components include a wireless communication module, a controller module, a memory module, a storage module and at least one cellular neural networks (CNN) based integrated circuit (IC) configured for performing convolutional operations in a deep learning model for extracting features out of input data. Each CNN based IC includes a number of CNN processing engines operatively coupled to at least one input/output data bus. CNN processing engines are connected in a loop with a clock-skew circuit. Wireless communication module is configured for transmitting pre-trained filter coefficients of the deep learning model, input data and classification results.
    Type: Grant
    Filed: September 11, 2018
    Date of Patent: June 25, 2019
    Assignee: Gyrfalcon Technology Inc.
    Inventors: Lin Yang, Patrick Z. Dong, Charles Jin Young, Jason Z. Dong, Dan Bin Liu, Baohua Sun
  • Patent number: 10296849
    Abstract: The disclosed subject matter involves identifying clusters and segments of a population of data for use in a recommendation service. Clusters of members or items are formed, where the clusters, or partitions are close to being equal in size. Items are distributed based on similarities identified with matrix factorization. The items are formed into clusters based on the similarities and the clusters are used in training of a generalized linear mixed model treating the clusters as random-level effects. The trained model may be used in the recommendation service. Other embodiments are described and claimed.
    Type: Grant
    Filed: February 15, 2017
    Date of Patent: May 21, 2019
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Yi Li, Victor Chen, Krishnaram Kenthapadi, Ganesh Venkataraman
  • Patent number: 10288653
    Abstract: A method for identifying frequently occurring waveform patterns in time series comprises segmenting each of one or more time series into a plurality of subsequences. Further, a subsequence matrix comprising each of the plurality of subsequences is generated. Further, the subsequence matrix is processed to obtain a candidate subsequence matrix comprising a plurality of non-trivial subsequences. Further, the plurality of non-trivial subsequences is clustered into a plurality of spherical clusters of a predetermined diameter. Further, a plurality of sub-clusters for each of one or more spherical clusters is obtained based on a mean of each of the plurality of non-trivial subsequences present in the spherical cluster. Further, one or more frequent waveform clusters, depicting frequently occurring waveform patterns, are ascertained from amongst the one or more spherical clusters based on a number of non-trivial subsequences present in each of the plurality of sub-clusters of the spherical cluster.
    Type: Grant
    Filed: November 12, 2014
    Date of Patent: May 14, 2019
    Assignee: TATA CONSULTANCY SERVICES LIMITED
    Inventors: Puneet Agarwal, Gautam Shroff, Rishabh Gupta
  • Patent number: 10282657
    Abstract: A neuromorphic synapse with a resistive memory cell connected in circuitry having first and second input terminals. The input terminals respectively receive pre-neuron and post-neuron action signals, each having a read portion and a write portion, in use. The circuitry includes an output terminal for providing a synaptic output signal which is dependent on resistance of the memory cell. The circuitry is configured such that the synaptic output signal is provided at the output terminal in response to application at the first input terminal of the read portion of the pre-neuron action signal, and such that a programming signal, for programming resistance of the memory cell, is applied to the cell in response to simultaneous application of the write portions of the pre-neuron and post-neuron action signals at the first and second input terminals respectively. The synapse can be adapted for operation with identical pre-neuron and post-neuron action signals.
    Type: Grant
    Filed: September 29, 2015
    Date of Patent: May 7, 2019
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Angeliki Pantazi, Abu Sebastian, Evangelos S. Eleftheriou, Tomas Tuma
  • Patent number: 10262299
    Abstract: The disclosed subject matter involves identifying clusters and segments of a population of data for use in a recommendation service. Clusters of members or items are formed, where the clusters, or partitions are close to being equal in size, items are distributed based on similarities identified with matrix factorization. A matrix used in the matrix factorization is customized based on the recommendation type. The items are formed into clusters based on the similarities and the clusters are used in training of a generalized linear mixed model treating the clusters as random-level effects. The trained model may be used in the recommendation service. Other embodiments are described and claimed.
    Type: Grant
    Filed: February 15, 2017
    Date of Patent: April 16, 2019
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Yi Li, Victor Chen, Krishnaram Kenthapadi, Ganesh Venkataraman
  • Patent number: 10255805
    Abstract: Systems, components, and methodologies are provided for improvements in operation of automotive vehicles by enabling emulation of traffic signal operation by genertic algorithms, providing tunable solutions for efficient and safe operation.
    Type: Grant
    Filed: January 29, 2018
    Date of Patent: April 9, 2019
    Assignees: Volkswagen AG, Audi AG, Porsche AG
    Inventors: Joerg Christian Wolf, Michael Zweck, Christoph Rucker
  • Patent number: 10242311
    Abstract: A convolution engine, such as a convolution neural network, operates efficiently with respect to sparse kernels by implementing zero skipping. An input tile is loaded and accumulated sums are calculated for the input tile for non-zero coefficients by shifting the tile according to a row and column index of the coefficient in the kernel. Each coefficient is applied individually to tile and the result written to an accumulation buffer before moving to the next non-zero coefficient. A 3D or 4D convolution may be implemented in this manner with separate regions of the accumulation buffer storing accumulated sums for different indexes along one dimension. Images are completely processed and results for each image are stored in the accumulation buffer before moving to the next image.
    Type: Grant
    Filed: August 8, 2017
    Date of Patent: March 26, 2019
    Assignee: VIVANTE CORPORATION
    Inventor: Mankit Lo