Neural Network Patents (Class 706/15)
  • Patent number: 11113124
    Abstract: Systems and methods for searching datasets and classifying datasets are disclosed. For example, a system may include one or more memory units storing instructions and one or more processors configured to execute the instructions to perform operations. The operations may include receiving a test dataset from a client device and generating a test data model output using a data model, based on the test dataset. The operations may include processing test data model output by implementing an encoding method, a factorizing method, and/or a vectorizing method. The operations may include retrieving a reference data model output from a dataset index, based on a reference dataset. The operations may include generating a similarity metric based on the reference data model output and the test data model output. The operations may include classifying the test dataset based on the similarity metric and transmitting, to the client device, information comprising the classification.
    Type: Grant
    Filed: May 10, 2019
    Date of Patent: September 7, 2021
    Assignee: Capital One Services, LLC
    Inventors: Austin Walters, Jeremy Goodsitt, Galen Rafferty, Vincent Pham, Anh Truong, Kate Key, Reza Farivar, Mark Watson
  • Patent number: 11106972
    Abstract: A hardware accelerator that is efficient at performing computations related to a neural network. In one embodiment, the hardware accelerator includes a first data buffer that receives input data of a layer in the neural network and shift the input data slice by slice downstream. The hardware accelerator includes a second data buffer that receives kernel data of the layer in the neural network and shift the kernel data slice by slice downstream. The hardware accelerator includes a first input shift register that receives an input data slice from the first data buffer. The first input shift register may correspond to a two-dimensional shift register configured to shift values in the input data slice in x and y directions. The hardware accelerator includes a second input shift register that receives a kernel data slice from the second data buffer. A multiplication block performs convolution of the input and kernel data.
    Type: Grant
    Filed: March 9, 2021
    Date of Patent: August 31, 2021
    Inventors: Henry Verheyen, Jianjun Wen
  • Patent number: 11100385
    Abstract: Apparatus and method for a scalable, free running neuromorphic processor. For example, one embodiment of a neuromorphic processing apparatus comprises: a plurality of neurons; an interconnection network to communicatively couple at least a subset of the plurality of neurons; a spike controller to stochastically generate a trigger signal, the trigger signal to cause a selected neuron to perform a thresholding operation to determine whether to issue a spike signal.
    Type: Grant
    Filed: December 30, 2016
    Date of Patent: August 24, 2021
    Assignee: INTEL CORPORATION
    Inventors: Raghavan Kumar, Gregory K. Chen, Huseyin E. Sumbul, Ram K. Krishnamurthy, Phil Knag
  • Patent number: 11087205
    Abstract: A neural network that may include multiple layers of neural cells; wherein a certain neural cell of a certain layer of neural cells may include a first plurality of one-bit inputs; an adder and leaky integrator unit; and an activation function circuit that has a one-bit output; wherein the first plurality of one-bit inputs are coupled to a first plurality of one-bit outputs of neural cells of a layer that precedes the certain layer; wherein the adder and leaky integration unit is configured to calculate a leaky integral of a weighted sum of a number of one-bit pulses that were received, during a time window, by the first plurality of one-bit inputs; and wherein the activation function circuit is configured to apply an activation function on the leaky integral to provide a one-bit output of the certain neural cell.
    Type: Grant
    Filed: January 23, 2018
    Date of Patent: August 10, 2021
    Assignee: DSP GROUP LTD.
    Inventor: Moshe Haiut
  • Patent number: 11086678
    Abstract: There is provided an information processing device capable of intuitively adding a hardware resource intended to execute the learning, the information processing device including: a display control unit configured to control display of information indicating progress of a learning process and an addition button used to add dynamically a second hardware resource intended to execute the learning process to a first hardware resource on which the learning process is being executed.
    Type: Grant
    Filed: December 13, 2016
    Date of Patent: August 10, 2021
    Assignee: SONY CORPORATION
    Inventors: Yoshiyuki Kobayashi, Akira Fukui, Naoki Ide, Yukio Oobuchi, Shingo Takamatsu
  • Patent number: 11082159
    Abstract: A transmitter apparatus and a receiver apparatus are provided. The transmitter apparatus includes: an encoder configured to generate a low density parity check (LDPC) by performing LDPC encoding; an interleaver configured to interleave the LDPC codeword; and a modulator configured to map the interleaved LDPC codeword onto a modulation symbol. The modulator maps a bit included in a predetermined group from among a plurality of groups constituting the LDPC codeword onto a predetermined bit in the modulation symbol.
    Type: Grant
    Filed: May 29, 2019
    Date of Patent: August 3, 2021
    Assignee: SAMSUNG ELECTRONICS CO., LTD.
    Inventors: Hong-sil Jeong, Se-ho Myung, Kyung-joong Kim
  • Patent number: 11080592
    Abstract: A neuromorphic architecture for a spiking neural network comprising a plurality of spiking neurons, each with a plurality of synapses and corresponding synaptic weights, the architecture further comprising a synaptic competition mechanism in connection with a spike-based learning mechanism based on spikes perceived behind a synapse, in which architecture synapses of different neurons connected to the same input compete for that input and based on the result of that competition, each neuron of the neural network develops an individual perception of the presented input spikes, the perception used by the learning mechanism to adjust the synaptic weights.
    Type: Grant
    Filed: October 5, 2017
    Date of Patent: August 3, 2021
    Assignee: International Business Machines Corporation
    Inventors: Stanislaw A. Wozniak, Angeliki Pantazi
  • Patent number: 11074499
    Abstract: Artificial neural networks (ANNs) are a distributed computing model in which computation is accomplished with many simple processing units, called neurons, with data embodied by the connections between neurons, called synapses, and by the strength of these connections, the synaptic weights. An attractive implementation of ANNs uses the conductance of non-volatile memory (NVM) elements to record the synaptic weight, with the important multiply—accumulate step performed in place, at the data. In this application, the non-idealities in the response of the NVM such as nonlinearity, saturation, stochasticity and asymmetry in response to programming pulses lead to reduced network performance compared to an ideal network implementation.
    Type: Grant
    Filed: November 20, 2017
    Date of Patent: July 27, 2021
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventor: Geoffrey W Burr
  • Patent number: 11055584
    Abstract: An image processing apparatus comprising: a learning unit configured to perform learning of a discriminator based on an image feature amount in a first image and a class of the first image defined by a first granularity; an evaluation unit configured to evaluate an image feature amount in a second image whose class is known by the discriminator after the learning; and a control unit configured to control the learning unit to, in a case in which the evaluation by the evaluation unit is that a predetermined criterion is not satisfied, perform the learning of the discriminator based on the image feature amount in the first image and a class of the first image defined by a second granularity coarser than the first granularity.
    Type: Grant
    Filed: July 6, 2018
    Date of Patent: July 6, 2021
    Assignee: CANON KABUSHIKI KAISHA
    Inventors: Masato Aoba, Yasuhiro Komori
  • Patent number: 11057056
    Abstract: A transmitting apparatus is provided. The transmitting apparatus includes: an encoder configured to generate a Low Density Parity Check (LDPC) codeword by LDPC encoding based on a parity check matrix; an interleaver configured to interleave the LDPC codeword; and a modulator configured to map the interleaved LDPC codeword onto a plurality of modulation symbols, wherein the modulator is configured to map bits included in a predetermined bit group from among a plurality of bit groups constituting the LDPC codeword onto a predetermined bit of each of the modulation symbols.
    Type: Grant
    Filed: June 14, 2019
    Date of Patent: July 6, 2021
    Assignee: SAMSUNG ELECTRONICS CO., LTD.
    Inventors: Hong-sil Jeong, Kyung-joong Kim, Se-ho Myung
  • Patent number: 11057057
    Abstract: A transmitting apparatus is provided. The transmitting apparatus includes: an encoder configured to generate a low-density parity check (LDPC) codeword by LDPC encoding of input bits based on a parity check matrix including information word bits and parity bits, the LDPC codeword including a plurality of bit groups each including a plurality of bits; an interleaver configured to interleave the LDPC codeword; and a modulator configured to map the interleaved LDPC codeword onto a modulation symbol, wherein the interleaver is further configured to interleave the LDPC codeword such that a bit included in a predetermined bit group from among the plurality of bit groups constituting the LDPC codeword onto a predetermined bit of the modulation symbol.
    Type: Grant
    Filed: June 20, 2019
    Date of Patent: July 6, 2021
    Assignee: SAMSUNG ELECTRONICS CO., LTD.
    Inventors: Kyung-joong Kim, Se-ho Myung, Hong-sil Jeong, Daniel Ansorregui Lobete, Belkacem Mouhouche
  • Patent number: 11057050
    Abstract: A transmitting apparatus is provided. The transmitting apparatus includes: an encoder configured to generate a low-density parity check (LDPC) codeword by LDPC encoding based on a parity check matrix; an interleaver configured to interleave the LDPC codeword; and a modulator configured to map the interleaved LDPC codeword onto a modulation symbol, wherein the modulator is further configured to map a bit included in a predetermined bit group from among a plurality of bit groups constituting the LDPC codeword onto a predetermined bit of the modulation symbol.
    Type: Grant
    Filed: June 14, 2019
    Date of Patent: July 6, 2021
    Assignee: SAMSUNG ELECTRONICS CO., LTD.
    Inventors: Se-ho Myung, Hong-sil Jeong, Kyung-joong Kim
  • Patent number: 11055621
    Abstract: Methods and apparatus for determining whether a media presentation device is in an on state or an off state are disclosed. A disclosed example method comprises determining contribution values from at least one of a signal measured from a sensing device or an output signal accessed from the presentation device, wherein the contribution values are indicative of a state of a presentation device. Summing, via a logic circuit, a first plurality of the contribution values corresponding to a first measurement cycle to generate a first intermediate fuzzy score for the first measurement cycle. Storing the first intermediate fuzzy score in a buffer including a plurality of intermediate fuzzy scores corresponding to respective measurement cycles. Combining, via the logic circuit, the intermediate fuzzy scores corresponding to a first time period to form a final fuzzy score.
    Type: Grant
    Filed: January 3, 2020
    Date of Patent: July 6, 2021
    Assignee: The Nielsen Company (US), LLC
    Inventors: Daniel J. Nelson, Brian Scott Mello, Luc Zio, David James Croy
  • Patent number: 11044519
    Abstract: A system for generating, providing and/or receiving an encapsulated service guide data.
    Type: Grant
    Filed: March 18, 2020
    Date of Patent: June 22, 2021
    Assignee: SHARP KABUSHIKI KAISHA
    Inventor: Sachin G. Deshpande
  • Patent number: 11017780
    Abstract: Methods and systems for classifying a multimedia file using interclass data is disclosed. One of the methods can use classification results from one or more engines of different classes to select a different engine for the original classification task. For example, given an audio segment with associated metadata and image data, the disclosed interclass method can use the classification results from a topic classification of metadata and/or an image classification result of the image data as inputs for selecting a new transcription engine to transcribe the audio segment.
    Type: Grant
    Filed: February 27, 2019
    Date of Patent: May 25, 2021
    Assignee: VERITONE, INC.
    Inventors: Chad Steelberg, Peter Nguyen, David Kettler, Karl Schwamb, Yu Zhao
  • Patent number: 11010690
    Abstract: A method, system and computer product for performing storage maintenance is described. A training set for storage volume reclamation is received. The training set for storage volume reclamation contains sets of storage parameters for storage volumes and corresponding user decisions whether the storage volumes are reclaimable. The training set is used to train a machine learning system to recognize reclaimable candidate storage volumes. The trained machine learning system is used to determine that a candidate storage volume for reclamation is likely a reclaimable storage volume.
    Type: Grant
    Filed: June 23, 2019
    Date of Patent: May 18, 2021
    Assignee: International Business Machines Corporation
    Inventors: John A Bowers, Andrew J Laforteza, Ryan D Mcnair, Benjamin J Randall, Teresa S Swingler
  • Patent number: 11007650
    Abstract: A robot procurement apparatus stores a deficient or surplus number of robots in each of service areas where robots provide services and performs a procurement process of determining a procurement target robot to a dispatch destination service area that is the service area to which the robot is to be dispatched based on the deficient or surplus number of robots in the dispatch destination service area and the deficient or surplus number of robots in any service area other than the dispatch destination service area. The robot procurement apparatus sets the deficient or surplus number of robots in each of the service areas based on load of the robot in the service area.
    Type: Grant
    Filed: March 23, 2018
    Date of Patent: May 18, 2021
    Assignee: HITACHI, LTD.
    Inventors: Jingze Dai, Hideya Yoshiuchi
  • Patent number: 11003776
    Abstract: Improved systems and methods for automated machine-learning, zero-day malware detection. Embodiments include a system and method for detecting malware using multi-stage file-typing and, optionally pre-processing, with fall-through options.
    Type: Grant
    Filed: May 26, 2017
    Date of Patent: May 11, 2021
    Assignee: BluVector, Inc.
    Inventors: Scott Miserendino, Ryan Peters, Donald Steiner
  • Patent number: 11004459
    Abstract: Examples described herein involve detecting known environmental conditions of using a neural network. An example implementation involves a playback device receiving data indicating a response of a listening environment to audio output of one or more playback devices as captured by a microphone and determining an input vector for a neural network. The playback device provides the determined input vector to the neural network, which includes an output layer comprising neurons that correspond to respective environmental conditions. The playback device detects that the input vector caused one or more neurons of the neural network to fire such that the neural network indicates that one or more particular environmental conditions are present in the listening environment. The playback device adjusts audio output of the one or more playback devices to at least partially offset the one or more particular environmental conditions.
    Type: Grant
    Filed: September 13, 2019
    Date of Patent: May 11, 2021
    Assignee: Sonos, Inc.
    Inventors: Klaus Hartung, Greg Bright
  • Patent number: 11003894
    Abstract: [Object] To provide an information processing system, a storage medium, and an information processing method that can make a response to a user on the basis of an episode constructed from an interaction with the user to enhance the user's memory.
    Type: Grant
    Filed: March 15, 2017
    Date of Patent: May 11, 2021
    Assignee: Sony Corporation
    Inventor: Masamichi Asukai
  • Patent number: 10990876
    Abstract: Graphical elements in a user interface (UI) may be detected in robotic process automation (RPA) using convolutional neural networks (CNNs). Such processes may be particularly well-suited for detecting graphical elements that are too small to be detected using conventional techniques. The accuracy of detecting graphical elements (e.g., control objects) may be enhanced by providing neural network-based processing that is robust to changes in various UI factors, such as different resolutions, different operating system (OS) scaling factors, different dots-per-inch (DPI) settings, and changes due to UI customization of applications and websites, for example.
    Type: Grant
    Filed: October 8, 2019
    Date of Patent: April 27, 2021
    Assignee: UiPath, Inc.
    Inventors: Mircea Neagovici, Stefan Adam, Virgil Tudor, Dragos Bobolea
  • Patent number: 10991466
    Abstract: An apparatus includes a processor and storage to store instructions that cause the processor to identify at least one correlation between a diagnosis group and a medication class for each patient of a first set of patients to derive a set of models for each diagnosis group that correlates the diagnosis group to at least one medication class based on the at least one identified correlation; and for each patient of a second set of patients, employ each model of each set of models to make at least one prediction of at least one diagnosis group as indicated in the corresponding diagnosis group record based on at least one medication class indicated in the corresponding medication class record, and compare the at least one prediction to the corresponding diagnosis group record to derive a tally of at least one of true positives or false positives for each prediction.
    Type: Grant
    Filed: May 3, 2016
    Date of Patent: April 27, 2021
    Assignee: SAS INSTITUTE INC.
    Inventors: Emily Chapman-McQuiston, Diane Emerton, Ruth Baldasaro, Daniel Kelly
  • Patent number: 10992675
    Abstract: Systems, methods, and other embodiments associated with anomaly detection using tripoint arbitration are described. In one embodiment, a method includes identifying a set of clusters that correspond to a nominal sample of data points in a sample space. A point z is determined to be an anomaly with respect to the nominal sample when, for each cluster, a tripoint arbitration similarity between data points in the cluster calculated with z as arbiter is greater than a threshold.
    Type: Grant
    Filed: December 15, 2017
    Date of Patent: April 27, 2021
    Assignee: Oracle International Corporation
    Inventors: Aleksey Urmanov, Anton Bougaev
  • Patent number: 10984314
    Abstract: Described is a system for selecting among intelligence elements of a neural model. An intelligence element is selected from a set of intelligence elements which change group attack probability estimates and processed via multiple operations. A semantic memory component learns group probability distributions and rules based on the group probability distributions. The rules determine which intelligence element related to the groups to select. Given an environment of new probability distributions, the semantic memory component recalls which rule to select to receive a particular intelligence element. An episodic memory component recalls a utility value for each information element A procedural memory component recalls and selects the information element considered to have the highest utility. A list of intelligence elements is published to disambiguate likely attackers.
    Type: Grant
    Filed: June 25, 2015
    Date of Patent: April 20, 2021
    Assignee: HRL Laboratories, LLC
    Inventors: Suhas E. Chelian, Giorgio A. Ascoli, James Benvenuto, Michael D. Howard, Rajan Bhattacharyya
  • Patent number: 10983922
    Abstract: Provided are a computer program product, system, and method for using a machine learning module to select one of multiple cache eviction algorithms to use to evict a track from the cache. A first cache eviction algorithm determines tracks to evict from the cache. A second cache eviction algorithm determines tracks to evict from the cache, wherein the first and second cache eviction algorithms use different eviction schemes. At least one machine learning module is executed to produce output indicating one of the first cache eviction algorithm and the second cache eviction algorithm to use to select a track to evict from the cache. A track is evicted that is selected by one of the first and second cache eviction algorithms indicated in the output from the at least one machine learning module.
    Type: Grant
    Filed: May 18, 2018
    Date of Patent: April 20, 2021
    Assignee: International Business Machines Corporation
    Inventors: Lokesh M. Gupta, Matthew G. Borlick, Kyler A. Anderson, Kevin J. Ash
  • Patent number: 10984334
    Abstract: A device may receive training spectral data associated with a manufacturing process that transitions from an unsteady state to a steady state. The device may generate, based on the training spectral data, a plurality of iterations of a support vector machine (SVM) classification model. The device may determine, based on the plurality of iterations of the SVM classification model, a plurality of predicted transition times associated with the manufacturing process. A predicted transition time, of the plurality of predicted transition times, may identify a time, during the manufacturing process, that a corresponding iteration of the SVM classification model predicts that the manufacturing process transitioned from the unsteady state to the steady state. The device may generate, based on the plurality of predicted transition times, a final SVM classification model associated with determining whether the manufacturing process has reached the steady state.
    Type: Grant
    Filed: May 4, 2017
    Date of Patent: April 20, 2021
    Assignee: VIAVI Solutions Inc.
    Inventors: Changmeng Hsiung, Peng Zou, Lan Sun
  • Patent number: 10984320
    Abstract: A computer-based method includes receiving an input signal at a neuron in a computer-based neural network that includes a plurality of neuron layers, applying a first non-linear transform to the input signal at the neuron to produce a plain signal, and calculating a weighted sum of a first component of the input signal and the plain signal at the neuron. In a typical implementation, the first non-linear transform is a function of the first component of the input signal and at least a second component of the input signal.
    Type: Grant
    Filed: May 1, 2017
    Date of Patent: April 20, 2021
    Assignee: Nnaisense SA
    Inventors: Rupesh Kumar Srivastava, Klaus Greff
  • Patent number: 10977581
    Abstract: A classifier is computed as follows. For a first set of values of primary field(s) of primary data instances of a labeled primary training dataset, a second set(s) of secondary fields of unclassified second data instances of secondary dataset(s) is identified. First set of values are matched to corresponding values in respective secondary field(s), and linked to other secondary fields of respective secondary data instance(s) of the respective matched secondary field. A set of classification features is generated. Each including: (i) condition(s), and (ii) a value selected from the linked other secondary fields of the respective secondary data instance(s) of the respective matched secondary field(s). The respective classification feature outputs a binary value computed by the condition(s) that compares between the value selected from the other linked secondary fields and a new received data instance. A classifier is computed according to a selected subset of the classification features.
    Type: Grant
    Filed: August 10, 2020
    Date of Patent: April 13, 2021
    Assignee: SparkBeyond Ltd.
    Inventors: Meir Maor, Ron Karidi, Sagie Davidovich, Amir Ronen
  • Patent number: 10970630
    Abstract: Various technologies pertaining to allocating computing resources of a neuromorphic computing system are described herein. Subgraphs of a neural algorithm graph to be executed by the neuromorphic computing system are identified. The subgraphs are each executed by a group of neuron circuits serially. Output data generated by execution of the subgraphs are provided to the same or a second group of neuron circuits at a same time or with associated timing data indicative of a time at which the output data was generated. The same or second group of neuron circuits performs one or more processing operations based upon the output data.
    Type: Grant
    Filed: June 15, 2017
    Date of Patent: April 6, 2021
    Assignees: National Technology & Engineering Solutions of Sandia, LLC, Lewis Rhodes Labs, Inc.
    Inventors: James Bradley Aimone, John H. Naegle, Jonathon W. Donaldson, David Follett, Pamela Follett
  • Patent number: 10970652
    Abstract: A method for determining a transfer apparatus based on user preferences and at least a transfer apparatus archive includes receiving, by a computer device, at least a transfer invocation and user preferences, generating for each candidate transfer apparatus, performance prognoses corresponding to the user preferences, wherein generating each performance prognoses comprises receiving a candidate transfer apparatus archive, training, as a function of the candidate transfer apparatus performance archive and a supervised machine-learning process, a candidate transfer apparatus model, generating performance prognoses as a function of the candidate transfer apparatus model and the at least a transfer invocation, selecting a candidate transfer apparatus as a function of the user preferences, generating an objective function of the user preferences, wherein the objective function outputs a ranking of performance prognoses and selecting a candidate transfer apparatus which maximizes the ranking, and providing the sele
    Type: Grant
    Filed: October 16, 2020
    Date of Patent: April 6, 2021
    Assignee: Hammel Companies, Inc.
    Inventor: Joseph Charles Dohrn
  • Patent number: 10970623
    Abstract: A reinforcement learning processor specifically configured to train reinforcement learning agents in the AI systems by the way of implementing an application-specific instruction set is disclosed. The application-specific instruction set incorporates ‘Single Instruction Multiple Agents (SIMA)’ instructions. SIMA type instructions are specifically designed to be implemented simultaneously on a plurality of reinforcement learning agents which interact with corresponding reinforcement learning environments. The SIMA type instructions are specifically configured to receive either a reinforcement learning agent ID or a reinforcement learning environment ID as the operand. The reinforcement learning processor is designed for parallelism in reinforcement learning operations. The reinforcement learning processor executing of a plurality of threads associated with an operation or task in parallel.
    Type: Grant
    Filed: October 17, 2017
    Date of Patent: April 6, 2021
    Assignee: Alphaics Corporation
    Inventor: Nagendra Nagaraja
  • Patent number: 10963916
    Abstract: Systems and methods are disclosed for dynamically analyzing and providing the quality of one or more content items at the time, or substantially close to the time, they are received by a data processing system. The systems and methods described herein can maintain and update the quality score for improving previously created content items after they have been published. The one or more content items can include one or more assets (e.g., one or more headlines, one or more descriptions, images, video, etc.). The data processing system can use numerical analysis methods to determine an overall quality (e.g., estimated clicks) of the content items received by the data processing system using a trained model.
    Type: Grant
    Filed: September 16, 2019
    Date of Patent: March 30, 2021
    Assignee: Google LLC
    Inventors: Sylvanus Garnet Bent, III, Prahlad Fogla, Jamie Nicole Powell, Shu Niu, Nam Hoang Mai, Tristan Dennen, Sean Burroughs Johnston, Siva Kumar Gorantla, Suzanna Whiteside Shwert, Maxwell Schram, Weikun Liang
  • Patent number: 10963786
    Abstract: Disclosed is a neural network enabled interface server and blockchain interface establishing a blockchain network implementing event detection, tracking and management for rule based compliance, with significant implications for anomaly detection, resolution and safety and compliance reporting.
    Type: Grant
    Filed: January 10, 2020
    Date of Patent: March 30, 2021
    Assignee: LedgerDomain Inc.
    Inventors: Benjamin James Taylor, Victor Bovee Dods, Leonid Alekseyev
  • Patent number: 10963777
    Abstract: A method for implementing a convolutional neural network (CNN) accelerator on a target includes utilizing one or more processing elements to implement a standard convolution layer. A configuration of the CNN accelerator is modified to change a data flow between components on the CNN accelerator. The one or more processing elements is utilized to implement a fully connected layer in response to the change in the data flow.
    Type: Grant
    Filed: February 27, 2020
    Date of Patent: March 30, 2021
    Assignee: Altera Corporation
    Inventors: Utku Aydonat, Gordon Raymond Chiu, Andrew Chaang Ling
  • Patent number: 10956819
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an output sequence from an input sequence. In one aspect, one of the systems includes an encoder neural network configured to receive the input sequence and generate encoded representations of the network inputs, the encoder neural network comprising a sequence of one or more encoder subnetworks, each encoder subnetwork configured to receive a respective encoder subnetwork input for each of the input positions and to generate a respective subnetwork output for each of the input positions, and each encoder subnetwork comprising: an encoder self-attention sub-layer that is configured to receive the subnetwork input for each of the input positions and, for each particular input position in the input order: apply an attention mechanism over the encoder subnetwork inputs using one or more queries derived from the encoder subnetwork input at the particular input position.
    Type: Grant
    Filed: August 7, 2020
    Date of Patent: March 23, 2021
    Assignee: Google LLC
    Inventors: Noam M. Shazeer, Aidan Nicholas Gomez, Lukasz Mieczyslaw Kaiser, Jakob D. Uszkoreit, Llion Owen Jones, Niki J. Parmar, Illia Polosukhin, Ashish Teku Vaswani
  • Patent number: 10956811
    Abstract: System and techniques for variable epoch spike train filtering are described herein. A spike trace storage may be initiated for an epoch. Here, the spike trace storage is included in a neural unit of neuromorphic hardware. Multiple spikes may be received at the neural unit during the epoch. The spike trace storage may be incremented for each of the multiple spikes to produce a count of received spikes. An epoch learning event may be obtained and a spike trace may be produced in response to the epoch learning event using the count of received spikes in the spike trace storage. Network parameters of the neural unit may be modified using the spike trace.
    Type: Grant
    Filed: July 31, 2017
    Date of Patent: March 23, 2021
    Assignee: Intel Corporation
    Inventors: Michael I. Davies, Tsung-Han Lin
  • Patent number: 10949747
    Abstract: A computer trains a neural network model. (A) Observation vectors are randomly selected from a plurality of observation vectors. (B) A forward and backward propagation of a neural network is executed to compute a gradient vector and a weight vector. (C) A search direction vector is computed. (D) A step size value is computed. (E) An updated weight vector is computed. (F) Based on a predefined progress check frequency value, second observation vectors are randomly selected, a progress check objective function value is computed given the weight vector, the step size value, the search direction vector, and the second observation vectors, and based on an accuracy test, the mini-batch size value is updated. (G) (A) to (F) are repeated until a convergence parameter value indicates training of the neural network is complete. The weight vector for a next iteration is the computed updated weight vector.
    Type: Grant
    Filed: November 17, 2020
    Date of Patent: March 16, 2021
    Assignee: SAS INSTITUTE INC.
    Inventors: Majid Jahani, Joshua David Griffin, Seyedalireza Yektamaram, Wenwen Zhou
  • Patent number: 10943182
    Abstract: A machine learning process is performed using one or more sources of information for enhanced oil recovery (EOR) materials to be used for an EOR process on a defined oil reservoir. Performance of the machine learning process produces an output comprising an indication of one or more EOR materials and their corresponding concentrations to be used in the EOR process. The indication of the one or more EOR materials and their corresponding concentrations is output to be used in the EOR process. Methods, apparatus, and computer program products are disclosed.
    Type: Grant
    Filed: March 27, 2017
    Date of Patent: March 9, 2021
    Assignee: International Business Machines Corporation
    Inventors: Peter W. Bryant, Rodrigo Neumann Barros Ferreira, Ronaldo Giro, Mathias B. Steiner
  • Patent number: 10936969
    Abstract: In general, certain embodiments of the present disclosure provide methods and systems for enabling a reproducible processing of machine learning models and scalable deployment on a distributed network. The method comprises building a machine learning model; training the machine learning model to produce a plurality of versions of the machine learning model; tracking the plurality of versions of the machine learning model to produce a change facilitator tool; sharing the change facilitator tool to one or more devices such that each device can reproduce the plurality of versions of the machine learning model; and generating a deployable version of the machine learning model through repeated training.
    Type: Grant
    Filed: September 26, 2017
    Date of Patent: March 2, 2021
    Inventor: Shabaz Basheer Patel
  • Patent number: 10929755
    Abstract: The present disclosure provides a method and a device for optimization processing of neural network models. The method includes the following: determining one or more target layers of the neural network model based on the number of neurons at each layer of the neural network model; for each of the one or more target layers, adding a virtual layer between the target layer and a preceding layer of the target layer, where neurons at the virtual layer are separately connected to neurons at the target layer and neurons at the preceding layer of the target layer, and addition of the virtual layer reduces the number of connections between the target layer and the preceding layer of the target layer; and training the neural network model after having added the virtual layers, to obtain an optimized neural network model.
    Type: Grant
    Filed: April 16, 2020
    Date of Patent: February 23, 2021
    Assignee: Advanced New Technologies Co., Ltd.
    Inventor: Jianbin Lin
  • Patent number: 10922627
    Abstract: Described herein is a system that transmits and combines local models, that individually comprise a set of local parameters computed via stochastic gradient descent (SGD), into a global model that comprises a set of global model parameters. The local models are computed in parallel at different geographic locations along with symbolic representations. Network transmission of the local models and the symbolic representations, rather than transmission of the large training data subsets processed to compute the local models and symbolic representations, conserves resources and decreases latency. The global model can then be used as a model to determine a likelihood of a course of action being successful for an organization. For example, the course of action can be a purchase of a security or a business operation strategy. In another example, the course of action can be a type of medical treatment for a patient.
    Type: Grant
    Filed: June 15, 2017
    Date of Patent: February 16, 2021
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Madanlal S. Musuvathi, Todd D. Mytkowicz, Saeed Maleki, Yufei Ding
  • Patent number: 10922271
    Abstract: A method of clustering files, comprises, by a processing unit: obtaining a clustering structure comprising a plurality of nodes arranged in hierarchical levels Li, with i from 1 to N, obtaining at least one data (Dsignal) representative of a file (Dfile) to be assigned to a category; (O1) comparing said data to each centroid of each node of the first level, (O2) if said comparison matches an acceptance threshold of one or more nodes, selecting a node among these nodes, (O3) comparing Dsignal to each centroid of each node of a next level which is linked to said selected node, (O4) if said comparison matches an acceptance threshold of one or more nodes, selecting a node among these nodes, repeating O3 and O4 until a stopping condition is met, thereby indicating that Dsignal or Draw belongs to a category of files represented by said selected node.
    Type: Grant
    Filed: October 8, 2018
    Date of Patent: February 16, 2021
    Assignee: MINEREYE LTD.
    Inventors: Avner Atias, Yaniv Avidan
  • Patent number: 10922498
    Abstract: The present application provides a method for simultaneously translating the language of a smart in-vehicle system and the related product, wherein the method comprises the steps of: a smart in-vehicle device receiving the first language to be played; the smart in-vehicle device acquiring the first voice of a navigation software, wherein the first voice is the second language, and constructing the input data at the current time t of a cyclic neural network according to the first voice; and inputting the input data to the preset the cyclic neural network for calculation to obtain output result, the second voice corresponding to the first language is obtained according to the output result, and the second voice is played.
    Type: Grant
    Filed: March 15, 2019
    Date of Patent: February 16, 2021
    Assignee: WING TAK LEE SILICONE RUBBER TECHNOLOGY (SHENZHEN) CO., LTD
    Inventor: Tak Nam Liu
  • Patent number: 10911821
    Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing a plurality of recurrent neural networks to generate media consumption predictions and providing media content to a target audience. For example, the disclosed system can train a plurality of long short-term memory neural networks for a plurality of users based on historical media consumption data over a plurality of time periods. In one or more embodiments, the disclosed system identifies a target audience including a subset of users and the corresponding neural networks. The disclosed system can then utilize the neural networks of the subset of users to generate a plurality of predictions for a future time period for the users. In some embodiments, the disclosed system then combines the predictions for the users to generate a media consumption prediction for the target audience for the future time period.
    Type: Grant
    Filed: September 27, 2018
    Date of Patent: February 2, 2021
    Assignee: ADOBE INC.
    Inventors: Jason Lopatecki, Julie Lee
  • Patent number: 10909445
    Abstract: A computer-implemented real-time visualization method, system, and computer program product including determining a current sentiment and a current state of a user from user data, creating at least one layer including at least one of an image and an animation based on at least one of an aggregation and a combination of the current sentiment and the current state of the user, and compiling the at least one layer into a single image or a single animation for display on an image display medium.
    Type: Grant
    Filed: February 27, 2019
    Date of Patent: February 2, 2021
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventor: Effendi Leobandung
  • Patent number: 10902534
    Abstract: Data sources containing a plurality of health information is combined throughout a plurality of travelers. Profile vectors are created from each individual member of the traveling party and a group profile vector is built as a representative whole of the traveling parties interests. Potential travel destinations vectors are clustered and mapped with dimensions comprised in the group profile vector. A recommended vacation or travel destination itinerary is proposed based on the highest overlaying dimensional score between the group profile vector and potential travel destinations.
    Type: Grant
    Filed: March 1, 2018
    Date of Patent: January 26, 2021
    Assignee: International Business Machines Corporation
    Inventors: Shubhadip Ray, Andrew S. Christiansen, Norbert Herman, Avik Sanyal
  • Patent number: 10896380
    Abstract: A system predicts user intent to take an action and delivers content items to the user that match that intent. A plurality of features or attributes for each tracking pixel in a set of tracking pixels can be acquired based on content items and landing pages associated with each tracking pixel. For example, features for a tracking pixel can be determined based on information associated with a content item that enabled a user to access a landing page from which the tracking pixel was fired or triggered. In this example, features for the tracking pixel can also be determined based on information associated with the landing page. The features for the tracking pixels can be utilized to train a machine learning model. The machine learning model can be trained to predict whether or not a particular user intends to produce a conversion (e.g., make a purchase).
    Type: Grant
    Filed: August 30, 2017
    Date of Patent: January 19, 2021
    Assignee: Facebook, Inc.
    Inventors: Christian Alexander Martine, Robert Oliver Burns Zeldin, Dinkar Jain, Jurgen Anne Francois Marie Van Gael, Anand Sumatilal Bhalgat, Tianshi Gao
  • Patent number: 10891352
    Abstract: Aggregate vectors corresponding to non-textual information/data are provided in a multi-dimensional space. A computing entity access a plurality of instances of medical information comprising medical codes. The computing entity generates one or more medical sentences from the plurality of instances of medical information. Each medical sentence comprises one or more medical codes. The computing entity generates an embedding vector dictionary comprising a plurality of multi-dimensional vectors based on a medical embedding model trained using machine learning and the one or more medical sentences. Each multi-dimensional vector corresponds to a medical code. The computing entity generates a plurality of aggregate vectors based on the embedding vector dictionary and analyzes at least a portion of the plurality of aggregate vectors to identify two or more aggregate vectors that are similar or different based on a distance between the two or more aggregate vectors in the multi-dimensional space.
    Type: Grant
    Filed: March 21, 2018
    Date of Patent: January 12, 2021
    Assignee: Optum, Inc.
    Inventors: Christopher A. Hane, Alexander Kravetz
  • Patent number: 10893068
    Abstract: A computerized system and method to detect ransomware cyber-attacks is described. The approach entails analyzing the features associated with a file access event by a process operating on a computing device, to ascertain whether the process is associated with a ransomware cyber-attack.
    Type: Grant
    Filed: June 29, 2018
    Date of Patent: January 12, 2021
    Assignee: FireEye, Inc.
    Inventors: Yasir Khalid, Nadeem Shahbaz, Raghunath Konda
  • Patent number: 10884403
    Abstract: A monitoring system for a heating, ventilation, and air conditioning (HVAC) system of a building receives, from a monitoring device located at the building, a frequency domain representation and a time domain current value that are based on an aggregate current supplied to a plurality of components of an indoor air handler of the HVAC system. The monitoring system assesses, based on the received frequency domain representation and time domain current value, whether a first fault has occurred in a first component of the plurality of components of the indoor air handler and whether a second fault has occurred in a second component of the plurality of components of the indoor air handler. The monitoring system generates and transmits an alert in response to assessing occurrence of at least one of the first fault and the second fault. The monitoring system is located remotely from the building.
    Type: Grant
    Filed: March 18, 2019
    Date of Patent: January 5, 2021
    Assignee: EMERSON ELECTRIC CO.
    Inventor: Jeffrey N. Arensmeier