Patents Issued in February 14, 2023
  • Patent number: 11580349
    Abstract: Systems and methods allow for the integrated circuit cards (ICCs) to removably couple to each other and transmit information to an access device as a single device. One among the two or more ICCs coupled together may read data from the remaining ICCs and provide the data to an access device via contactless communication. The ICC may include a substrate; an integrated circuit embedded in the substrate; input ports exposed on a first surface of the substrate, and output ports exposed on a second surface of the substrate. The input ports and the output ports are electrically coupled to the integrated circuit. The output ports are configured to be removably coupled to the input ports of a second ICC.
    Type: Grant
    Filed: August 31, 2021
    Date of Patent: February 14, 2023
    Assignee: VISA INTERNATIONAL SERVICE ASSOCIATION
    Inventor: Ved Prakash Sajjan Kumar Agarwal
  • Patent number: 11580350
    Abstract: Systems and methods for emotionally intelligent automated chatting are provided. The systems and method provide emotionally intelligent automated (or artificial intelligence) chatting by determining a context and an emotion of a conversation with a user. Based on these determinations, the systems and methods may select one or more responses from a database of responses to a reply to a user query. Further, the systems and methods are able update or train based on user feedback and/or world feedback.
    Type: Grant
    Filed: December 21, 2016
    Date of Patent: February 14, 2023
    Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventor: Xianchao Wu
  • Patent number: 11580351
    Abstract: A technique is described herein for automatically logging journeys taken by a user, and then automatically classifying the purposes of the journeys. In one implementation, the technique obtains journey data from one or more movement-sensing devices as a user travels from a starting location to an ending location in a vehicle. The technique generates a set of features based on the journey data, and then uses a machine-trainable model (such as a neural network) to make its classification based on the features. The machine-trainable model accepts at least one feature that is based on statistical information regarding at least one aspect of prior journeys that the user has taken. Overall, the technique provides a resource-efficient solution that rapidly provides personalized results to individual respective users. In some implementations, the technique performs its personalization without sharing journey data with a remote server.
    Type: Grant
    Filed: November 22, 2018
    Date of Patent: February 14, 2023
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Justin James Wagle, Nathaniel Gunther Roth, Qian Liu, Pnina Eliyahu, Syed Farhan Raza, Timothy Edward Bellay, Rahul Anantha Padmanabha Udipi
  • Patent number: 11580352
    Abstract: A device, system, and method is provided for storing a sparse neural network. A plurality of weights of the sparse neural network may be obtained. Each weight may represent a unique connection between a pair of a plurality of artificial neurons in different layers of a plurality of neuron layers. A minority of pairs of neurons in adjacent neuron layers are connected in the sparse neural network. Each of the plurality of weights of the sparse neural network may be stored with an association to a unique index. The unique index may uniquely identify a pair of artificial neurons that have a connection represented by the weight. Only non-zero weights may be stored that represent connections between pairs of neurons (and zero weights may not be stored that represent no connections between pairs of neurons).
    Type: Grant
    Filed: July 29, 2019
    Date of Patent: February 14, 2023
    Assignee: Nano Dimension Technologies, Ltd.
    Inventors: Eli David, Eri Rubin
  • Patent number: 11580353
    Abstract: Embodiments relate to a neural engine circuit that includes an input buffer circuit, a kernel extract circuit, and a multiply-accumulator (MAC) circuit. The MAC circuit receives input data from the input buffer circuit and a kernel coefficient from the kernel extract circuit. The MAC circuit contains several multiply-add (MAD) circuits and accumulators used to perform neural networking operations on the received input data and kernel coefficients. MAD circuits are configured to support fixed-point precision (e.g., INT8) and floating-point precision (FP16) of operands. In floating-point mode, each MAD circuit multiplies the integer bits of input data and kernel coefficients and adds their exponent bits to determine a binary point for alignment. In fixed-point mode, input data and kernel coefficients are multiplied. In both operation modes, the output data is stored in an accumulator, and may be sent back as accumulated values for further multiply-add operations in subsequent processing cycles.
    Type: Grant
    Filed: May 4, 2018
    Date of Patent: February 14, 2023
    Assignee: Apple Inc.
    Inventor: Christopher L. Mills
  • Patent number: 11580354
    Abstract: An apparatus and a method use a convolutional neural network (CNN) including a plurality of convolution layers in the field of artificial intelligence (AI) systems and applications thereof. A computing apparatus using a CNN including a plurality of convolution layers includes a memory storing one or more instructions; and one or more processors configured to execute the one or more instructions stored in the memory to obtain input data; identify a filter for performing a convolution operation with respect to the input data, on one of the plurality of convolution layers; identify a plurality of sub-filters corresponding to different filtering regions within the filter; provide a plurality of feature maps based on the plurality of sub-filters; and obtain output data, based on the plurality of feature maps.
    Type: Grant
    Filed: June 6, 2019
    Date of Patent: February 14, 2023
    Assignee: SAMSUNG ELECTRONICS CO., LTD.
    Inventors: Insang Cho, Wonjae Lee, Chanyoung Hwang
  • Patent number: 11580355
    Abstract: A circuit system and a method of analyzing audio or video input data that is capable of detecting, classifying, and post-processing patterns in an input data stream. The circuit system may consist of one or more digital processors, one or more configurable spiking neural network circuits, and digital logic for the selection of two-dimensional input data. The system may use the neural network circuits for detecting and classifying patterns and one or more the digital processors to perform further detailed analyses on the input data and for signaling the result of an analysis to outputs of the system.
    Type: Grant
    Filed: October 3, 2019
    Date of Patent: February 14, 2023
    Assignee: Electronic Warfare Associates, Inc.
    Inventors: Dirk Niggemeyer, Lester A. Foster, Elizabeth M. Rudnick
  • Patent number: 11580356
    Abstract: Certain aspects of the present disclosure provide techniques for performing piecewise pointwise convolution, comprising: performing a first piecewise pointwise convolution on a first subset of data received via a first branch input at a piecewise pointwise convolution layer of a convolutional neural network (CNN) model; performing a second piecewise pointwise convolution on a second subset of data received via a second branch input at the piecewise pointwise convolution layer; determining a piecewise pointwise convolution output by summing a result of the first piecewise pointwise convolution and a result of the second piecewise pointwise convolution; and providing the piecewise pointwise convolution output to a second layer of the CNN model.
    Type: Grant
    Filed: July 17, 2020
    Date of Patent: February 14, 2023
    Assignee: Qualcomm Incorporated
    Inventors: Jamie Menjay Lin, Jin Won Lee, Jilei Hou
  • Patent number: 11580357
    Abstract: A memory for storing a directed acyclic graph (DAG) for access by an application being executed by one or more processors of a computing device is described. The DAG includes a plurality of nodes, wherein each node represents a data point within the DAG. The DAG further includes a plurality of directional edges. Each directional edge connects a pair of the nodes and represents a covering-covered relationship between two nodes. Each node comprises a subgraph consisting of the respective node and all other nodes reachable via a covering path that comprises a sequence of covering and covered nodes. Each node comprises a set of node parameters including at least an identifier and an address range. Each node and the legal address specify a cover path. Utilizing DAG Path Addressing with bindings the memory can be organized to store a generalization hierarchy of logical propositions.
    Type: Grant
    Filed: September 22, 2022
    Date of Patent: February 14, 2023
    Assignee: Practical Posets LLC
    Inventor: John W. Esch
  • Patent number: 11580358
    Abstract: The present disclosure describes improvements in optimization systems. During an optimization loop, an advanced objective function is used to determine an objective value, a specification metric, and a rule coverage metric for a particular solution. The specification metric characterizes compliance of the solution with certain formal specifications. The rule coverage metric characterizes the degree to which all rules (or a particular rule) are tested during testing of the system. The objective value and metrics may influence future operation of the optimization loop.
    Type: Grant
    Filed: May 12, 2020
    Date of Patent: February 14, 2023
    Assignee: Thales, S.A.
    Inventors: Nicholas Ernest, Timothy Arnett, Brandon Kunkel
  • Patent number: 11580359
    Abstract: The technology disclosed provides a so-called “pointer sentinel mixture architecture” for neural network sequence models that has the ability to either reproduce a token from a recent context or produce a token from a predefined vocabulary. In one implementation, a pointer sentinel-LSTM architecture achieves state of the art language modeling performance of 70.9 perplexity on the Penn Treebank dataset, while using far fewer parameters than a standard softmax LSTM.
    Type: Grant
    Filed: October 25, 2019
    Date of Patent: February 14, 2023
    Assignee: salesforce.com, inc.
    Inventors: Stephen Joseph Merity, Caiming Xiong, James Bradbury, Richard Socher
  • Patent number: 11580360
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting intermediate reinforcement learning goals. One of the methods includes obtaining a plurality of demonstration sequences, each of the demonstration sequences being a sequence of images of an environment while a respective instance of a reinforcement learning task is being performed; for each demonstration sequence, processing each image in the demonstration sequence through an image processing neural network to determine feature values for a respective set of features for the image; determining, from the demonstration sequences, a partitioning of the reinforcement learning task into a plurality of subtasks, wherein each image in each demonstration sequence is assigned to a respective subtask of the plurality of subtasks; and determining, from the feature values for the images in the demonstration sequences, a respective set of discriminative features for each of the plurality of subtasks.
    Type: Grant
    Filed: November 6, 2017
    Date of Patent: February 14, 2023
    Assignee: Google LLC
    Inventor: Pierre Sermanet
  • Patent number: 11580361
    Abstract: An apparatus to facilitate neural network (NN) training is disclosed. The apparatus includes training logic to receive one or more network constraints and train the NN by automatically determining a best network layout and parameters based on the network constraints.
    Type: Grant
    Filed: April 24, 2017
    Date of Patent: February 14, 2023
    Assignee: Intel Corporation
    Inventors: Gokcen Cilingir, Elmoustapha Ould-Ahmed-Vall, Rajkishore Barik, Kevin Nealis, Xiaoming Chen, Justin E. Gottschlich, Prasoonkumar Surti, Chandrasekaran Sakthivel, Abhishek Appu, John C. Weast, Sara S. Baghsorkhi, Barnan Das, Narayan Biswal, Stanley J. Baran, Nilesh V. Shah, Archie Sharma, Mayuresh M. Varerkar
  • Patent number: 11580362
    Abstract: According to one aspect of an embodiment a learning apparatus includes a first acquiring unit that acquires first output information that is output by an output layer when predetermined input information is input to a model that includes an input layer, a plurality of intermediate layers, and the output layer. The learning apparatus includes a second acquiring unit that acquires intermediate output information that is based on pieces of intermediate information that are output by the plurality of intermediate layers when the input information is input to the model. The learning apparatus includes a learning unit that learns the model based on the first output information and the intermediate output information.
    Type: Grant
    Filed: September 13, 2018
    Date of Patent: February 14, 2023
    Assignee: YAHOO JAPAN CORPORATION
    Inventors: Tran Dung, Kenichi Iso
  • Patent number: 11580363
    Abstract: A compatibility score generator implementing a neural network is trained for assessing compatibility of items. Elements of a feature vector representing each item and of a compatibility data structure indicating items considered compatible are retrieved. The neural network is trained using training data corresponding to the items and indicating compatibility between pairs of items. The compatibility data structure is modified by removing indications that items of a pair of items are compatible. An encoding function generating encoded representations for the items based on the compatibility data structure is evaluated. Encoded representations are provided to a decoder that learns a likelihood that the indication had been removed when modified. The neural network and the decoder are optimized based on a loss function that reflects the decoder's ability to correctly determine whether the indication had been removed. The encoded representations generate a compatibility score for at least two items of interest.
    Type: Grant
    Filed: November 14, 2019
    Date of Patent: February 14, 2023
    Assignee: SERVICENOW CANADA INC.
    Inventors: Perouz Taslakian, David Vazquez Bermudez, Guillem Cucurull Preixens
  • Patent number: 11580364
    Abstract: A method of unsupervised learning of a metric representation and a corresponding system for a mobile device determines a metric position information for a mobile device from an environmental representation. The mobile device comprises at least one sensor for acquiring sensor data and an odometer system configured to acquire displacement data of the mobile device. An environmental representation is generated based on the acquired sensor data by applying an unsupervised learning algorithm. The mobile device moves along a trajectory and the displacement data and the sensor data are acquired while the mobile device is moving along the trajectory. A set of mapping parameters is calculated based on the environmental representation and the displacement data. A metric position estimation is determined based on a further environmental representation and the calculated set of mapping parameters.
    Type: Grant
    Filed: February 28, 2019
    Date of Patent: February 14, 2023
    Assignee: HONDA RESEARCH INSTITUTE EUROPE GMBH
    Inventors: Mathias Franzius, Benjamin Metka, Ute Bauer-Wersing
  • Patent number: 11580365
    Abstract: According to one aspect, a long short-term memory (LSTM) cell for sensor fusion may include M number of forget gates, M number of input gates, and M number output gates. The M number of forget gates may receive M sets of sensor encoding data from M number of sensors and a shared hidden state. The M number of input gates may receive the corresponding M sets of sensor data and the shared hidden state. The M number output gates may generate M partial shared cell state outputs and M partial shared hidden state outputs based on the M sets of sensor encoding data, the shared hidden state, and a shared cell state.
    Type: Grant
    Filed: October 25, 2019
    Date of Patent: February 14, 2023
    Assignee: HONDA MOTOR CO., LTD.
    Inventor: Athmanarayanan Lakshmi Narayanan
  • Patent number: 11580366
    Abstract: An event-driven neural network including a plurality of interconnected core circuits is provided. Each core circuit includes an electronic synapse array that has multiple digital synapses interconnecting a plurality of digital electronic neurons. A synapse interconnects an axon of a pre-synaptic neuron with a dendrite of a post-synaptic neuron. A neuron integrates input spikes and generates a spike event in response to the integrated input spikes exceeding a threshold. Each core circuit also has a scheduler that receives a spike event and delivers the spike event to a selected axon in the synapse array based on a schedule for deterministic event delivery.
    Type: Grant
    Filed: October 28, 2019
    Date of Patent: February 14, 2023
    Assignee: International Business Machines Corporation
    Inventors: Filipp Akopyan, John V. Arthur, Rajit Manohar, Paul A. Merolla, Dharmendra S. Modha, Alyosha Molnar, William P. Risk, III
  • Patent number: 11580367
    Abstract: The present disclosure provides a neural network processing system that comprises a multi-core processing module composed of a plurality of core processing modules and for executing vector multiplication and addition operations in a neural network operation, an on-chip storage medium, an on-chip address index module, and an ALU module for executing a non-linear operation not completable by the multi-core processing module according to input data acquired from the multi-core processing module or the on-chip storage medium, wherein the plurality of core processing modules share an on-chip storage medium and an ALU module, or the plurality of core processing modules have an independent on-chip storage medium and an ALU module. The present disclosure improves an operating speed of the neural network processing system, such that performance of the neural network processing system is higher and more efficient.
    Type: Grant
    Filed: August 9, 2016
    Date of Patent: February 14, 2023
    Assignee: Institute of Computing Technology, Chinese Academy of Sciences
    Inventors: Zidong Du, Qi Guo, Tianshi Chen, Yunji Chen
  • Patent number: 11580368
    Abstract: Provided is an artificial neural network circuit including unit weight memory cells including weight memory devices configured to store weight data and weight pass transistors, unit threshold memory cells including a threshold memory device programmed to store a threshold and a threshold pass transistor, a weight-threshold column in which the plurality of unit weight memory cells and the plurality of unit threshold memory cells are connected, and a sense amplifier configured to receive an output signal of the weight-threshold column as an input and receive a reference voltage as another input.
    Type: Grant
    Filed: November 18, 2019
    Date of Patent: February 14, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Jae-Joon Kim, Hyungjun Kim, Yulhwa Kim
  • Patent number: 11580369
    Abstract: An inference apparatus comprises a plurality of PEs (Processing Elements) and a control part. The control part operates a convolution operation in a convolutional neural network using each of a plurality of pieces of input data and a weight group including a plurality of weights corresponding to each of the plurality of pieces of input data by controlling the plurality of PEs. Further, each of the plurality of PEs executes a computation including multiplication of a single piece of the input data by a single weight and also executes multiplication included in the convolution operation using an element with a non-zero value included in each of the plurality of pieces of input data.
    Type: Grant
    Filed: October 22, 2018
    Date of Patent: February 14, 2023
    Assignee: NEC CORPORATION
    Inventor: Seiya Shibata
  • Patent number: 11580370
    Abstract: Artificial neuromorphic circuit includes synapse and post-neuron circuits. Synapse circuit includes phase change element and receives first and second pulse signals. Post-neuron circuit includes input, output and integration terminals. Integration terminal is charged to membrane potential according to first pulse signal. Post-neuron circuit further includes first and second control circuits, and first and second delay circuits. First control circuit generates firing signal at output terminal based on membrane potential. Second control circuit generates first control signal based on firing signal. First delay circuit delays firing signal to generate second control signal. Second delay circuit delays second control signal to generate third control signal.
    Type: Grant
    Filed: November 15, 2019
    Date of Patent: February 14, 2023
    Assignees: JIANGSU ADVANCED MEMORY TECHNOLOGY CO., LTD., ALTO MEMORY TECHNOLOGY CORPORATION
    Inventors: Chung-Hon Lam, Ching-Sung Chiu
  • Patent number: 11580371
    Abstract: A method, apparatus, and system are discussed to efficiently process and execute Artificial Intelligence operations. An integrated circuit has a tailored architecture to process and execute Artificial Intelligence operations, including computations for a neural network having weights with a sparse value. The integrated circuit contains at least a scheduler, one or more arithmetic logic units, and one or more random access memories configured to cooperate with each other to process and execute these computations for the neural network having weights with the sparse value.
    Type: Grant
    Filed: March 12, 2020
    Date of Patent: February 14, 2023
    Assignee: Roviero, Inc.
    Inventor: Deepak Mital
  • Patent number: 11580372
    Abstract: A hardware architecture for implementing a convolutional neural network.
    Type: Grant
    Filed: July 8, 2022
    Date of Patent: February 14, 2023
    Assignee: Recogni Inc.
    Inventor: Eugene M. Feinberg
  • Patent number: 11580373
    Abstract: Computations in Artificial neural networks (ANNs) are accomplished using simple processing units, called neurons, with data embodied by the connections between neurons, called synapses, and by the strength of these connections, the synaptic weights. Crossbar arrays may be used to represent one layer of the ANN with Non-Volatile Memory (NVM) elements at each crosspoint, where the conductance of the NVM elements may be used to encode the synaptic weights, and a highly parallel current summation on the array achieves a weighted sum operation that is representative of the values of the output neurons. A method is outlined to transfer such neuron values from the outputs of one array to the inputs of a second array with no need for global clock synchronization, irrespective of the distances between the arrays, and to use such values at the next array, and/or to convert such values into digital bits at the next array.
    Type: Grant
    Filed: January 20, 2017
    Date of Patent: February 14, 2023
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Geoffrey W Burr, Pritish Narayanan
  • Patent number: 11580374
    Abstract: An artificial neuron including: a membrane capacitor; an input of an external synaptic excitation in current, the membrane capacitor integrating the input current; a negative-feedback impulse circuit, supplied by a power supply at a negative voltage between ?200 mV and 0 mV and at a positive voltage between 0 mV and +200 mV, including: a bridge based on pMOS and nMOS transistors in series and linked by a midpoint to the membrane capacitor, the midpoint defining the output of the artificial neuron, at least one delay capacitor between the gate and the source of one of the transistors of the bridge, at least two CMOS inverters between the membrane capacitor and the gates of the transistors of the bridge.
    Type: Grant
    Filed: April 7, 2017
    Date of Patent: February 14, 2023
    Assignees: UNIVERSITE DE LILLE, CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE
    Inventors: Alain Cappy, Francois Danneville, Virginie Hoel, Christophe Loyez
  • Patent number: 11580375
    Abstract: Methods and systems for accelerated training of a machine learning based model for semiconductor applications are provided. One method for training a machine learning based model includes acquiring information for non-nominal instances of specimen(s) on which a process is performed. The machine learning based model is configured for performing simulation(s) for the specimens. The machine learning based model is trained with only information for nominal instances of additional specimen(s). The method also includes re-training the machine learning based model with the information for the non-nominal instances of the specimen(s) thereby performing transfer learning of the information for the non-nominal instances of the specimen(s) to the machine learning based model.
    Type: Grant
    Filed: December 29, 2016
    Date of Patent: February 14, 2023
    Assignee: KLA-Tencor Corp.
    Inventors: Kris Bhaskar, Laurent Karsenti, Scott Young, Mohan Mahadevan, Jing Zhang, Brian Duffy, Li He, Huajun Ying, Hung Nien, Sankar Venkataraman
  • Patent number: 11580376
    Abstract: An electronic apparatus is provided. The electronic apparatus includes: a memory storing a trained model including a plurality of layers; and a processor initializing a parameter matrix and a plurality of split variables of a trained model, calculating a new parameter matrix having a block-diagonal matrix for the plurality of split variables and the trained model to minimize a loss function for the trained model, a weight decay regularization term, and an objective function including a split regularization term defined by the parameter matrix and the plurality of split variables, vertically splitting the plurality of layers according to the group based on the computed split parameters and reconstruct the trained model using the computed new parameter matrix as parameters of the vertically split layers.
    Type: Grant
    Filed: June 7, 2018
    Date of Patent: February 14, 2023
    Assignee: Korea Advanced Institute of Science and Technology
    Inventors: Sungju Hwang, Gunhee Kim, Juyong Kim, Yookoon Park
  • Patent number: 11580377
    Abstract: The embodiments of this application provide a method and device for optimizing neural network. The method includes: binarizing and bit-packing input data of a convolution layer along a channel direction, and obtaining compressed input data; binarizing and bit-packing respectively each convolution kernel of the convolution layer along the channel direction, and obtaining each corresponding compressed convolution kernel; dividing the compressed input data sequentially in a convolutional computation order into blocks of the compressed input data with the same size of each compressed convolution kernel, wherein the data input to one time convolutional computation form a data block; and, taking a convolutional computation on each block of the compressed input data and each compressed convolution kernel sequentially, obtaining each convolutional result data, and obtaining multiple output data of the convolution layer according to each convolutional result data.
    Type: Grant
    Filed: June 21, 2018
    Date of Patent: February 14, 2023
    Assignees: TU SIMPLE, INC., BEIJING TUSEN ZHITU TECHNOLOGY CO., LTD.
    Inventors: Yuwei Hu, Jiangming Jin, Lei Su, Dinghua Li
  • Patent number: 11580378
    Abstract: A computer-implemented method comprises instantiating a policy function approximator. The policy function approximator is configured to calculate a plurality of estimated action probabilities in dependence on a given state of the environment. Each of the plurality of estimated action probabilities corresponds to a respective one of a plurality of discrete actions performable by the reinforcement learning agent within the environment. An initial plurality of estimated action probabilities in dependence on a first state of the environment are calculated. Two or more of the plurality of discrete actions are concurrently performed within the environment when the environment is in the first state. In response to the concurrent performance, a reward value is received. In response to the received reward value being greater than a baseline reward value, the policy function approximator is updated, such that it is configured to calculate an updated plurality of estimated action probabilities.
    Type: Grant
    Filed: November 12, 2018
    Date of Patent: February 14, 2023
    Assignee: ELECTRONIC ARTS INC.
    Inventors: Jack Harmer, Linus Gisslén, Magnus Nordin, Jorge del Val Santos
  • Patent number: 11580379
    Abstract: Techniques for phased deployment of machine learning models are described. Customers can call a training API to initiate model training, but then must wait while the training completes before the model can be used to perform inference. Depending on the type of model, machine learning algorithm being used for training, size of the training dataset, etc. this training process may take hours or days to complete. This leads to significant downtime where inference requests cannot be served. Embodiments improve upon existing systems by providing phased deployment of custom models. For example, a simple, less accurate model, can be provided synchronously in response to a request for a custom model. At the same time, one or more machine learning models can be trained asynchronously in the background. When the machine learning model is ready for use, the customers' traffic and jobs can be transferred over to the better model.
    Type: Grant
    Filed: January 29, 2019
    Date of Patent: February 14, 2023
    Assignee: Amazon Technologies, Inc.
    Inventors: David Leen, Sravan Babu Bodapati
  • Patent number: 11580380
    Abstract: Systems and methods for distributed training of deep learning models are disclosed. An example local device to train deep learning models includes a reference generator to label input data received at the local device to generate training data, a trainer to train a local deep learning model and to transmit the local deep learning model to a server that is to receive a plurality of local deep learning models from a plurality of local devices, the server to determine a set of weights for a global deep learning model, and an updater to update the local deep learning model based on the set of weights received from the server.
    Type: Grant
    Filed: August 19, 2017
    Date of Patent: February 14, 2023
    Assignee: Movidius Limited
    Inventor: David Moloney
  • Patent number: 11580381
    Abstract: For machine training and application of a trained complex-valued machine learning model, an activation function of the machine learning model, such as a neural network, includes a learnable parameter that is complex or defined in a complex domain with two dimensions, such as real and imaginary or magnitude and phase dimensions. The complex learnable parameter is trained for any of various applications, such as MR fingerprinting, other medical imaging, or non-medical uses.
    Type: Grant
    Filed: April 25, 2019
    Date of Patent: February 14, 2023
    Assignee: Siemens Healthcare GmbH
    Inventors: Guillaume Daval Frerot, Xiao Chen, Simon Arberet, Boris Mailhe, Mariappan S. Nadar, Peter Speier, Mathias Nittka
  • Patent number: 11580382
    Abstract: A method for providing a training data set used for training a signal classification neural network is provided. The method includes generating at least one first virtual waveform primitive comprising a predetermined signal level and at least one second virtual waveform primitive comprising a signal edge. The training data set is formed and comprises a predetermined number of generated virtual waveform primitives including first virtual waveform primitives and second virtual waveform primitives. Each virtual waveform primitive comprises a sequence of time and amplitude discrete values. The training data set is used for training the signal classification neural network.
    Type: Grant
    Filed: April 26, 2019
    Date of Patent: February 14, 2023
    Assignee: ROHDE & SCHWARZ GMBH & CO. KG
    Inventor: Andreas Werner
  • Patent number: 11580383
    Abstract: A large amount of training data is typically required to perform deep network leaning, making it difficult to achieve using a few pieces of data. In order to solve this problem, the neural network device according to the present invention is provided with: a feature extraction unit which extracts features from training data using a learning neural network; an adversarial feature generation unit which generates an adversarial feature from the extracted features using the learning neural network; a pattern recognition unit which calculates a neural network recognition result using the training data and the adversarial feature; and a network learning unit which performs neural network learning so that the recognition result approaches a desired output.
    Type: Grant
    Filed: March 16, 2017
    Date of Patent: February 14, 2023
    Assignee: NEC CORPORATION
    Inventor: Masato Ishii
  • Patent number: 11580384
    Abstract: The present approach relates to a system capable of life-long learning in a deep learning context. The system includes a deep learning network configured to process an input dataset and perform one or more tasks from among a first set of tasks. As an example, the deep learning network may be part of an imaging system, such as a medical imaging system, or may be used in industrial applications. The system further includes a learning unit communicatively coupled to the deep learning network 102 and configured to modify the deep learning network so as to enable it to perform one or more tasks in a second task list without losing the ability to perform the tasks from the first list.
    Type: Grant
    Filed: July 25, 2019
    Date of Patent: February 14, 2023
    Assignee: GE Precision Healthcare LLC
    Inventors: Rahul Venkataramani, Sai Hareesh Anamandra, Hariharan Ravishankar, Prasad Sudhakar
  • Patent number: 11580385
    Abstract: An AI robot for cleaning in consideration of a user's action includes a camera to acquire a first image data for the user, a cleaning unit including a suction unit and a mopping unit, a driving unit configured to drive the AI robot, and a processor to determine the user's action using the first image data, determine a cleaning schedule in consideration of the user's action, and control the cleaning unit and the driving unit based on the determined cleaning schedule.
    Type: Grant
    Filed: August 12, 2019
    Date of Patent: February 14, 2023
    Assignee: LG ELECTRONICS INC.
    Inventor: Jichan Maeng
  • Patent number: 11580386
    Abstract: Disclosed herein are a convolutional layer acceleration unit, an embedded system having the convolutional layer acceleration unit, and a method for operating the embedded system. The method for operating an embedded system, the embedded system performing an accelerated processing capability programmed using a Lightweight Intelligent Software Framework (LISF), includes initializing and configuring, by a parallelization managing function entity (FE), entities present in resources for performing mathematical operations in parallel, and processing in parallel, by an acceleration managing FE, the mathematical operations using the configured entities.
    Type: Grant
    Filed: December 27, 2019
    Date of Patent: February 14, 2023
    Assignee: Electronics and Telecommunications Research Institute
    Inventor: Seung-Tae Hong
  • Patent number: 11580387
    Abstract: A computer produces predictions throughout a raster field in response to point data, by obtaining a partially empty matrix of point data, filling a matrix of extrapolated raster data by dilating the point data in a first convolutional neural network, and generating a matrix of aggregate raster data by combining the extrapolated raster data with organic raster data in a second convolutional neural network.
    Type: Grant
    Filed: December 29, 2019
    Date of Patent: February 14, 2023
    Assignee: International Business Machines Corporation
    Inventors: Johannes W. Schmude, Siyuan Lu, Hendrik F. Hamann, Akihisa Sakurai, Taku Izumiyama, Masao Hasegawa
  • Patent number: 11580388
    Abstract: Embodiments of the present disclosure include techniques for processing neural networks. Various forms of parallelism may be implemented using topology that combines sequences of processors. In one embodiment, the present disclosure includes a computer system comprising a plurality of processor groups, the processor groups each comprising a plurality of processors. A plurality of network switches are coupled to subsets of the plurality of processor groups. A subset of the processors in the processor groups may be configurable to form sequences, and the network switches are configurable to form at least one sequence across one or more of the plurality of processor groups to perform neural network computations. Various alternative configurations for creating Hamiltonian cycles are disclosed to support data parallelism, pipeline parallelism, layer parallelism, or combinations thereof.
    Type: Grant
    Filed: January 3, 2020
    Date of Patent: February 14, 2023
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Torsten Hoefler, Mattheus C. Heddes, Deepak Goel, Jonathan R Belk
  • Patent number: 11580389
    Abstract: A dynamic graph includes a plurality of nodes and edges at a plurality of time steps; each node corresponds to a geographic location in a first area where pest infestation information is available for a subset of locations. Each edge connects two of the nodes which are geographically proximate, has a direction based on wind direction, and has a weight based on relative wind speed. Assign node features based on weather data as well as labels corresponding to pest infestation severity. Train a graph convolutional network on the dynamic graph. Based on predicted future weather conditions for a second area different than the first area, use the trained graph convolutional network to predict, via inductive learning, pest infestation severity for future times for a new set of nodes corresponding to new geographic locations in the second area for which no pest infestation information is available.
    Type: Grant
    Filed: January 14, 2020
    Date of Patent: February 14, 2023
    Assignee: International Business Machines Corporation
    Inventors: Sambaran Bandyopadhyay, Sachin Gupta
  • Patent number: 11580390
    Abstract: A medical system comprises processing circuitry configured to: receive a first trained model, wherein the trained model has been trained using a first data set acquired in a first cohort; receive a second data set acquired in a second cohort; input data included in the second data set and data representative of the first trained model into a second trained model; and receive from the second trained model an affinity-relating value which represents an affinity between the data included in the second data set and the first trained model.
    Type: Grant
    Filed: January 22, 2020
    Date of Patent: February 14, 2023
    Assignee: CANON MEDICAL SYSTEMS CORPORATION
    Inventors: Owen Anderson, Aneta Lisowska, Alison O'Neil
  • Patent number: 11580391
    Abstract: Disclosed herein is attack-less adversarial training for robust adversarial defense. The attack-less adversarial training for robust adversarial defense includes the steps of: (a) generating individual intervals (ci) by setting the range of color (C) and then discretizing the range of color (C) by a predetermined number (k); (b) generating one batch from an original image (X) and training a learning model with the batch; (c) predicting individual interval indices (?ialat) from respective pixels (xi) of the original image (X) by using an activation function; (d) generating a new image (Xalat) through mapping and randomization; and (e) training a convolutional neural network with the image (Xalat) generated in step (d) and outputting a predicted label (?).
    Type: Grant
    Filed: February 6, 2020
    Date of Patent: February 14, 2023
    Assignee: Dongseo University Headquarters
    Inventors: Jiacang Ho, Byung Gook Lee, Dae-Ki Kang
  • Patent number: 11580392
    Abstract: An apparatus for providing similar contents, using a neural network, includes a memory storing instructions, and a processor configured to execute the instructions to obtain a plurality of similarity values between a user query and a plurality of images, using a similarity neural network, obtain a rank of each the obtained plurality of similarity values, and provide, as a most similar image to the user query, at least one among the plurality of images that has a respective one among the plurality of similarity values that corresponds to a highest rank among the obtained rank of each of the plurality of similarity values. The similarity neural network is trained with a divergence neural network for outputting a divergence between a first distribution of first similarity values for positive pairs, among the plurality of similarity values, and a second distribution of second similarity values for negative pairs, among the plurality of similarity values.
    Type: Grant
    Filed: February 28, 2020
    Date of Patent: February 14, 2023
    Assignee: SAMSUNG ELECTRONICS CO., LTD.
    Inventors: Mete Kemertas, Leila Pishdad, Konstantinos Derpanis, Afsaneh Fazly
  • Patent number: 11580393
    Abstract: A neural network deep learning data control apparatus includes: a memory; an encoding circuit configured to receive a data sequence, generate a compressed data sequence in which consecutive invalid bits in a bit string of the data sequence are compressed into a single bit of the compressed data sequence, generate a validity determination sequence indicating a valid bit and an invalid bit in a bit string of the compressed data sequence, and write the compressed data sequence and the validity determination sequence to the memory; and a decoding circuit configured to read the compressed data sequence and the validity determination sequence from the memory, and determine a bit in the bit string of the compressed data sequence set for transmission to a neural network circuit, based on the validity determination sequence, such that the neural network circuit omits an operation with respect to non-consecutive invalid bits.
    Type: Grant
    Filed: June 12, 2020
    Date of Patent: February 14, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventor: Hyung-Dal Kwon
  • Patent number: 11580394
    Abstract: Techniques in advanced deep learning provide improvements in one or more of accuracy, performance, and energy efficiency, such as accuracy of learning, accuracy of prediction, speed of learning, performance of learning, and energy efficiency of learning. An array of processing elements performs flow-based computations on wavelets of data. Each processing element has a respective compute element and a respective routing element. Each compute element has processing resources and memory resources. Each router enables communication via wavelets with at least nearest neighbors in a 2D mesh. Stochastic gradient descent, mini-batch gradient descent, and continuous propagation gradient descent are techniques usable to train weights of a neural network modeled by the processing elements. Reverse checkpoint is usable to reduce memory usage during the training.
    Type: Grant
    Filed: June 24, 2020
    Date of Patent: February 14, 2023
    Assignee: Cerebras Systems Inc.
    Inventors: Sean Lie, Michael Morrison, Michael Edwin James, Gary R. Lauterbach, Srikanth Arekapudi
  • Patent number: 11580395
    Abstract: A latent code defined in an input space is processed by the mapping neural network to produce an intermediate latent code defined in an intermediate latent space. The intermediate latent code may be used as appearance vector that is processed by the synthesis neural network to generate an image. The appearance vector is a compressed encoding of data, such as video frames including a person's face, audio, and other data. Captured images may be converted into appearance vectors at a local device and transmitted to a remote device using much less bandwidth compared with transmitting the captured images. A synthesis neural network at the remote device reconstructs the images for display.
    Type: Grant
    Filed: October 13, 2020
    Date of Patent: February 14, 2023
    Assignee: NVIDIA Corporation
    Inventors: Tero Tapani Karras, Samuli Matias Laine, David Patrick Luebke, Jaakko T. Lehtinen, Miika Samuli Aittala, Timo Oskari Aila, Ming-Yu Liu, Arun Mohanray Mallya, Ting-Chun Wang
  • Patent number: 11580396
    Abstract: Systems and methods for artificial intelligence discovered codes are described herein. A method includes obtaining received samples from a receive decoder, obtaining decoded bits from the receive decoder based on the receiver samples, training an encoder neural network of a transmit encoder, the encoder neural network receiving parameters that comprise the information bits, the received samples, and the decoded bits. The encoder neural network is optimized using a loss function applied to the decoded bits and the information bits to calculate a forward error correcting code.
    Type: Grant
    Filed: October 13, 2020
    Date of Patent: February 14, 2023
    Assignee: Aira Technologies, Inc.
    Inventors: RaviKiran Gopalan, Anand Chandrasekher, Yihan Jiang
  • Patent number: 11580397
    Abstract: A method for selectively dropping out feature elements from a tensor in a neural network is disclosed. The method includes receiving a first tensor from a first layer of a neural network. The first tensor includes multiple feature elements arranged in a first order. A compressed mask for the first tensor is obtained. The compressed mask includes single-bit mask elements respectively corresponding to the multiple feature elements of the first tensor and has a second order that is different than the first order of their corresponding feature elements in the first tensor. Feature elements from the first tensor are selectively dropped out based on the compressed mask to form a second tensor which is propagated to a second layer of the neural network.
    Type: Grant
    Filed: February 4, 2022
    Date of Patent: February 14, 2023
    Assignee: SambaNova Systems, Inc.
    Inventors: Sathish Terakanambi Sheshadri, Ram Sivaramakrishnan, Raghu Prabhakar
  • Patent number: 11580398
    Abstract: Methods and systems for performing diagnostic functions for a deep learning model are provided. One system includes one or more components executed by one or more computer subsystems. The one or more components include a deep learning model configured for determining information from an image generated for a specimen by an imaging tool. The one or more components also include a diagnostic component configured for determining one or more causal portions of the image that resulted in the information being determined and for performing one or more functions based on the determined one or more causal portions of the image.
    Type: Grant
    Filed: September 1, 2017
    Date of Patent: February 14, 2023
    Assignee: KLA-Tenor Corp.
    Inventors: Jing Zhang, Ravi Chandra Donapati, Mark Roulo, Kris Bhaskar