Abstract: A method of updating a first neural network is disclosed. The method includes providing a computer system with a computer-readable memory that stores specific computer-executable instructions for the first neural network and a second neural network separate from the first neural network. The method also includes providing one or more processors in communication with the computer-readable memory. The one or more processors are programmed by the computer-executable instructions to at least process a first data with the first neural network, process a second data with the second neural network, update a weight in a node of the second neural network by a delta amount as a function of the processing of the second data with the second neural network, and update a weight in a node of the first neural network as a function of the delta amount. A computer system for updating a first neural network is also disclosed. Other features of the preferred embodiments are also disclosed.

Abstract: The terminal apparatus comprises a machine learning part that can execute a process of computing a first model update parameter of a first neural network using training data and a process of computing a second model update parameter of a second neural network using training data for a simulated attack; an encryption processing part that encrypts the first, the second model update parameter using a predetermined homomorphic encryption; a data transmission part that transmits the encrypted first, second model update parameters to a predetermined computation apparatus; and an update part that receives from the computation apparatus model update parameters of the first, the second neural networks computed using the first, the second model update parameters received from another terminal apparatus and updates the first, the second neural networks.

Abstract: A method of operation of a semiconductor device that includes the steps of coupling each of a plurality of digital inputs to a corresponding row of non-volatile memory (NVM) cells that stores an individual weight, initiating a read operation based on a digital value of a first bit of the plurality of digital inputs, accumulating along a first bit-line coupling a first array column weighted bit-line current, in which the weighted bit-line current corresponds to a product of the individual weight stored therein and the digital value of the first bit, and converting and scaling, an accumulated weighted bit-line current of the first column, into a scaled charge of the first bit in relation to a significance of the first bit.

Abstract: An overall gradient vector is computed at a server from a set of ISA vectors corresponding to a set of worker machines. An ISA vector of a worker machine including ISA instructions corresponding to a set of gradients, each gradient corresponding to a weight of a node of a neural network being distributedly trained in the worker machine. A set of register values is optimized for use in an approximation computation with an opcode to produce an x-th approximate gradient of an x-th gradient. A server ISA vector is constructed in which a server ISA instruction in an x-th position corresponds to the x-th gradient in the overall gradient vector. A processor at the worker machine is caused to update a set of weights of the neural network, using the set of optimized register values and the server ISA vector, thereby completing one iteration of training.

Abstract: Disclosed examples to estimate audience sizes of media include a coefficient generator to determine coefficient values for a polynomial based on normalized weighted sums of variances, a normalized weighted sum of covariances, and cardinalities corresponding to a first plurality of vectors of counts from a first database proprietor and a second plurality of vectors of counts from a second database proprietor, a real roots solver to determine a real root value of the polynomial, the real root value indicative of a number of audience members represented in the first plurality of vectors of counts that are also represented in the second plurality of vectors of counts, and an audience size generator to determine the audience size based on the real root value and the cardinalities of the first plurality of vectors of counts and the second plurality of vectors of counts.

Abstract: An industrial 5G dynamic multi-priority multi-access method based on deep reinforcement learning includes the following steps: establishing an industrial 5G network model; establishing a dynamic multi-priority multi-channel access neural network model based on deep reinforcement learning; collecting state, action and reward information of multiple time slots of all industrial 5G terminals in the industrial 5G network as training data; training the neural network model by using the collected data until the packet loss ratio and end-to-end latency meet industrial communication requirements; collecting the state information of all the industrial 5G terminals in the industrial 5G network at the current time slot as the input of the neural network model; conducting multi-priority channel allocation; and conducting multi-access by the industrial 5G terminals according to a channel allocation result.

Abstract: The present disclosure provides a neural network weight matrix adjusting method, a writing control method and a related apparatus, The method comprises: judging whether a weight distribution of a neural network weight matrix is lower than a first preset threshold; if yes, multiplying all weight values in the neural network weight matrix by a first constant; if no, judging whether the weight distribution of the neural network weight matrix is higher than a second preset threshold, wherein the second preset threshold is greater than the first preset threshold; and dividing all weight values in the neural network weight matrix by a second constant, if the weight distribution of the neural network weight matrix is higher than the second preset threshold; wherein the first constant and the second constant are both greater than 1, thereby improving the operation precision.

Abstract: A method for evaluating an external machine learning program while limiting access to internal training data includes providing labeled training data from a first source, receiving, by the first source, a machine learning program from a second source different from the first source, blocking, by the first source, access by the second source to the labeled training data, and training, by the first source, the machine learning program according to a supervised machine learning process using the labeled training data. The method further includes generating a first set of metrics from the supervised machine learning process that provide feedback about training of the neural network model, analyzing the first set of metrics to identify subset data therein, and, in order to permit evaluation of the neural network model, transmitting, to the second source, those metrics from the first set of metrics that do not include the subset data.

Abstract: A method for processing a neural network includes receiving a graph corresponding to an artificial neural network including multiple nodes connected by edges. The method determines a set of independent nodes of multiple nodes to be executed in a neural network. The method also determines a next node in the set of independent nodes to add to an ordered set of the multiple nodes corresponding to an order of execution via a hardware resource for processing the neural network. The next node is determined based on a common hardware resource with a first preceding node in the ordered set or a frequency of nodes in the set of independent nodes to be executed via a same hardware resource. The ordered set of the plurality of nodes is generated based on the next node. The method may be repeated until each of the nodes of the graph are included in the ordered set of the nodes.

Abstract: The present invention relates to method and apparatus for training a neural network for object recognition. A training method which includes inputting a training image set containing an object to be recognized, dividing the image samples in the training image set into simple samples and hard samples, for each kind of the image sample and the variation image sample, performing, a transitive transfer, calculating a distillation loss of the transferred student feature of the image sample relative to a teacher feature extracted from corresponding image sample of the other kind, classifying, the image sample, and calculating a classification loss of the image sample, calculating a total loss related to the training image set, and updating parameters of the neural network according to the calculated total loss.

Abstract: Example neural architecture search methods and image processing methods and apparatuses in the field of computer vision in the field of artificial intelligence are provided. The example neural architecture search method includes determining search space and a plurality of construction units, superimposing the plurality of construction units to obtain a search network, adjusting, in the search space, network architectures of the construction units in the search network, to obtain optimized construction units, and establishing a target neural network based on the optimized construction units. In each construction unit, some channels of an output feature map of each node are processed by using a to-be-selected operation to obtain a processed feature map, and the processed feature map and a remaining feature map are stitched and then input to a next node.

Abstract: There are proposed a method, device, apparatus, and medium for protecting sensitive data. In a method, to-be-processed data is received from a server device. A processing result of a user for the to-be-processed data is received, the processing result comprising sensitive data of the user for the processing of the to-be-processed data. A gradient for training a server model at the server device is determined based on a comparison between the processing result and a prediction result for the to-be-processed data. The gradient is updated in a change direction associated with the gradient so as to generate an updated gradient to be sent to the server device. Noise is added only in the change direction associated with the gradient. The corresponding overhead of processing noise in a plurality of directions can be reduced, and no excessive noise data interfering with training will be introduced to the updated gradient.

Abstract: A method is described that includes executing a convolutional neural network layer on an image processor having an array of execution lanes and a two-dimensional shift register. The two-dimensional shift register provides local respective register space for the execution lanes. The executing of the convolutional neural network includes loading a plane of image data of a three-dimensional block of image data into the two-dimensional shift register.

Abstract: Embodiments for systems and methods of sequential event prediction with noise-contrastive estimation for marked temporal point process are disclosed.

Abstract: Techniques for mixed precision quantization of a machine learning (ML) model. A target bandwidth increase is received (302), for the ML model (114) including objects of a first data type represented by a first number of bits. The target bandwidth increase relates to changing a first portion of the objects to a second data type represented by a second number of bits different from the first number of bits (310). The method further includes sorting the objects in the ML model based on bandwidth (304). The method further includes identifying the first portion of the objects to change from the first data type to the second data type, based on the target bandwidth increase and the sorting of the plurality of objects (508). The method further includes changing the first portion of the objects from the first data type to the second data type (508).

Abstract: A method and resulting structures for a semiconductor device includes forming a source terminal of a semiconductor fin on a substrate. An energy barrier is formed on a surface of the source terminal. A channel is formed on a surface of the energy barrier, and a drain terminal is formed on a surface of the channel. The drain terminal and the channel are recessed on either sides of the channel, and the energy barrier is etched in recesses formed by the recessing. The source terminal is recessed using timed etching to remove a portion of the source terminal in the recesses formed by etching the energy barrier. A first bottom spacer is formed on a surface of the source terminal and a sidewall of the semiconductor fin, and a gate stack is formed on the surface of the first bottom spacer.

Abstract: Systems and methods for precision writing of weight values to a memory capable of storing multiple levels in each cell are disclosed. Embodiments include logic to compare an electrical parameter read from a memory cell with a base reference and an interval reference, and stop writing once the electrical parameter is between the base reference and the base plus the interval reference. The interval may be determined using a greater number of levels than the number of stored levels, to prevent possible overlap of read values of the electrical parameter due to memory device variations.

Abstract: Artificial intelligence is an extremely computationally intensive field such that it can be expensive, time consuming, and energy consuming. Fortunately, many of the calculations required for artificial intelligence can be performed in parallel such that specialized processors can greatly increase computational performance. Specifically, artificial intelligence generally requires large numbers of matrix operations to implement neural networks such that specialized matrix processor circuits can improve performance. To perform all these matrix operations, the matrix processor circuits must be quickly and efficiently supplied with data to process or else the matrix processor circuits end up idle or spending large amounts of time loading in different weight matrix data.

Abstract: Methods, apparatus, systems and articles of manufacture are disclosed to build privacy preserving models. An example apparatus disclosed herein includes a training manager to generate a first modeling plan for client-side resources, and transmit the first modeling plan to the client-side resources. The example apparatus also includes a data aggregator to search for a primary validation flag in response to retrieving client-side model parameters, and an accuracy calculator to, in response to detecting the primary validation flag, perform a secondary validation corresponding to the client-side model parameters using a server-side ground truth data set, and determine whether to update the global model with the client-side model parameters based on a comparison of results of the secondary validation and a validation threshold.

Abstract: There is a need for more effective and efficient anomaly detection. This need can be addressed by, for example, solutions for performing/executing graph convolutional anomaly detection. In one example, a method includes identifying related graph database input data associated with a predictive entity; generating related graph feature data for the predictive entity; generating, based on the related graph feature data and using a graph convolutional neural network model, an anomaly detection score for the predictive entity, wherein at least a portion of the graph convolutional neural network model is trained using confirmation feedback data; performing an anomaly confirmation to generate the confirmation feedback data object for the predictive entity, and integrating the confirmation feedback data object for the predictive entity into the confirmation feedback data associated with the graph convolutional anomaly detection.

Abstract: A text correction engine meets different and changing end user requirements, with the ability to change a desired output by providing sufficient amounts of data, and by finetuning the appropriate text correction engine at the point of origin of the data. It is possible to retain confidentiality of data by retraining the base deep learning model at the base deep learning model's point of origin, to improve the base deep learning model's performance, making the base deep learning model more accurate for different contexts. Separate training of an end user model, leaving the base deep learning model intact, streamlines end user model training, and highlights desirable changes in the base deep learning model for further training or retraining.

Abstract: Audio processing may be performed with a convolutional neural network that includes positional embeddings. Audio data may be received at an audio processing system. A convolutional neural network that concatenates frequency-positional embeddings at an input layer may be used to process the audio data. A result of processing the audio data through the convolutional neural network may be used to perform an audio processing task.

Abstract: An embodiment includes a method, comprising: pruning a layer of a neural network having multiple layers using a threshold; and repeating the pruning of the layer of the neural network using a different threshold until a pruning error of the pruned layer reaches a pruning error allowance.

Abstract: In a system with control logic and a processing element array, two modes of operation may be provided. In the first mode of operation, the control logic may configure the system to perform matrix multiplication or 1×1 convolution. In the second mode of operation, the control logic may configure the system to perform 3×3 convolution. The processing element array may include an array of processing elements. Each of the processing elements may be configured to compute the dot product of two vectors in a single clock cycle, and further may accumulate the dot products that are sequentially computed over time.

Abstract: In an approach to improve implementing program code modifications within a predetermined system embodiments simulate an impact of an implemented modification to a software code against one or more predetermined constraints using a target environment. Further, embodiments comparing a first executed simulation against a second executed simulation of the software code, wherein the second executed simulation comprises the implemented modifications and a current version of the software code. Additionally, embodiments, generate guidance for a user based on the comparison of the first and second executed simulations, wherein the generated guidance comprises positive and negative impacts of the implemented software code modifications regarding compliance with the one or more predetermined constraints, and output, by a user interface, the generated guidance to the user detailing the impact of the implemented modification.

Abstract: A method of training an action selection neural network to perform a demonstrated task using a supervised learning technique. The action selection neural network is configured to receive demonstration data comprising actions to perform the task and rewards received for performing the actions. The action selection neural network has auxiliary prediction task neural networks on one or more of its intermediate outputs. The action selection policy neural network is trained using multiple combined losses, concurrently with the auxiliary prediction task neural networks.

Abstract: A system and method are disclosed for providing an averaging of models for federated learning and blind learning systems. The method includes selecting, at a server, a generator g and a number p, transmitting, to at least two n client devices, the generator g and the number p, receiving, from each client device i of the at least two client devices, a respective value ki=gri mod p and transmitting the set of respective values ki to each client device i of the at least two client devices where respective added group of shares are generated on each client device i. The method includes receiving each respective added group of shares from each client device i of the at least two client devices and adding all the respective added group of shares to make a global sum of shares and dividing the global sum of shares by n.

Abstract: Embodiments described herein provide for a non-transitory machine-readable medium storing instructions to cause one or more processors to perform operations comprising receiving a machine learning model from a server at a client device, training the machine learning model using local data at the client device, generating an update for the machine learning model, the update including a weight vector that represents a difference between the received machine learning model and the trained machine learning model, privatizing the update for the machine learning model, and transmitting the privatized update for the machine learning model to the server.

Abstract: The simplification of neural network models is described. For example, a method for simplifying a neural network model includes providing the neural network model to be simplified, defining a first temporal filter for the conveyance of input from a neuron to an other spatially-extended neuron along the arborized projection, defining a second temporal filter for the conveyance of input from yet another neuron to the spatially-extended neuron along the arborized projection, replacing, in the neural network model, the first, spatially-extended neuron with a first, spatially-constrained neuron and the arborized projection with a first connection extending between the first, spatially-constrained neuron and the second neuron, wherein the first connection filters input from the second neuron in accordance with the first temporal filter and a second connection extending between the first spatially-constrained neuron and the third neuron.

Abstract: An end-to-end cloud-based machine learning platform providing two data pipelines: a first machine learning pipeline provides data transformation, a second machine learning pipeline optimizes those data transformations. The first pipeline transforms raw data into model features, and features into machine learning models. It provides training, inference, and experimentation of online and off-line models for personalizing experiences for game players. The second pipeline optimizes models generated by the first pipeline leveraging a reinforcement learning (RL) model and an evolution strategy (ES) model. The second pipeline learns with its first RL model, from experimentation, the best performing models. To improve the training of new models, the second pipeline also transfers its learning from its first RL model to its second ES model to generate the training of new models in the first pipeline. The second pipeline can be considered as an overlay pipeline to the first one.

Abstract: A method for securing a genuine machine learning model against adversarial samples includes the steps of attaching a trigger to a sample to be classified and classifying the sample with the trigger attached using a backdoored model that has been backdoored using the trigger. In a further step, it is determined whether an output of the backdoored model is the same as a backdoor class of the backdoored model, and/or an outlier detection method is applied to logits compared to honest logits that were computed using a genuine sample. These steps are repeated using different triggers and backdoored models respectively associated therewith. It is compared a number of times that an output of the backdoored models is not the same as the respective backdoor class, and/or a difference determined by applying the outlier detection method, against one or more thresholds so as to determine whether the sample is adversarial.

Abstract: A neural network processing unit (NPU) includes a processing element array, an NPU memory system configured to store at least a portion of data of an artificial neural network model processed in the processing element array, and an NPU scheduler configured to control the processing element array and the NPU memory system based on artificial neural network model structure data or artificial neural network data locality information.

Abstract: A computing system generates a plurality of training data sets for generating the NLP model. The computing system trains a teacher network to extract and classify tokens from a document. The training includes a pre-training stage where the teacher network is trained to classify generic data in the plurality of training data sets and a fine-tuning stage where the teacher network is trained to classify targeted data in the plurality of training data sets. The computing system trains a student network to extract and classify tokens from a document by distilling knowledge learned by the teacher network during the fine-tuning stage from the teacher network to the student network. The computing system outputs the NLP model based on the training. The computing system causes the NLP model to be deployed in a remote computing environment.

Abstract: In one embodiment, a method for machine learning acceleration includes receiving instructions to perform convolution on an input tensor using a filter tensor, determining that the size of a first dimension of the input tensor is less than a processing capacity of each of multiple subarrays of computation units in a tensor processor, selecting a second dimension of the input tensor along which to perform the convolution, selecting, based on the second dimension, one or more dimensions of the filter tensor, generating (1) first instructions for reading, using vector read operations, activation elements in the input tensor organized such that elements with different values in the second dimension are stored contiguously in memory, and (2) second instructions for reading weights of the filter tensor along the selected one or more dimensions, and using the first and second instructions to provide the activation elements and the weights to the subarrays.

Abstract: A method, system, and computer program product, including generating, using a linear probe, confidence scores through flattened intermediate representations and theoretically-justified weighting of samples during a training of the simple model using the confidence scores of the intermediate representations.

Abstract: Embodiments of the present disclosure relate to a texture unit circuit in a neural processor circuit. The neural processor circuit includes a tensor access operation circuit with the texture unit circuit, a data processor circuit, and at least one neural engine circuit. The texture unit circuit fetches a source tensor from a system memory by referencing an index tensor in the system memory representing indexing information into the source tensor. The data processor circuit stores an output version of the source tensor obtained from the tensor access operation circuit and sends the output version of the source tensor as multiple of units of input data to the at least one neural engine circuit. The at least one neural engine circuit performs at least convolution operations on the units of input data and at least one kernel to generate output data.

Abstract: Disclosed is a process for testing a suspect model to determine whether it was derived from a source model. An example method includes receiving, from a model owner node, a source model and a fingerprint associated with the source model, receiving a suspect model at a service node, based on a request to test the suspect model, applying the fingerprint to the suspect model to generate an output and, when the output has an accuracy that is equal to or greater than a threshold, determining that the suspect model is derived from the source model. Imperceptible noise can be used to generate the fingerprint which can cause predictable outputs from the source model and a potential derivative thereof.

Abstract: A method includes receiving a first data set comprising embeddings of first and second types, generating a fixed adjacency matrix from the first dataset, and applying a first stochastic binary mask to the fixed adjacency matrix to obtain a first subgraph of the fixed adjacency matrix. The method also includes processing the first subgraph through a first layer of a graph convolutional network (GCN) to obtain a first embedding matrix, and applying a second stochastic binary mask to the fixed adjacency matrix to obtain a second subgraph of the fixed adjacency matrix. The method includes processing the first embedding matrix and the second subgraph through a second layer of the GCN to obtain a second embedding matrix, and then determining a plurality of gradients of a loss function, and modifying the first stochastic binary mask and the second stochastic binary mask using at least one of the plurality of gradients.

Abstract: A method for optimizing a deep learning operator, includes: calling a method of reading an image object to read target data from an L1 cache of an image processor to the processor in response to detecting the target data in the L1 cache, performing a secondary quantization operation on the target data in the processor to obtain an operation result and writing the operation result into a main memory of the image processor. The target data is fixed-point data obtained after performing a quantization operation on data to be quantized in advance and the data to be quantized is one of the following: float-point data of an initial network layer of the neural network model and fixed-point data outputted from a network layer previous to the current network layer.

Abstract: A method and system for computing one or more outputs of a neural network having a plurality of layers is provided. The method and system can include determining a plurality of sub-computations from total computations of the neural network to execute in parallel wherein the computations to execute in parallel involve computations from multiple layers. The method and system also can also include avoiding repeating overlapped computations and/or multiple memory reads and writes during execution.

Abstract: Systems and methods for model evaluation. A model is evaluated by performing a decomposition process for a model output, relative to a baseline input data set.

Abstract: The embodiment of the present disclosure provides an Industrial Internet of Things system for inspection operation management of an inspection robot and a method thereof. The system includes a user platform, a service platform, a management platform, a sensor network platform, and an object platform that are interacted sequentially from top to bottom. The management platform is configured to perform operations including: determining an inspection task, the inspection task including detecting at least one detection site; sending instructions to an inspection robot based on the inspection task to move the inspection robot to a target position to be inspected; obtaining detection data based on the inspection robot, and determining subsequent detection or processing operations based on the detection data.

Abstract: The present disclosure provides a traditional Chinese medicine (TCM) syndrome classification method based on multi-graph attention, which comprehensively considers the contribution of symptoms and syndrome elements in syndrome classification by constructing a graph structure, integrates a symptom-symptom graph and a symptom-syndrome element graph into classification, uses a multi-graph attention network to aggregate the features of symptoms and syndrome elements, and finally realizes syndrome classification through a multi-layer perceptron. At the same time, extensive experiments are carried out on real data sets, the effectiveness of the multi-graph attention network is verified, more accurate classification is realized, and better classification results have been achieved.

Abstract: A brain-inspired cognitive learning method can obtain good learning results in various environments and tasks by selecting the most suitable algorithm models and parameters based on the environments and tasks, and can correct wrong behavior. The framework includes four main modules: a cognitive feature extraction module, a cognitive control module, a learning network module, and a memory module. The memory module includes a data base, a cognitive case base, and an algorithm and hyper-parameter base, which store data of dynamic environments and tasks, cognitive cases, and concrete algorithms and hyper-parameter values, respectively. For dynamic environments and tasks, the most suitable algorithm model and hyper-parameter combination can be flexibly selected. In addition, with “good money drives out bad”, mislabeled data is corrected using correctly labeled data, to achieve robustness of training data.

Abstract: An information processing method implemented by a computer includes: executing a generation processing that includes generating a first mini-batch by performing data extension processing on learning data and processing to generate a second mini-batch without performing the data extension processing on the learning data; and executing a learning processing by using a neural network, the learning processing being configured to perform first learning by using the first mini-batch, and then perform second learning by using the second mini-batch.

Abstract: A face recognition method based on an evolutionary convolutional neural network is provided. The method optimizes the design of convolutional neural network architecture and the initialization of connection weights by using a genetic algorithm and finds an optimal neural network through continuous evolutionary calculation, thus reducing dependence on artificial experience during the design of the convolutional neural network architecture. The method encodes the convolutional neural networks by using a variable-length genetic encoding algorithm, so as to improve the diversity of structures of convolutional neural networks. Additionally, in order to cross over extended chromosomes, structural units at corresponding positions are separately crossed over and then recombined, thereby realizing the crossover of chromosomes with different lengths.

Abstract: Techniques in advanced deep learning provide improvements in one or more of accuracy, performance, and energy efficiency, such as accuracy of learning, accuracy of prediction, speed of learning, performance of learning, and energy efficiency of learning. An array of processing elements performs flow-based computations on wavelets of data. Each processing element has a respective compute element and a respective routing element. Each compute element has processing resources and memory resources. Each router enables communication via wavelets with at least nearest neighbors in a 2D mesh. Stochastic gradient descent, mini-batch gradient descent, and continuous propagation gradient descent are techniques usable to train weights of a neural network modeled by the processing elements. Reverse checkpoint is usable to reduce memory usage during the training.

Abstract: The present invention discloses a two-dimensional photonic neural network convolutional acceleration chip based on series connection structure, which is integrated with a modulator, M microring delay weighting units, M?1 secondary delay waveguide, a wavelength-division multiplexer and a photodetector.

Abstract: A system and method of autonomous data hub processing that uses semantic metadata, machine learning models, and a permissioned blockchain to autonomously standardize, identify and correct errors in supply chain data is disclosed. Embodiments input supply chain data stored in a supply chain database, train with the machine learning model trainer, one or more machine learning models to identify one or more data errors in the supply chain data, clean the one or more identified data errors from the supply chain data, and store cleaned supply chain data. Embodiments also update one or more machine learning models to identify one or more data errors in cleaned supply chain data, and join and aggregate one or more sets of cleaned supply chain data.

Abstract: Dynamic campaign optimization systems and methods may be used to continuously test many alternative campaign configurations while allowing all configurations, including configurations formerly identified as successful and unsuccessful, to be re-tested in order to identify successful configurations that may previously have been identified as unsuccessful.