Abstract: Systems and methods for model evaluation. A model is evaluated by performing a decomposition process for a model output, relative to a baseline input data set.

Abstract: A method and system for computing one or more outputs of a neural network having a plurality of layers is provided. The method and system can include determining a plurality of sub-computations from total computations of the neural network to execute in parallel wherein the computations to execute in parallel involve computations from multiple layers. The method and system also can also include avoiding repeating overlapped computations and/or multiple memory reads and writes during execution.

Abstract: The embodiment of the present disclosure provides an Industrial Internet of Things system for inspection operation management of an inspection robot and a method thereof. The system includes a user platform, a service platform, a management platform, a sensor network platform, and an object platform that are interacted sequentially from top to bottom. The management platform is configured to perform operations including: determining an inspection task, the inspection task including detecting at least one detection site; sending instructions to an inspection robot based on the inspection task to move the inspection robot to a target position to be inspected; obtaining detection data based on the inspection robot, and determining subsequent detection or processing operations based on the detection data.

Abstract: A brain-inspired cognitive learning method can obtain good learning results in various environments and tasks by selecting the most suitable algorithm models and parameters based on the environments and tasks, and can correct wrong behavior. The framework includes four main modules: a cognitive feature extraction module, a cognitive control module, a learning network module, and a memory module. The memory module includes a data base, a cognitive case base, and an algorithm and hyper-parameter base, which store data of dynamic environments and tasks, cognitive cases, and concrete algorithms and hyper-parameter values, respectively. For dynamic environments and tasks, the most suitable algorithm model and hyper-parameter combination can be flexibly selected. In addition, with “good money drives out bad”, mislabeled data is corrected using correctly labeled data, to achieve robustness of training data.

Abstract: The present disclosure provides a traditional Chinese medicine (TCM) syndrome classification method based on multi-graph attention, which comprehensively considers the contribution of symptoms and syndrome elements in syndrome classification by constructing a graph structure, integrates a symptom-symptom graph and a symptom-syndrome element graph into classification, uses a multi-graph attention network to aggregate the features of symptoms and syndrome elements, and finally realizes syndrome classification through a multi-layer perceptron. At the same time, extensive experiments are carried out on real data sets, the effectiveness of the multi-graph attention network is verified, more accurate classification is realized, and better classification results have been achieved.

Abstract: An information processing method implemented by a computer includes: executing a generation processing that includes generating a first mini-batch by performing data extension processing on learning data and processing to generate a second mini-batch without performing the data extension processing on the learning data; and executing a learning processing by using a neural network, the learning processing being configured to perform first learning by using the first mini-batch, and then perform second learning by using the second mini-batch.

Abstract: A face recognition method based on an evolutionary convolutional neural network is provided. The method optimizes the design of convolutional neural network architecture and the initialization of connection weights by using a genetic algorithm and finds an optimal neural network through continuous evolutionary calculation, thus reducing dependence on artificial experience during the design of the convolutional neural network architecture. The method encodes the convolutional neural networks by using a variable-length genetic encoding algorithm, so as to improve the diversity of structures of convolutional neural networks. Additionally, in order to cross over extended chromosomes, structural units at corresponding positions are separately crossed over and then recombined, thereby realizing the crossover of chromosomes with different lengths.

Abstract: The present invention discloses a two-dimensional photonic neural network convolutional acceleration chip based on series connection structure, which is integrated with a modulator, M microring delay weighting units, M?1 secondary delay waveguide, a wavelength-division multiplexer and a photodetector.

Abstract: Techniques in advanced deep learning provide improvements in one or more of accuracy, performance, and energy efficiency, such as accuracy of learning, accuracy of prediction, speed of learning, performance of learning, and energy efficiency of learning. An array of processing elements performs flow-based computations on wavelets of data. Each processing element has a respective compute element and a respective routing element. Each compute element has processing resources and memory resources. Each router enables communication via wavelets with at least nearest neighbors in a 2D mesh. Stochastic gradient descent, mini-batch gradient descent, and continuous propagation gradient descent are techniques usable to train weights of a neural network modeled by the processing elements. Reverse checkpoint is usable to reduce memory usage during the training.

Abstract: A system and method of autonomous data hub processing that uses semantic metadata, machine learning models, and a permissioned blockchain to autonomously standardize, identify and correct errors in supply chain data is disclosed. Embodiments input supply chain data stored in a supply chain database, train with the machine learning model trainer, one or more machine learning models to identify one or more data errors in the supply chain data, clean the one or more identified data errors from the supply chain data, and store cleaned supply chain data. Embodiments also update one or more machine learning models to identify one or more data errors in cleaned supply chain data, and join and aggregate one or more sets of cleaned supply chain data.

Abstract: Lifelong Deep Neural Network (L-DNN) technology revolutionizes Deep Learning by enabling fast, post-deployment learning without extensive training, heavy computing resources, or massive data storage. It uses a representation-rich, DNN-based subsystem (Module A) with a fast-learning subsystem (Module B) to learn new features quickly without forgetting previously learned features. Compared to a conventional DNN, L-DNN uses much less data to build robust networks, dramatically shorter training time, and learning on-device instead of on servers. It can add new knowledge without re-training or storing data. As a result, an edge device with L-DNN can learn continuously after deployment, eliminating massive costs in data collection and annotation, memory and data storage, and compute power. This fast, local, on-device learning can be used for security, supply chain monitoring, disaster and emergency response, and drone-based inspection of infrastructure and properties, among other applications.

Abstract: Dynamic campaign optimization systems and methods may be used to continuously test many alternative campaign configurations while allowing all configurations, including configurations formerly identified as successful and unsuccessful, to be re-tested in order to identify successful configurations that may previously have been identified as unsuccessful.

Abstract: Methods and apparatuses that generate a simulation object for a physical system are described. The simulation object includes a trained computing structure to determine future output data of the physical system in real time. The computing structure is trained with a plurality of input units and one or more output units. The plurality of input units include regular input units to receive input data and output data of the physical system. The output units include one or more regular output units to predict a dynamic rate of change of the input data over a period of time. The input data and output data of the physical system are obtained for training the computing structure. The input data represent a dynamic input excitation to the physical system over the period of time. And the output data represents a dynamic output response of the physical system to the dynamic input excitation over the period of time.

Abstract: A method and apparatus for performing refactored multiply-and-accumulate operations is provided. A summing array includes a plurality of non-volatile memory elements arranged in columns. Each non-volatile memory element in the summing array is programmed to a high resistance state or a low resistance state based on weights of a neural network. The summing array is configured to generate a summed signal for each column based, at least in part, on a plurality of input signals. A multiplying array is coupled to the summing array, and includes a plurality of non-volatile memory elements. Each non-volatile memory element in the multiplying array is programmed to a different conductance level based on the weights of the neural network. The multiplying array is configured to generate an output signal based, at least in part, on the summed signals from the summing array.

Abstract: A system for an artificial neural network (ANN) includes a processor configured to output a memory control signal including an ANN data locality; a main memory in which data of an ANN model corresponding to the ANN data locality is stored; and a memory controller configured to receive the memory control signal from the processor and to control the main memory based on the memory control signal. The memory controller may be further configured to control, based on the memory control signal, a read or write operation of data of the main memory required for operation of the artificial neural network. Thus, the system optimizes an ANN operation of the processor by utilizing the ANN data locality of the ANN model, which operates at a processor-memory level.

Abstract: A device configured to provide access to a digital document to a user device and to receive an access request for a first masked data element within the digital document. The device is further configured to generate a first blockchain transaction that identifies a machine learning model that is stored in a blockchain. The device is further configured to publish the first blockchain transaction in a blockchain ledger for the blockchain and to receive a second blockchain transaction from the machine learning model in response to publishing the blockchain transaction in the blockchain ledger. The second transaction indicates whether the user is approved for accessing the masked data element. The device is further configured to provide access to the first masked data element on the user device for the user in response to determining that the user is approved for accessing the masked data element.

Abstract: Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.

Abstract: Methods and systems are disclosed herein for using anomaly detection in timeseries data of user sentiment to detect incidents in computing systems and identify events within an enterprise. An anomaly detection system may receive social media messages that include a timestamp indicating when each message was published. The system may generate sentiment identifiers for the social media messages. The sentiment identifiers and timestamps associated with the social media messages may be used to generate a timeseries dataset for each type of sentiment identifier. The timeseries datasets may be input into an anomaly detection model to determine whether an anomaly has occurred. The system may retrieve textual data from the social media messages associated with the detected anomaly and may use the text to determine a computing system or event associated with the detected anomaly.

Abstract: A method for obstacle avoidance in degraded environments of robots based on intrinsic plasticity of an SNN is disclosed. A decision network in a synaptic autonomous learning module takes lidar data, distance from a target point and velocity at a previous moment as state input, and outputs the velocity of left and right wheels of the robot through the autonomous adjustment of the dynamic energy-time threshold, so as to carry out autonomous perception and decision making. The method solves the difficulty of the lack of intrinsic plasticity in the SNN, which leads to the difficulty of adapting to degraded environments due to the homeostasis imbalance of the model, is successfully deployed in mobile robots to maintain a stable trigger rate for autonomous navigation and obstacle avoidance in degraded, disturbed and noisy environments, and has validity and applicability on different degraded scenes.

Abstract: An arithmetic operation device is provided that removes a part of parameters of a predetermined number of parameters from a first machine learning model which includes the predetermined number of parameters and is trained so as to output second data corresponding to input first data, determines the number of bits of a weight parameter according to required performance related to an inference to generate a second machine learning model, and acquires data output from the second machine learning model so as to correspond to the input first data with a smaller computational complexity than the first machine learning model.

Abstract: A system comprises a memory that stores computer-executable components; and a processor, operably coupled to the memory, that executes the computer-executable components. The system includes a receiving component that receives a corpus of data; a relation extraction component that generates noisy knowledge graphs from the corpus; and a training component that acquires global representations of entities and relation by training from output of the relation extraction component.

Abstract: An information processing system preserves data used in machine learning by distributing the data to a plurality of servers, reads setting information indicating a method of partitioning for cross-validation in the machine learning, specifies, based on the setting information, a validation server that executes the cross-validation among the plurality of servers, and validation data which is data used in the cross-validation, specifies an arrangement of the data in the plurality of servers, specifies deficiency data, which is data that is included in the validation data and that is not stored in the validation server, and causes a server that stores the deficiency data among the plurality of servers to transmit the deficiency data to the validation server, based on an arrangement of the specified deficiency data.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using recurrent neural networks. One of the systems includes a main recurrent neural network comprising one or more recurrent neural network layers and a respective hyper recurrent neural network corresponding to each of the one or more recurrent neural network layers, wherein each hyper recurrent neural network is configured to, at each of a plurality of time steps: process the layer input at the time step to the corresponding recurrent neural network layer, the current layer hidden state of the corresponding recurrent neural network layer, and a current hypernetwork hidden state of the hyper recurrent neural network to generate an updated hypernetwork hidden state.

Abstract: Techniques are disclosed for detecting adversarial attacks. A machine learning (ML) system processes the input into and output of a ML model using an adversarial detection module that does not include a direct external interface. The adversarial detection module includes a detection model that generates a score indicative of whether the input is adversarial using, e.g., a neural fingerprinting technique or a comparison of features extracted by a surrogate ML model to an expected feature distribution for the output of the ML model. In turn, the adversarial score is compared to a predefined threshold for raising an adversarial flag. Appropriate remedial measures, such as notifying a user, may be taken when the adversarial score satisfies the threshold and raises the adversarial flag.

Abstract: A method includes dividing a plurality of filters in a first layer of a neural network into a first set of filters and a second set of filters, applying each of the first set of filters to an input of the neural network, aggregating, at a second layer of the neural network, a respective one of a first set of outputs with a respective one of a second set of outputs, splitting respective weights of specific neurons activated in each remaining layer, at each specific neuron from each remaining layer, applying a respective filter associated with each specific neuron and a first corresponding weight, obtaining a second set of neuron outputs, for each specific neuron, aggregating one of the first set of neuron outputs with one of a second set of neuron outputs and generating an output of the neural network based on the aggregated neuron outputs.

Abstract: An apparatus and method are described for reducing the parameter density of a deep neural network (DNN). A layer-wise pruning module to prune a specified set of parameters from each layer of a reference dense neural network model to generate a second neural network model having a relatively higher sparsity rate than the reference neural network model; a retraining module to retrain the second neural network model in accordance with a set of training data to generate a retrained second neural network model; and the retraining module to output the retrained second neural network model as a final neural network model if a target sparsity rate has been reached or to provide the retrained second neural network model to the layer-wise pruning model for additional pruning if the target sparsity rate has not been reached.

Abstract: Adaptive exploration in deep reinforcement learning may be performed by inputting a current time frame of an action and observation sequence sequentially into a function approximator, such as a deep neural network, including a plurality of parameters, the action and observation sequence including a plurality of time frames, each time frame including action values and observation values, approximating a value function using the function approximator based on the current time frame to acquire a current value, updating an action selection policy through exploration based on an ?-greedy strategy using the current value, and updating the plurality of parameters.

Abstract: Using a deep learning inference system, respective similarities are measured for each of a set of intermediate representations to input information used as an input to the deep learning inference system. The deep learning inference system includes multiple layers, each layer producing one or more associated intermediate representations. Selection is made of a subset of the set of intermediate representations that are most similar to the input information. Using the selected subset of intermediate representations, a partitioning point is determined in the multiple layers used to partition the multiple layers into two partitions defined so that information leakage for the two partitions will meet a privacy parameter when a first of the two partitions is prevented from leaking information. The partitioning point is output for use in partitioning the multiple layers of the deep learning inference system into the two partitions.

Abstract: Provided is an apparatus configured to determine a common neural network based on a comparison between a first neural network included in a first application program and a second neural network included in a second application program, utilize the common neural network when the first application program or the second application program is executed.

Abstract: Methods, systems, and apparatus for selecting actions to be performed by an agent interacting with an environment. One system includes a high-level controller neural network, low-level controller network, and subsystem. The high-level controller neural network receives an input observation and processes the input observation to generate a high-level output defining a control signal for the low-level controller. The low-level controller neural network receives a designated component of an input observation and processes the designated component and an input control signal to generate a low-level output that defines an action to be performed by the agent in response to the input observation.

Abstract: A client device configured with a neural network includes a processor, a memory, a user interface, a communications interface, a power supply and an input device, wherein the memory includes a trained neural network received from a server system that has trained and configured the neural network for the client device. A server system and a method of training a neural network are disclosed.

Abstract: A normalization method for machine learning and an apparatus thereof are provided. The normalization method according to some embodiments of the present disclosure may calculate a value of a normalization parameter for an input image through a normalization model before inputting the input image to a target model and normalize the input image using the calculated value of the normalization parameter. Because the normalization model is updated based on a prediction loss of the target model, the input image can be normalized to an image suitable for a target task, so that stability of the learning and performance of the target model can be improved.

Abstract: An indication of semantic relationships among classes is obtained. A neural network whose loss function is based at least partly on the semantic relationships is trained. The trained neural network is used to identify one or more classes to which an input observation belongs.

Abstract: An off-policy reinforcement learning actor-critic neural network system configured to select actions from a continuous action space to be performed by an agent interacting with an environment to perform a task. An observation defines environment state data and reward data. The system has an actor neural network which learns a policy function mapping the state data to action data. A critic neural network learns an action-value (Q) function. A replay buffer stores tuples of the state data, the action data, the reward data and new state data. The replay buffer also includes demonstration transition data comprising a set of the tuples from a demonstration of the task within the environment. The neural network system is configured to train the actor neural network and the critic neural network off-policy using stored tuples from the replay buffer comprising tuples both from operation of the system and from the demonstration transition data.

Abstract: A method for utilizing a graph path cache to facilitate real-time data consumption by a plurality of machine learning models is disclosed. The method includes receiving an input from a source, the input relating to a request to characterize a data element; retrieving a data attribute that corresponds to the data element from a data management system; determining, in real-time using the graph path cache, a graph attribute that corresponds to the data element by performing deep link analysis on a graph database; executing, in real-time, a model by using the data attribute and the graph attribute, the model corresponding to the request in the input; and transmitting, in real-time, a result of the executed model to the source in response to the input.

Abstract: Provided are systems and methods for operating a neural network processor, wherein the processor includes an input selector circuit that can be configured to select the data that will be input into the processor's computational array. In various implementations, the selector circuit can determine, for a row of the array, whether the row input will be the output from a buffer memory or data that the input selector circuit has selected for a different row. The row can receive an input feature map from a set of input data or an input feature map that was selected for inputting into a different row, such that the input feature map is input into more than one row at a time. The selector circuit can also include a delay circuit, so that the duplicated input feature map can be input into the computational array later than the original input feature map.

Abstract: A method of performing unsupervised detection of repeating patterns in a series (TS) of events (E21, E12, E5, . . . ), comprising the steps of: a) Providing a plurality of neurons (NR1-NRP), each neuron being representative of W event types; b) Acquiring an input packet (IV) comprising N successive events of the series; c) Attributing to at least some neurons a potential value (PT1-PTP), representative of the number of common events between the input packet and the neuron; d) Modifying the event types of neurons having a potential value exceeding a first threshold TL; and e) Generating a first output signal (OS1-OSP) for all neurons having a potential value exceeding a second threshold TF, and a second output signal, different from the first one, for all other neurons. A digital electronic circuit and system configured for carrying out the above method.

Abstract: A processor-implemented neural network method includes acquiring connection weight of an analog neural network (ANN) node of a pre-trained ANN; and determining, a firing rate of a spiking neural network (SNN) node of an SNN, corresponding to the ANN node, based on an activation of the ANN node which is determined based on the connection weight. and the firing rate is also determined based on information indicating a timing at which the SNN node initially fires.

Abstract: A method of performing convolution in a neural network with variable dilation rate is provided. The method includes receiving a size of a first kernel and a dilation rate, determining at least one of size of one or more disintegrated kernels based on the size of the first kernel, a baseline architecture of a memory and the dilation rate, determining an address of one or more blocks of an input image based on the dilation rate, and one or more parameters associated with a size of the input image and the memory. Thereafter, the one or more blocks of the input image and the one or more disintegrated kernels are fetched from the memory, and an output image is obtained based on convolution of each of the one or more disintegrated kernels and the one or more blocks of the input image.

Abstract: A method for identifying a fault of at least one mechanical machine, including causing a first plurality of sensors coupled to a corresponding first plurality of mechanical machines to acquire a first plurality of sets of signals emanating from the first plurality of mechanical machines, the first plurality of mechanical machines sharing at least one characteristic, supplying at least the first plurality of sets of signals of the first plurality of mechanical machines to a pre-existing fault classifier previously trained to automatically identify faults of a second plurality of mechanical machines based on signals emanating therefrom and previously acquired by a second plurality of sensors, the second plurality of sensors being of a different type than the first plurality of sensors, the second plurality of mechanical machines sharing the at least one characteristic, modifying the pre-existing fault classifier by employing transfer learning, based at least on the first plurality of sets of signals of the firs

Abstract: Provided is an operation method for a memory device, the memory device being used for implementing an Artificial Neural Network (ANN). The operation method includes: reading from the memory device a weight matrix of a current layer of a plurality of layers of the ANN to extract a plurality of neuro values; determining whether to perform calibration; when it is determined to perform calibration, recalculating and updating a mean value and a variance value of the neuro values; and performing batch normalization based on the mean value and the variance value of the neuro values.

Abstract: Message faults are expected to be an increasing problem in 5G and 6G, due to signal fading at high frequencies, heavy background interference, and high user densities. Retransmissions are expensive in time, power, and the additional background they generate. Prior art includes “soft-combining” among multiple copies, an especially ineffective fault mitigation procedure when SNR is low. Nevertheless, the waveform signals of even badly faulted message elements are rich with information about the correct value. Therefore, procedures are disclosed herein for determining which message elements of a corrupted message, or its associated error-detection code, are faulted, by measuring characteristic parameters of the signal waveform of each message element, and correlating those parameters with the associated error-detection code. In many cases, the corrupted message may be corrected without a retransmission, according to some embodiments.

Abstract: The present disclosure provides a training and application method of a multi-layer neural network model, apparatus and storage medium. A number of channels of a filter in at least one convolutional layer in the multi-layer neural network model is expanded, and a convolution computation is performed by using the filter after expanding the number of channels, so that the performance of the network model does not degrade while simplifying the network model.

Abstract: The present invention provides a frequency modulation dynamic modeling method and device for a wind farm, and an electronic device. The method includes: acquiring first frequency modulation data measured at a grid-connected point of the wind farm under a plurality of preset working conditions; establishing a state space model corresponding to each of the plurality of working conditions according to the first frequency modulation data; measuring the nonlinearity between the state space models corresponding to each two of the plurality of working conditions by using a gap measurement method; combining the first frequency modulation data according to the nonlinearity to obtain second frequency modulation data; and training a preset initial LSTM neural network according to the second frequency modulation data until a preset training requirement is met, and obtaining a trained frequency modulation dynamic model of the wind farm.

Abstract: A location system for determining a position of a device is described here. The location system comprises a memory and a processor, the processor executes the computer-executable instruction to perform operations. The operations include sending a first instruction to a locator beacon to determine a first location of the device based on angle of arrival calculation of a first set of one or more packets received from the device. The operations further include sending a second instruction to the device to determine a second location of the device based on angle of departure calculation of a second set of one or more packets received from the locator beacon. Furthermore, the operation includes receiving the first location from the locator beacon and receiving the second location from the device. The operations further include determining the position of the device based on a function of the first location and the second location.

Abstract: Data is prone to various attacks such as cyber-security attacks, in any industry. State of the art systems in the domain of data security fail to identify adversarial attacks in real-time, and this leads to security issues, as well as results in the process/system providing unintended results. The disclosure herein generally relates to data security analysis, and, more particularly, to a method and system for assessing impact of adversarial attacks on time series data and providing defenses against such attacks. The system performs adversarial attacks on a selected data-driven model to determine impact of the adversarial attacks on the selected data model, and if the impact is such that performance of the selected data model is less than a threshold, then the selected data model is retrained.

Abstract: A neural network method of parameter quantization obtains channel profile information for first parameter values of a floating-point type in each channel included in each of feature maps based on an input in a first dataset to a floating-point parameters pre-trained neural network, and determines a probability density function (PDF) type, for each channel, appropriate for the channel profile information based on a classification network receiving the channel profile information as a dataset. The neural network method of parameter quantization determines a fixed-point representation, based on the determined PDF type, for each channel, statistically covering a distribution range of the first parameter values, and generates a fixed-point quantized neural network based on the fixed-point representation determined for each channel.

Abstract: In one aspect, this specification describes a recurrent neural network system implemented by one or more computers that is configured to process input sets to generate neural network outputs for each input set. The input set can be a collection of multiple inputs for which the recurrent neural network should generate the same neural network output regardless of the order in which the inputs are arranged in the collection. The recurrent neural network system can include a read neural network, a process neural network, and a write neural network. In another aspect, this specification describes a system implemented as computer programs on one or more computers in one or more locations that is configured to train a recurrent neural network that receives a neural network input and sequentially emits outputs to generate an output sequence for the neural network input.

Abstract: An apparatus comprises a processing device configured to receive a service request associated with a given asset, to obtain a log file associated with the given asset, to split the log file into log segments, to generate sets of log pattern identifiers for the log segments, and to determine risk scores for the log segments utilizing a machine learning model that takes as input the sets of log pattern identifiers and provides as output information characterizing risk of the log segments. The processing device is also configured to identify critical areas of the log file based at least in part on the determined risk scores, a given critical area comprising a sequence of log segments having determined risk scores above a designated risk score threshold. The processing device is further configured to analyze the identified critical areas to determine remedial actions to be applied for resolving the service request.

Abstract: An exemplary embodiment may describe a convolutional explainable neural network. A CNN-XNN may receive input, such as 2D or multi-dimensional data, a patient history, or any other relevant information. The input data is segmented into various objects and a knowledge encoding layer may identify and extract various features from the segmented objects. The features may be weighted. An output layer may provide predictions and explanations based on the previous layers. The explanation may be determined using a reverse indexing mechanism (Backmap). The explanation may be processed using a Kernel Labeler method that allows the labelling of the progressive refinement of patterns, symbols and concepts from any data format that allows a pattern recognition kernel to be defined allowing integration of neurosymbolic processing within CNN-XNNs. The optional addition of meta-data and causal logic allows for the integration of connectionist models with symbolic logic processing.