Abstract: A method of online batch normalization, on-device learning, or continual learning which are applicable to mobile devices, IoT devices, and the like is provided. The method includes steps of: (a) computing device instructing convolutional layer to acquire k-th batch, and to generate feature maps for k-th batch by applying convolution operations to input images included in k-th batch respectively; and (b) computing device instructing batch normalization layer to calculate adjusted averages and adjusted variations of the feature maps by referring to the feature maps in case k is 1, and the feature maps and previous feature maps, included in at least part of previous batches among previously generated first to (k?1)-th batches in case k is integer from 2 to m, and to apply batch normalization operations to the feature maps. Further, the method may be performed for military purpose, or other devices such as drones, robots.

Abstract: Providing an end-to-end citizen engagement, in one aspect, may comprise obtaining data of multiple disintegrated sources from one or more of communication and social computing channels via one or more adapters. Data refactoring and management, integration and process orchestration of the data according to a data model as data attributes of the data model may be provided. One or more analytics may be performed based on the data attributes stored according to the data model and input specified to the one or more analytics. One or more results computed by performing the one or more analytics may be provided. One or more application logics supporting one or more front-end applications may be produced. One or more front-end applications for automated sensing of user activities and sensor-based personal assistant capability may be provided.

Abstract: A thermodynamic RAM technology stack, two or more memristors or pairs of memristors comprising AHaH (Anti-Hebbian and Hebbian) computing components, and one or more AHaH nodes composed of such memristor pairs that form at least a portion of the thermodynamic RAM technology stack. The levels of the thermodynamic-RAM technology stack include the memristor, a Knowm synapse, an AHaH node, a kT-RAM, kT-RAM instruction set, a sparse spike encoding, a kT-RAM emulator, and a SENSE Server.

Abstract: A method and system for constructing a classifier for a set of records to be classified (108) into predicted classes (351-353) are provided. The set of records that are to be classified (108) are clustered into a plurality of clusters. A first classifier (106) is created that classifies records into the plurality of clusters (321-323) and the first classifier (106) is applied to a set of training records (110), each of the training records (331-334) having a predicted class (306). A classifier (107A-C) may then be created for each sub-set of training records (341-343) classed into each of the plurality of clusters and the classifier (107A-C) applied to a sub-set of records to be classified (311-313) formed in the corresponding cluster.

Abstract: An image recognizing apparatus includes a processor that controls first and second learning processes, the first learning process in second layers including holding, based on a large/small relation between neuron data size and parameter size of the second layer, in a memory area, an error gradient of parameters to be sent to the corresponding layer of the second layers; and the second learning process between first layers including holding, in a memory area of each first layers, an error gradient of parameters to be sent to the corresponding layer of the first layers, which is computed based on the error gradient or an error gradient of a previous layer of the first layers, based on a large/small relation between neuron data size and parameter size of the first layer.

Abstract: Time-varying input signals are denoised by a neural network. The neural network learns features associated with noise added to reference signals. The neural network recognizes features of noisy time-varying input signals mixed with the noise that at least partially match at least some of the features associated with the noise. The neural network predicts denoised time-varying output signals that correspond to the time-varying input signals based on the recognized features of the noisy time-varying input signals that at least partially match at least some of the features associated with the noise.

Abstract: A system and method are disclosed for determining and alerting a user as to whether an object will successfully scan before the post-processing of the scan data. In embodiments, before post-processing of the scan data begins, the scan data is processed by a machine learning algorithm which is able to determine whether and/or how likely the scan data is to return an accurate scanned reproduction of the scanned object. The machine learning algorithm may also suggest new positions for the object in the environment where the scan is more likely to be successful.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a target sequence from a source sequence. In one aspect, the system includes a recurrent neural network configured to, at each time step, receive am input for the time step and process the input to generate a progress score and a set of output scores; and a subsystem configured to, at each time step, generate the recurrent neural network input and provide the input to the recurrent neural network; determine, from the progress score, whether or not to emit a new output at the time step; and, in response to determining to emit a new output, select an output using the output scores and emit the selected output as the output at a next position in the output order.

Abstract: Embodiments of the invention relate to a neural network system for simulating neurons of a neural model. One embodiment comprises a memory device that maintains neuronal states for multiple neurons, a lookup table that maintains state transition information for multiple neuronal states, and a controller unit that manages the memory device. The controller unit updates a neuronal state for each neuron based on incoming spike events targeting said neuron and state transition information corresponding to said neuronal state.

Abstract: A processor has an instruction fetch unit that fetches ISA instructions from memory and execution units that perform operations on instruction operands to generate results according to the processor's ISA. A hardware neural network unit (NNU) execution unit performs computations associated with artificial neural networks (ANN). The NNU has an array of ALUs, a first memory that holds data words associated with ANN neuron outputs, and a second memory that holds weight words associated with connections between ANN neurons. Each ALU multiplies a portion of the data words by a portion of the weight words to generate products and accumulates the products in an accumulator as an accumulated value. Activation function units normalize the accumulated values to generate outputs associated with ANN neurons. The ISA includes at least one instruction that instructs the processor to write data words and the weight words to the respective first and second memories.

Abstract: A computing system having a computational memory and a method configured to perform computations using an approximate message passing process. The system exploits memcomputing which is a prominent non-von Neumann computational approach expected to significantly improve an energy efficiency of computing systems. The computational memory includes at least one memristive array comprising a plurality of memristive devices arranged in a crossbar topology and the computing system may further comprise digital combinational control circuitry adapted to perform read and write operations on the at least one memristive array and to store at least one state variable of the approximate message passing process. An output of the at least one memristive array represents a result of a computation of the approximate message passing process. The control circuitry may comprise circuitry to iteratively perform computations that may not require high precision.

Abstract: Embodiments are directed to a two-terminal resistive processing unit (RPU) having a first terminal, a second terminal and an active region. The active region effects a non-linear change in a conduction state of the active region based on at least one first encoded signal applied to the first terminal and at least one second encoded signal applied to the second terminal. The active region is configured to locally perform a data storage operation of a training methodology based at least in part on the non-linear change in the conduction state. The active region is further configured to locally perform a data processing operation of the training methodology based at least in part on the non-linear change in the conduction state.

Abstract: Embodiments of the invention relate to mapping neural dynamics of a neural model on to a lookup table. One embodiment comprises defining a phase plane for a neural model. The phase plane represents neural dynamics of the neural model. The phase plane is coarsely sampled to obtain state transition information for multiple neuronal states. The state transition information is mapped on to a lookup table.

Abstract: Embodiments of the present invention provide a method for feature extraction comprising generating synaptic connectivity information for a neurosynaptic core circuit. The core circuit comprises one or more electronic neurons, one or more electronic axons, and an interconnect fabric including a plurality of synapse devices for interconnecting the neurons with the axons. The method further comprises initializing the interconnect fabric based on the synaptic connectivity information generated, and extracting a set of features from input received via the electronic axons. The set of features extracted comprises a set of features with reduced correlation.

Abstract: Disclosed are various embodiments of memristive neural networks comprising neural nodes. Memristive nanofibers are used to form artificial synapses in the neural networks. Each memristive nanofiber may couple one or more neural nodes to one or more other neural nodes. In one case, a memristive neural network includes a first neural node, a second neural node, and a memristive fiber that couples the first neural node to the second neural node. The memristive fiber comprises a conductive core and a memristive shell, where the conductive core forms a communications path between the first neural node and the second neural node and the memristive shell forms a memristor synapse between the first neural node and the second neural node.

Abstract: Embodiments of the invention relate to a scalable neural hardware for the noisy-OR model of Bayesian networks. One embodiment comprises a neural core circuit including a pseudo-random number generator for generating random numbers. The neural core circuit further comprises a plurality of incoming electronic axons, a plurality of neural modules, and a plurality of electronic synapses interconnecting the axons to the neural modules. Each synapse interconnects an axon with a neural module. Each neural module receives incoming spikes from interconnected axons. Each neural module represents a noisy-OR gate. Each neural module spikes probabilistically based on at least one random number generated by the pseudo-random number generator unit.

Abstract: Physical synthesis for a circuit design can include determining, using a processor, features relating to a signal path of the circuit design not meeting a timing requirement, processing the features through a first neural network model using the processor, wherein the first neural network model is trained to indicate an effectiveness of a first physical synthesis optimization, and selectively performing, using the processor, the first physical synthesis optimization for the signal path based upon a result from the first neural network model.

Abstract: One embodiment provides for a compute apparatus to perform machine learning operations, the compute apparatus comprising a decode unit to decode a single instruction into a decoded instruction, the decoded instruction to cause the compute apparatus to perform a complex machine learning compute operation.

Abstract: The advancements of the Internet of Things and the big data analytics systems demand new model for analyzing large volumes of data from a plurality of software systems, machines and embedded sensors used for a plurality of application areas such as natural ecosystems, bioinformatics, smart homes, smart cities, automobiles and airplanes. These complex systems need efficient methods for near real time collection, processing, analysis and sharing of data from and among the plurality of sensors, machines and humans. This invention identities and proposes implementation of a new model (CALSTATDN) for machine learning over large volumes of data combining methods of calculus (CAL), statistics (STAT) and database normalization (DN) in order to reduce error in learning process and to increase performance by several orders of magnitude. This invention further describes machine learning techniques for storing and processing of high speed real-time streaming data with variations in time, space and other dimensions.

Abstract: A neuromorphic memory system including neuromorphic memory arrays. Each neuromorphic memory array includes rows and columns of neuromorphic memory cells. A column of postsynaptic circuits is electrically coupled to postsynaptic spike timing dependent plasticity (STDP) lines. Each postsynaptic STDP line is coupled to a row of neuromorphic memory cells. A column of summing circuits is electrically coupled to postsynaptic leaky integrate and fire (LIF) lines. Each postsynaptic LIF line is coupled to the row of neuromorphic memory cells at a respective memory array. Each summing circuit provides a sum of signals from the postsynaptic LIF lines to a respective postsynaptic circuit.

Abstract: This present disclosure relates to systems and methods for providing an Adaptive Analytical Behavioral and Health Assistant. These systems and methods may include collecting one or more of patient behavior information, clinical information, or personal information; learning one or more patterns that cause an event based on the collected information and one or more pattern recognition algorithms; identifying one or more interventions to prevent the event from occurring or to facilitate the event based on the learned patterns; preparing a plan based on the collected information and the identified interventions; and/or presenting the plan to a user or executing the plan.

Abstract: What is disclosed is a system and method for modelling a class posterior probability of context dependent phonemes in a speech recognition system. A representation network is trained by projecting a N-dimensional feature vector into G intermediate layers of nodes. At least some features are associated with a class label vector. A last intermediate layer ZG of the representation network is discretized to obtain a discretized layer {circumflex over (Z)}. Feature vector Q is obtained by randomly selecting V features from discretized layer {circumflex over (Z)}. Q is repeatedly hashed to obtain a vector Qf where Qf is an output of the fth hashing. An equivalent scalar representation is determined for each Qf. In a manner more fully disclosed herein, a posterior probability Pf is determined for each (x, b) pair based on the equivalent scalar representation of each respective Qf. The obtained posterior probabilities are used to improve classification accuracy in a speech recognition system.

Abstract: First/second memories hold rows of N weight/data words. The first memory address has log2 W bits and an extra bit. Each of N processing units (PU) of index J has first and second registers, an accumulator, an arithmetic unit performs an operation thereon to accumulate a result, first multiplexing logic for PUs 0 through (N/2)?1 receives first memory weight words J and J+(N/2) and for PUs N/2 through N?1 receives first memory weight words J and J?(N/2) and outputs a selected weight word to the first register, and second multiplexing logic receives second memory data word J and data word output by the second register of PU J?1 and outputs a selected data word to the second register. PU 0 second multiplexing logic also receives PU (N/2)?1 second register data word, and PU N/2 second multiplexing logic also receives PU N?1 second register data word.

Abstract: Machine learning (ML) significantly reduces false alarms generated by an automated analysis tool performing static security analysis. Using either user-supplied or system-generated annotation of particular findings, a “hypothesis” is generated about how to classify other static analysis findings. The hypothesis is implemented as a machine learning classifier. To generate the classifier, a set of features are abstracted from a typical witness, and the system compares feature sets against one another to determine a set of weights for the classifier. The initial hypothesis is then validated against a second set of findings, and the classifier is adjusted as necessary based on how close it fits the new data. Once the approach converges on a final classifier, it is used to filter remaining findings in the report.

Abstract: Disclosed are systems, methods, circuits and associated computer executable code for deep learning based natural language understanding, wherein training of one or more neural networks, includes: producing character strings inputs ‘noise’ on a per-character basis, and introducing the produced ‘noise’ into machine training character strings inputs fed to a ‘word tokenization and spelling correction language-model’, to generate spell corrected word sets outputs; feeding machine training word sets inputs, including one or more ‘right’ examples of correctly semantically-tagged word sets, to a ‘word semantics derivation model’, to generate semantically tagged sentences outputs. Upon models reaching a training ‘steady state’, the ‘word tokenization and spelling correction language-model’ is fed with input character strings representing ‘real’ linguistic user inputs, generating word sets outputs that are fed as inputs to the word semantics derivation model for generating semantically tagged sentences outputs.

Abstract: A contribution value necessary for achieving a target accuracy from a correct answer probability of each participant is calculated, a contribution value of the participant is added for an answer in accordance with the calculation, and it is set at a condition for determining completion of a task, that is, determining that a correct answer is obtained and no additional participant is necessary. The contribution value is calculated as the inverse of the number of participants at which the target accuracy is reached with a predetermined correct answer probability. The contribution value of the participant is added to the contribution value for the task in which that participant participates. At the time when the sum of the contribution values for a task exceeds one or when one option is certain to be a correct answer, the result having the largest sum of the contribution values is output.

Abstract: A circuit for performing neural network computations for a neural network comprising a plurality of layers, the circuit comprising: activation circuitry configured to receive a vector of accumulated values and configured to apply a function to each accumulated value to generate a vector of activation values; and normalization circuitry coupled to the activation circuitry and configured to generate a respective normalized value from each activation value.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for classification using a neural network. One of the methods for processing an input through each of multiple layers of a neural network to generate an output, wherein each of the multiple layers of the neural network includes a respective multiple nodes includes for a particular layer of the multiple layers: receiving, by a classification system, an activation vector as input for the particular layer, selecting one or more nodes in the particular layer using the activation vector and a hash table that maps numeric values to nodes in the particular layer, and processing the activation vector using the selected nodes to generate an output for the particular layer.

Abstract: A system for mitigating network attacks is provided. The system includes a protected network including a plurality of devices. The system further includes one or more attack mitigation devices communicatively coupled to the protected network. The attack mitigation devices are configured and operable to employ a recurrent neural network (RNN) to obtain probability information related to a request stream. The request stream may include a plurality of at least one of: HTTP, RTSP and/or DNS messages. The attack mitigation devices are further configured to analyze the obtained probability information to detect one or more atypical requests in the request stream. The attack mitigation services are also configured and operable to perform, in response to detecting one or more atypical requests, mitigation actions on the one or more atypical requests in order to block an attack.

Abstract: An algorithm for modeling and optimizing control of a complex and dynamic system is provided to facilitate an allocation of the resources on the network that is the most efficient. The algorithm serves to depict the complex network of available resources using market-based negotiation wherein resources are defined as available buyers and sellers in an efficient market. Selling agents are offering their available resources for sale in accordance with parameters that correspond to the actual limitations of that actual resource and the buyers are looking to make a purchase from one of the sellers that presents a resource with the greatest utility to them. In order to overcome inefficiencies that result from the potential of inefficient allocation, the present invention has further endeavored to introduce an efficiency-arbitrage agent that scans the overall body of transactions to identify and remedy inefficient market transactions.

Abstract: Disclosed are a data processing method, a processor, and a data processing device. The method comprises: an arbiter sends data D(a,1) to a first processing circuit; the first processing circuit processes the data D(a,1) to obtain data D(1,2), the first processing circuit being a processing circuit among m processing circuits; the first processing circuit sends the data D(1,2) to a second processing circuit; the second processing circuit to an mth processing circuit separately process the received data; and the arbiter receives data D(m,a) sent by the mth processing circuit. The processor comprises an arbiter and a first processing circuit to an (m+1)th processing circuit. Each processing circuit in the first processing circuit to the (m+1)th processing circuit can receive first data to be processed sent by the arbiter, and process the first data to be processed. The scheme is helpful to improve efficiency of data processing.

Abstract: A system and method for dynamic caching of content of sites accessed over a network by a user is provided. The system includes a processor, a first storage device for maintaining cache accounts for storing the content of the sites accessed over the network by the user based on activity over the network by the user with the sites, a second storage device for storing statistics, and a non-transitory physical medium. The medium has instructions stored thereon that, when executed by the processor, causes the processor to gather statistics on suitability of the sites for caching based on the network activity, store the caching suitability statistics on the second storage device, and dynamically create, delete, or resize the cache accounts based on the caching suitability statistics.

Abstract: An example device may include multiply-accumulate circuitry and voltage-tracking modulator circuitry. The multiply-accumulate circuitry may be to increase and decrease an accumulation voltage held by an accumulator based on a number of input signals. The voltage-tracking modulator circuitry may be to generate an output signal based on the accumulation voltage, wherein the output signal is a continuous-time binary signal that tracks changes of the accumulation voltage by varying pulse widths of the output signal. The example device may be used as a neuron in a neural network.

Abstract: Embodiments of the present invention are directed to providing new systems and methods for using deep learning techniques to generate embeddings for high dimensional data objects that can both simulate prior art embedding algorithms and also provide superior performance compared to the prior art methods.

Abstract: In one embodiment, a neuromorphic robot includes a curved outer housing, and multiple touch sensors provided on the outer housing, wherein the robot is configured to interpret a touch of a user sensed with the touch sensors.

Abstract: Certain aspects of the present disclosure support assigning neurons and/or synapses to group tags where group tags have an associated set of parameters. By using group tags, neurons or synapses in a population can be assigned a group tag. Then, by changing a parameter associated with the group tag, all synapses or neurons in the group may have that parameter changed.

Abstract: Methods and systems for feature extraction of LIDAR surface manifolds. LIDAR point data with respect to one or more LIDAR surface manifolds can be generated. An AHAH-based feature extraction operation can be automatically performed on the point data for compression and processing thereof. The results of the AHAH-based feature extraction operation can be output as a compressed binary label representative of the at least one surface manifold rather than the point data to afford a high-degree of compression for transmission or further processing thereof. Additionally, one or more voxels of a LIDAR point cloud composed of the point data can be scanned in order to recover the compressed binary label, which represents prototypical surface patches with respect to the LIDAR surface manifold(s).

Abstract: A method for scalable user intent mining is provided. The method includes detecting named entities from a plurality of query logs in a public query log dataset and generating features of the plurality of query logs based on the detected named entities. The method also includes applying a multimodal restricted boltzmann machine (RBM) on the generated features of the plurality of query logs to train a public multimodal RBM and generating a plurality of public query representations. Further, the method includes receiving a search query from a user, determining whether there are a plurality of history queries of the user. When there is no history query, user intent is predicted using the public multimodal RBM. When there are the history queries, the public multimodal RBM is applied on the plurality of history queries to train a personalized multimodal RBM, and the user intent is predicted using the personalized multimodal RBM.

Abstract: A pre-training apparatus and method for recognition speech, which initialize, by layers, a deep neural network to correct a node connection weight. The pre-training apparatus for speech recognition includes an input unit configured to receive speech data, a model generation unit configured to initialize a connection weight of a deep neural network, based on the speech data, and an output unit configured to output information about the connection weight. In order for a state of a phoneme unit corresponding to the speech data to be output, the model generation unit trains the connection weight by piling a plurality of hidden layers according to a determined structure of the deep neural network, applies an output layer to a certain layer between the plurality of hidden layers to correct the trained connection weight in each of the plurality of hidden layers, thereby initializing the connection weight.

Abstract: A method of processing information is provided. The method involves receiving a message; processing the message with a trained artificial neural network based processor, having at least one set of outputs which represent information in a non-arbitrary organization of actions based on an architecture of the artificial neural network based processor and the training; representing as a noise vector at least one data pattern in the message which is incompletely represented in the non-arbitrary organization of actions; analyzing the noise vector distinctly from the trained artificial neural network; searching at least one database; and generating an output in dependence on said analyzing and said searching.

Abstract: A system and method for dynamic caching of content of sites accessed over a network by a user is provided. The system includes a processor, a first storage device for maintaining cache accounts for storing the content of the sites accessed over the network by the user based on activity over the network by the user with the sites, a second storage device for storing statistics, and a non-transitory physical medium. The medium has instructions stored thereon that, when executed by the processor, causes the processor to gather statistics on suitability of the sites for caching based on the network activity, store the caching suitability statistics on the second storage device, and dynamically create, delete, or resize the cache accounts based on the caching suitability statistics.

Abstract: In one embodiment, a first data set is received by a network device that is indicative of the statuses of a plurality of network devices when a type of network attack is not present. A second data set is also received that is indicative of the statuses of the plurality of network devices when the type of network attack is present. At least one of the plurality simulates the type of network attack by operating as an attacking node. A machine learning model is trained using the first and second data set to identify the type of network attack. A real network attack is then identified using the trained machine learning model.

Abstract: A neural interface for measuring or stimulating brain neural activity, either as a standalone unit or as a part of a larger system of similar neural interfaces. The neural interface includes a bolt-shaped housing having a tool-engaging head and threaded shank with internal circuitry and at least one electrode. In use, the housing is threaded into a cranial bore such that the electrode contacts the outer surface of the meninges. The neural interface circuitry includes an SAR ADC that provides at least rail-to-rail operation to convert received signals from the electrode(s) into digital data that can be modulated and wirelessly transmitted by intra-skin or other suitable communication.

Abstract: A method for encoding computer processes for malicious program detection.

Abstract: In an example, a memristor apparatus with variable transmission delay may include a first memristor programmable to have one of a plurality of distinct resistance levels, a second memristor, a transistor connected between the first memristor and the second memristor, and a capacitor having a capacitance, in which the capacitor is connected between the first memristor and the transistor. In addition, application of a reading voltage across the second memristor is delayed by a time period equivalent to the programmed resistance level of the first memristor and the capacitance of the capacitor.

Abstract: Deep learning is used to identify a potential risk that a contract will be unenforceable due to a drafting error whereby one or more terms or phrases are ambiguous. The system involves mining and using existing classifications of data (e.g., from a litigation database) to train one or more deep learning algorithms, and then examining the internal electronic drafts of contracts with the trained algorithm, to generate a scored output that will enable enterprise personnel to be alerted to the ambiguity risks and take action in time to prevent the risks from resulting in harm to the enterprise.

Abstract: Deep learning is used to identify specific, potential risks to an enterprise (of which product liability is the prime example here) while such risks are still internal electronic communications. The system involves mining and using existing classifications of data (e.g., from an internal litigation database, or from external sources such as customer complaints, and/or warranty claims) to train one or more deep learning algorithms, and then examining the enterprise's internal electronic communications with the trained algorithm, to generate a scored output that will enable enterprise personnel to be alerted to risks and take action in time to prevent the risks from resulting in harm to the enterprise or others.

Abstract: As part of an analysis of the likelihood that a given input (e.g. a file, etc.) includes malicious code, a convolutional neural network can be used to review a sequence of chunks into which an input is divided to assess how best to navigate through the input and to classify parts of the input in a most optimal manner. At least some of the sequence of chunks can be further examined using a recurrent neural network in series with the convolutional neural network to determine how to progress through the sequence of chunks. A state of the at least some of the chunks examined using the recurrent neural network summarized to form an output indicative of the likelihood that the input includes malicious code. Methods, systems, and articles of manufacture are also described.

Abstract: A computing unit is disclosed, comprising a first memory bank for storing input activations and a second memory bank for storing parameters used in performing computations. The computing unit includes at least one cell comprising at least one multiply accumulate (“MAC”) operator that receives parameters from the second memory bank and performs computations. The computing unit further includes a first traversal unit that provides a control signal to the first memory bank to cause an input activation to be provided to a data bus accessible by the MAC operator. The computing unit performs one or more computations associated with at least one element of a data array, the one or more computations being performed by the MAC operator and comprising, in part, a multiply operation of the input activation received from the data bus and a parameter received from the second memory bank.

Abstract: One embodiment of the invention provides a method comprising defining a brainlet representing a platform-agnostic network of neurons, synapses, and axons. The method further comprises compiling the brainlet into a corelet for mapping onto neurosynaptic substrate, and mapping the corelet onto the neurosynaptic substrate. The corelet is compatible with one or more conditions related to the neurosynaptic substrate.