Abstract: Artificial neuromorphic circuit includes synapse circuit and post-neuron circuit. Synapse circuit includes phase change element, first switch, and second switch. Phase change element includes first terminal and second terminal. First switch includes first terminal and second terminal. Second switch includes first terminal, second terminal, and control terminal. First switch is configured to receive first pulse signal. Second switch is coupled to phase change element and first switch. Second switch is configured to receive second pulse signal. Post-neuron circuit includes capacitor and input terminal. Input terminal of post-neuron circuit charges capacitor in response to first pulse signal. Post-neuron circuit generates firing signal based on voltage level of capacitor and threshold voltage. Post-neuron circuit generates control signal based on firing signal. Control signal controls turning on of second switch.

Abstract: A neural processing system includes a first frontend module, a second frontend module, a first backend module, and a second backend module. The first frontend module executes a feature extraction operation using a first feature map and a first weight, and outputs a first operation result and a second operation result. The second frontend module executes the feature extraction operation using a second feature map and a second weight, and outputs a third operation result and a fourth operation result. The first backend module receives an input of the first operation result provided from the first frontend module and the fourth operation result provided from the second frontend module via a second bridge to sum up the first operation result and the fourth operation result.

Abstract: A method and system for executing processes in an operating system is disclosed. The method may include assigning a tag Identifier (ID) and a first tree ID to each of a plurality of processes based on associated at least one attribute. The method may further include determining content patterns associated with each of the plurality of processes using a deep learning network. The method may include assigning a second tree ID to each of at least one process from the plurality of processes based on the identified content pattern. The method may further include generating a set of clusters for the plurality of processes based on the second tree ID assigned to each of the at least one process. The method may include executing each process within a cluster from the set of clusters based on execution of a single process within the cluster.

Abstract: Herein are techniques for domain adaptation of a machine learning (ML) model. These techniques impose differential privacy onto federated learning by the ML model. In an embodiment, each of many client devices receive, from a server, coefficients of a general ML model. For respective new data point(s), each client device operates as follows. Based on the new data point(s), a respective private ML model is trained. Based on the new data point(s), respective gradients are calculated for the coefficients of the general ML model. Random noise is added to the gradients to generate respective noisy gradients. A combined inference may be generated based on: the private ML model, the general ML model, and one of the new data point(s). The noisy gradients are sent to the server. The server adjusts the general ML model based on the noisy gradients from the client devices. This client/server process may be repeated indefinitely.

Abstract: A neural architecture search system for generating a neural network includes one or more processors and a memory. The memory includes a generator module, a self-supervised training module, and an output module. The modules cause the one or more processors to generate a candidate neural network by a controller neural network, obtain training data, generate an output by the candidate neural network performing a specific task using the training data as an input, determine a loss value using a loss function that considers the output of the candidate neural network and at least a portion of the training data, adjust the one or more model weights of the controller neural network based on the loss value, and output the candidate neural network. The candidate neural network may be derived from the controller neural network and one or more model weights of the controller neural network.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training and using machine learning clustering models to determine conditions a satellite communication system. In some implementations, feature vectors for a time period are obtained. Each feature vector includes feature values that represent properties of a satellite communication system at a respective time during the time period. Each feature vector is provided as input to a machine learning model that assigns the feature vector to a based on the properties of the satellite communication system represented by the feature vector. Each cluster corresponds to a respective potential operating condition of the satellite communication system.

Abstract: A packer classification apparatus extracts features based on a section that holds packer information from files and classifies packers using a Deep Neural Network(DNN) for detection of new/variant packers. A packer classification apparatus according to an embodiment uses PE section information. packer classification apparatus includes a collection classification module collecting a data set and classifying data by packer type to prepare for a model learning, a token hash module tokenizing a character string obtained after extracting labels and section names of each data and combining the section names, and obtaining a certain standard output value using Feature Hashing, and a type classification module generating a learning model after learning the data set with a Deep Neural Network(DNN) algorithm using extracted features, and classifying files for each packer type using the learning model after extracting features for the files to be classified.

Abstract: A system and method design and optimize neural networks. The system and method include a data store that stores a plurality of gene vectors that represent diverse and distinct neural networks and an evaluation queue stored with the plurality of gene vectors. Secondary nodes construct, train, and evaluate the neural network and automatically render a plurality of fitness values asynchronously. A primary node executes a gene amplification on a select plurality of gene vectors, a crossing-over of the amplified gene vectors, and a mutation of the crossing-over gene vectors automatically and asynchronously, which are then transmitted to the evaluation queue. The process continuously repeats itself by processing the gene vectors inserted into the evaluation queue until a fitness level is reached, a network's accuracy level plateaus, a processing time period expires, or when some stopping condition or performance metric is met or exceeded.

Abstract: Some embodiments provide an electronic device that includes a set of processing units and a set of machine-readable media. The set of machine-readable media stores sets of instructions for applying a network of computation nodes to an input received by the device. The set of machine-readable media stores at least two sets of machine-trained parameters for configuring the network for different types of inputs. A first of the sets of parameters is used for applying the network to a first type of input and a second of the sets of parameters is used for applying the network to a second type of input.

Abstract: A method of performing convolution in a neural network with variable dilation rate is provided. The method includes receiving a size of a first kernel and a dilation rate, determining at least one of size of one or more disintegrated kernels based on the size of the first kernel, a baseline architecture of a memory and the dilation rate, determining an address of one or more blocks of an input image based on the dilation rate, and one or more parameters associated with a size of the input image and the memory. Thereafter, the one or more blocks of the input image and the one or more disintegrated kernels are fetched from the memory, and an output image is obtained based on convolution of each of the one or more disintegrated kernels and the one or more blocks of the input image.

Abstract: Fair deep reinforcement learning is provided. A microstate of an environment and reaction of items in a plurality of microstates within the environment are observed after an agent performs an action in the environment. Semi-supervised training is utilized to determine bias weights corresponding to the action for the microstate of the environment and the reaction of the items in the plurality of microstates within the environment. The bias weights from the semi-supervised training are merged with non-bias weights using an artificial neural network. Over time, it is determined where bias is occurring in the semi-supervised training based on merging the bias weights with the non-bias weights in the artificial neural network. A deep reinforcement learning model that decreases reliance on the bias weights is generated based on determined bias to increase fairness.

Abstract: A computational system can include digital circuitry and analog circuitry, for instance a digital processor and a quantum processor. The quantum processor can operate as a sample generator providing samples. Samples can be employed by the digital processing in implementing various machine learning techniques. For example, the digital processor can operate as a restricted Boltzmann machine. The computational system can operate as a quantum-based deep belief network operating on a training data-set.

Abstract: A method includes providing a set of deep learning neural networks where no pair of deep learning neural networks within the set of deep learning neural networks produces a semantically equivalent output by design. An input to the set of deep learning neural networks is provided, where responsive to the input, one or more of the deep learning neural networks produces an output. The output of the deep learning neural networks is input into a case-based reasoning (CBR) system. The CBR system generates an output responsive to the input received by the CBR system if the input received by the CBR system is known by the CBR system. The output of the CBR system is then determined to be a correct/incorrect output. One of the deep learning neural networks is trained on the correct output if the correct output is specific to the particular deep learning neural network.

Abstract: A machine learning system includes a coach machine learning system that uses machine learning to help a student machine learning system learn its system. By monitoring the student learning system, the coach machine learning system can learn (through machine learning techniques) “hyperparameters” for the student learning system that control the machine learning process for the student learning system. The machine learning coach could also determine structural modifications for the student learning system architecture. The learning coach can also control data flow to the student learning system.

Abstract: A method may include a transforming a trained machine learning model including by replacing at least one layer of the trained machine learning model with a dictionary matrix and a coefficient matrix. The dictionary matrix and the coefficient matrix may be formed by decomposing a weight matrix associated with the at least one layer of the trained machine learning model. A product of the dictionary matrix and the coefficient matrix may form a reduced-dimension representation of the weight matrix associated with the at least one layer of the trained machine learning model. The transformed machine learning model may be deployed to a client. Related systems and computer program products are also provided.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for domain-specific pruning of neural networks are described. An exemplary method includes obtaining a first neural network trained based on a first training dataset; obtaining one or more second training datasets respectively from one or more domains; and training, based on the first neural network and the one or more second training datasets, a second neural network comprising the first neural network and one or more branches extended from the first neural network, wherein the second neural network is applicable for inferencing in the one or more domains, and the training comprises: training the one or more branches based respectively on the one or more second training datasets and an output of the first neural network.

Abstract: One embodiment provides for a method of transmitting data between multiple compute nodes of a distributed compute system, the method comprising multi-dimensionally partitioning data of a feature map across multiple nodes for distributed training of a convolutional neural network; performing a parallel convolution operation on the multiple partitions to train weight data of the neural network; and exchanging data between nodes to enable computation of halo regions, the halo regions having dependencies on data processed by a different node.

Abstract: A reinforcement learning device includes a processor that determines a first action on a control target by using a basic controller that defines an action on the control target depending on a state of the control target. The processor performs a first reinforcement learning within a first action range around the first action in order to acquire a first policy for determining an action on the control target depending on a state of the control target. The first action range is smaller than a limit action range for the control target. The processor determines a second action on the control target by using the first policy. The processor updates the first policy to a second policy by performing a second reinforcement learning within a second action range around the second action. The second action range is smaller than the limit action range.

Abstract: A video on demand (VOD) service system is based on an artificial intelligence (AI) video learning platform. A VOD service system based on an AI video learning platform may perform video learning according to AI-based Super Resolution Convolutional Neural Networks (SRCNNs) to calculate a weight required for restoring a high image quality video from a high image quality VOD file, and then restore a low image quality VOD file to a high image quality VOD file using the calculated weight corresponding to the VOD file later on.

Abstract: Described is a system for learning and predicting key phrases. The system learns based on a dataset of historical forecasting questions, their associated time-series data for a quantity of interest, and associated keyword sets. The system learns the optimal policy of actions to take given the associated keyword sets and the optimal set of keywords which are predictive of the quantity of interest. Given a new forecasting question, the system extracts an initial keyword set from a new forecasting question, which are perturbed to generate an optimal predictive key-phrase set. Key-phrase time-series data are extracted for the optimal predictive key-phrase set, which are used to generate a forecast of future values for a value of interest. The forecast can be used for a variety of purposes, such as advertising online.

Abstract: A computer system receives and stores data sets, a target metric, and a parameter that indicates a desired number of synthesized data sets. A hardware processor performs operations where each processing node of a neural network weights input data set values, determines gating operations to select processing operations, and generates a node output by applying the gating operations to weighted input data set values. The neural network is trained by modifying the gating operations, the input weight values, and the node output weight value until convergence. One or more nodes is selected having a larger magnitude node output weight value. Selected input data set values are processed with selected processing nodes using a selected subset of gating operations to produce the desired number of synthesized data sets. Names are generated for each of the synthesized data sets.

Abstract: Systems, apparatuses and methods may provide for a chip that includes a memory array having a plurality of rows corresponding to neurons in a spiking neural network (SNN) and a row decoder coupled to the memory array, wherein the row decoder activates a row in the memory array in response to a pre-synaptic spike in a neuron associated with the row. Additionally, the chip may include a sense amplifier coupled to the memory array, wherein the sense amplifier determines post-synaptic information corresponding to the activated row. In one example, the chip includes a processor to determine a state of a plurality of neurons in the SNN based at least in part on the post-synaptic information and conduct a memory array update, via the sense amplifier, of one or more synaptic weights in the memory array based on the state of the plurality of neurons.

Abstract: A system for detecting an anomaly in an execution of an operation of a machine determines a local matrix profile (LMP) of a test signal with respect to the baseline signals. LMP is a time series of values, each LMP value for a time instance is determined for a segment of the test signal based on a minimum distance between the segment of the test signal with corresponding segments of the baseline signals, such that each LMP value is a value of a local dissimilarity of the execution of the operation of the machine with respect to the baseline executions of the operation of the machine. The system determines an accumulation of the LMP values above an LMP threshold and detects an anomaly when the accumulation above an anomaly detection threshold.

Abstract: A method for unsupervised sorting, in real time, of action potentials of biological neurons by a network of artificial neurons including input, intermediate and output layers, the method according to which: the input layer receives an electrical signal measuring an electrical activity of biological neurons, the electrical signal having a variable amplitude as a function of action potentials emitted by the plurality of biological neurons over time; the input layer converts the amplitude of the electrical signal into a train of first spikes; the input layer transmits the train of first spikes to the intermediate layer; the intermediate layer converts the train of first spikes into a train of second spikes; the intermediate layer transmits the train of second spikes to the output layer; as a function of the train of second spikes, the output layer sorts each occurrence of each type of action potential present in the electrical signal.

Abstract: The invention relates to a system (1) for generating moving light effects, having a display (2) comprising a plurality of lighting units (10a), which is designed to emit one or more temporally changing color patterns (4, 4*, 4**), a peripheral device (6) comprising a lighting arrangement (9) having one or more lighting units (10b), and a control module (11) formed for actuating the lighting units (10a) of the display (2) and the one or more lighting units (10b) of the peripheral device (6). Said control module is designed to operate the one or more lighting units (10b) of the peripheral device (6) and the lighting units (10a) of the display (2) generating the color pattern (4, 4*) in one-sided or mutual dependence on one another. The invention further relates to a method for generating moving light effects and to a salesroom having such a system (1).

Abstract: A composition provides a protective barrier and includes glyceryl dibehenate, tribehenin, and glyceryl behenate, a surfactant that includes glyceryl stearate, one or more fatty compounds, and one or more triglyceride. The composition optionally includes a hydrating agent and may be essentially free of or is devoid of one or more of petrolatum, mineral oil and water. The composition provides occlusivity and hydration effects that are comparable to compositions that include one or more of petrolatum, mineral oil and lanolin.

Abstract: A vehicle having the first ANN model initially installed therein to generate outputs from inputs generated by one or more sensors of the vehicle. The vehicle selects an input based on an output generated from the input using the first ANN model. The vehicle has a module to incrementally train the first ANN model through unsupervised machine learning from sensor data that includes the input selected by the vehicle. Optionally, the sensor data used for the unsupervised learning may further include inputs selected by other vehicles in a population. Sensor inputs selected by vehicles are transmitted to a centralized computer server, which trains the first ANN model through supervised machine learning from sensor received inputs from the vehicles in the population and generates a second ANN model as replacement of the first ANN model previously incrementally improved via unsupervised machine learning in the population.

Abstract: Use of a NAND array architecture to realize a binary neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a BNN is stored in a pair of series connected memory cells. A binary input is applied as a pattern of voltage values on a pair of word lines connected to the unit synapse to perform the multiplication of the input with the weight by determining whether or not the unit synapse conducts. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a counter.

Abstract: A Convolution Multiply and Accumulate (CMAC) system for performing a convolution operation is disclosed. The CMAC system receives image data pertaining to an image. The image data comprises a set of feature matrix, a kernel size and depth information. Further, the CMAC system generates a convoluted data based on convolution operation for each feature matrix. The CMAC system performs an accumulation of the convoluted data to generate accumulated data, when the convolution operation for each feature matrix is performed. The CMAC system further performs an addition of a predefined value to the accumulated data to generate added data. Further, the CMAC system filters the added data to provide a convolution result for the image, thereby performing the convolution operation of the image.

Abstract: A system and method for generating approximations of query results. The method includes sending a received query to a neural network, wherein the received query is executable on a target data set; receiving from the neural network a predicted result to the received query; providing the predicted result as a first output to a device having initiated the received query; determining a real result of the query from a data set stored in the database when the predicted result is insufficiently accurate; and providing the real result as a second output to a device having initiated the received query.

Abstract: An image processing apparatus obtains a first output image by applying an image to a first training network model, obtains a second output image by applying the image to a second training network model, and obtains a reconstructed image based on the first output image and the second output image. The first training network model is a model that uses a fixed parameter obtained through training of a plurality of sample images, the second training network model is trained to minimize a difference between a target image corresponding to the image and the reconstructed image.

Abstract: Provided are an application prediction method, an application preloading method and an application preloading apparatus. The application prediction method includes: obtaining a user behavior sample in a preset time period, where the user behavior sample includes an association record of usage timing of at least two applications determined from two or more applications on a terminal including a usage record of the at least two application and a usage timing relationship of the at least two applications; and training a preset prediction model according to the association record of usage timing to generate an application prediction model, thereby may take full advantage of the association record of usage timing of the applications which may truly reflect the user behavior, optimize the application preloading mechanism, improve the accuracy of the prediction of the application to be started effectively, and further reduce power consumption of the terminal system and the memory usage.

Abstract: A method may include receiving a graph-based model in a first format, including a static topology of the graph-based model. The method may also include encoding the graph-based model from the first format into a neural network topology optimizer (NNTO) readable format such that the topology of the encoded graph-based model is configured to be altered; creating a first group of entities based on at least a same portion of the encoded graph-based model; and performing a learning operation by tuning parameters of the first group of entities to produce an optimization score for each entity. Additionally, the method may include performing a validation operation; determining that an improvement in validation performance for at least one entity is within a threshold amount of improvement; selecting a solution entity; and adding the selected solution entity into the graph-based model in place of the same portion.

Abstract: A method for calculating an output of a neural network, including the steps of generating a first neural network that includes discrete edge weights from a neural network that includes precise edge weights by stochastic rounding; of generating a second neural network that includes discrete edge weights from the neural network that includes precise edge weights by stochastic rounding; and of calculating an output by adding together the output of the first neural network and of the second neural network.

Abstract: Systems and methods for forming radio frequency beams in communication systems are provided. Signals from one or more devices are received at a base station and are processed using a vector based deep learning (VBDL) model or network. The VBDL model can receive and process vector and/or spatial information related to or part of the received signals. An optimal beamforming vector for a received signal is determined by the VBDL model, without reference to a codebook. The VBDL model can incorporate parameters that are pruned during training to provide efficient operation of the model.

Abstract: Exemplary embodiments relate to adapting a generic language model during runtime using domain-specific language model data. The system performs an audio frame-level analysis, to determine if the utterance corresponds to a particular domain and whether the ASR hypothesis needs to be rescored. The system processes, using a trained classifier, the ASR hypothesis (a partial hypothesis) generated for the audio data processed so far. The system determines whether to rescore the hypothesis after every few audio frames (representing a word in the utterance) are processed by the speech recognition system.

Abstract: A detective method, applied in a detective system comprising an activity-or-behavior model constructor, for activity-or-behavior model construction and automatic detection of activities of a subject system, comprises steps of using an unsupervised machine learning technique, a Natural Language Processing technique (NLP) and a supervised machine learning technique. As such, an activity-or-behavior model is built for predicting the future behaviors of the subject system and automatically detecting abnormal activities or behaviors of the subject system. The activity-or-behavior model is capable to handle multidimensional sensor data input from a plurality of sensor data streams and incorporate the sensor data values and a selected temporal information about at least one sensor data stream and between different sensor data streams.

Abstract: Embodiments of the present invention relate to providing fault-tolerant power minimization in a multi-core neurosynaptic network. In one embodiment of the present invention, a method of and computer program product for fault-tolerant power-driven synthesis is provided. Power consumption of a neurosynaptic network is modeled as wire length. The neurosynaptic network comprises a plurality of neurosynaptic cores connected by a plurality of routers. At least one faulty core of the plurality of neurosynaptic cores is located. A placement blockage is modeled at the location of the at least one faulty core. A placement of the neurosynaptic cores is determined by minimizing the wire length.

Abstract: Embodiments of the invention relate to a time-division multiplexed neurosynaptic module with implicit memory addressing for implementing a neural network. One embodiment comprises maintaining neuron attributes for multiple neurons and maintaining incoming firing events for different time steps. For each time step, incoming firing events for said time step are integrated in a time-division multiplexing manner. Incoming firing events are integrated based on the neuron attributes maintained. For each time step, the neuron attributes maintained are updated in parallel based on the integrated incoming firing events for said time step.

Abstract: Disclosed embodiments provide a framework to assist customers in obtaining relevant responses from brands and other users to the intents communicated by these customers. In response to obtaining an intent, an intent messaging service identifies one or more users that can be provided with the intent to solicit responses to the intent. The one or more users are selected based on characteristics of the intent. The intent messaging service evaluates the responses to the intent from the one or more users to identify relevant responses that can be presented to the customer. The intent messaging service provides the relevant responses to the intent to the customer, which can determine which users to interact with to address the intent.

Abstract: In one implementation, a method of including a person in a CGR experience or excluding the person from the CGR experience is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes, while presenting a CGR experience, capturing an image of scene; detecting, in the image of the scene, a person; and determining an identity of the person. The method includes determining, based on the identity of the person, whether to include the person in the CGR experience or exclude the person from the CGR experience. The method includes presenting the CGR experience based on the determination.

Abstract: A neural network training apparatus is described which has a network of worker nodes each having a memory storing a subgraph of a neural network to be trained. The apparatus has a control node connected to the network of worker nodes. The control node is configured to send training data instances into the network to trigger parallelized message passing operations which implement a training algorithm which trains the neural network. At least some of the message passing operations asynchronously update parameters of individual subgraphs of the neural network at the individual worker nodes.

Abstract: An intelligent monitoring device including a processor and an accelerometer and/or a device that includes at least one processor and at least one image sensor. The intelligent monitoring device is configured to observe at least one machine. The intelligent monitoring device is further configured to utilize its processor, or the processor of a coupled system, to recognize actions carried out on or by the at least one machine and infer the state of the machine. The intelligent monitoring device or the coupled system is further configured to provide alerts or help respond to queries about the status of the at least one machine. For example, the intelligent monitoring camera will infer the state of a washing machine based on its observations and provide an alert (optionally only to the user recognized to have loaded the washer) when the washing machine is ready to be unloaded.

Abstract: Systems and methods that approximate and use branching operations on data encrypted by fully homomorphic encryption (FHE). The systems and methods may use polynomial approximation to convert “if” statements into “soft if” statements that may be applied to the FHE encrypted data in a manner that preserves the security of the systems and methods.

Abstract: A system and method of animating an image of an object may include: receiving a first image, depicting a “puppet” object; sampling an input video, depicting a second, “driver” object, to obtain at least one second image; obtaining, by a first machine-learning (ML) model, a first identity-invariant feature of the puppet object, from the first image; obtaining at least one second identity-invariant feature of the driver object, from the respective at least one second image; calculating, by a second ML model, a transformation function, based on the first identity-invariant feature and the at least one second identity-invariant feature; applying the calculated transformation function on the first image, to produce one or more third images, depicting a target object, including at least one identity-invariant feature of the driver object; and appending the one or more third images to produce an output video depicting animation of the puppet object.

Abstract: Disclosed herein are techniques for obtain weights for neural network computations. In one embodiment, an integrated circuit may include memory configured to store a first weight and a second weight; a row of processing elements comprising a first processing element and a second processing element, the first processing element comprising a first weight register, the second processing element comprising a second weight register, both of the first weight register and the second weight register being controllable by a weight load signal; and a controller configured to: provide the first weight from the memory to the row of processing elements; set the weight load signal to enable the first weight to propagate through the row to reach the first processing element; and set the weight load signal to store the first weight at the first weight register and the flush value at the second weight register.

Abstract: One embodiment relates to a neuromorphic network including electronic neurons and an interconnect circuit for interconnecting the neurons. The interconnect circuit includes synaptic devices for interconnecting the neurons via axon paths, dendrite paths and membrane paths. Each synaptic device includes a variable state resistor and a transistor device with a gate terminal, a source terminal and a drain terminal, wherein the drain terminal is connected in series with a first terminal of the variable state resistor. The source terminal of the transistor device is connected to an axon path, the gate terminal of the transistor device is connected to a membrane path and a second terminal of the variable state resistor is connected to a dendrite path, such that each synaptic device is coupled between a first axon path and a first dendrite path, and between a first membrane path and said first dendrite path.

Abstract: A system for reconfiguring neural network architecture responsive to a system state is provided. A controller for modifying a neural network learning engine is configured to monitor a data stream having a data pattern by comparing the data pattern to a trained data pattern; identify a change in the data pattern of the data stream; determine a state of the neural network learning engine, the state defining one or more neural network parameters for monitoring the data stream with the neural network learning engine; and in response to identifying the change in the data pattern and determining the state, reconfigure an architectural configuration of the neural network learning engine by modifying the one or more neural network parameters.

Abstract: In one aspect, this specification describes a recurrent neural network system implemented by one or more computers that is configured to process input sets to generate neural network outputs for each input set. The input set can be a collection of multiple inputs for which the recurrent neural network should generate the same neural network output regardless of the order in which the inputs are arranged in the collection. The recurrent neural network system can include a read neural network, a process neural network, and a write neural network. In another aspect, this specification describes a system implemented as computer programs on one or more computers in one or more locations that is configured to train a recurrent neural network that receives a neural network input and sequentially emits outputs to generate an output sequence for the neural network input.

Abstract: Disclosed herein are techniques for obtain weights for neural network computations. In one embodiment, an integrated circuit may include an arithmetic circuit configured to perform arithmetic operations for a neural network. The integrated circuit may also include a weight processing circuit configured to: acquire data from a memory device; receive configuration information indicating a size of each quantized weight of a set of quantized weights; extract the set of quantized weights from the data based on the size of the each weight indicated by the configuration information; perform de-quantization processing on the set of quantized weights to generate a set of de-quantized weights; and provide the set of de-quantized weights to the arithmetic circuit to enable the arithmetic circuit to perform the arithmetic operations. The memory device may be part of or external to the integrated circuit.