Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using embedded function with a deep network. One of the methods includes receiving an input comprising a plurality of features, wherein each of the features is of a different feature type; processing each of the features using a respective embedding function to generate one or more numeric values, wherein each of the embedding functions operates independently of each other embedding function, and wherein each of the embedding functions is used for features of a respective feature type; processing the numeric values using a deep network to generate a first alternative representation of the input, wherein the deep network is a machine learning model composed of a plurality of levels of non-linear operations; and processing the first alternative representation of the input using a logistic regression classifier to predict a label for the input.

Abstract: A system that uses one or more artificial intelligence models that predict an effect of a predicted event on a current state of the system. For example, the model may predict how a rate of change in time-series data may be altered throughout the first time period based on the predicted event.

Abstract: Systems, apparatuses, and methods for adaptively mapping a machine learning model to a multi-core inference accelerator engine are disclosed. A computing system includes a multi-core inference accelerator engine with multiple inference cores coupled to a memory subsystem. The system also includes a control unit which determines how to adaptively map a machine learning model to the multi-core inference accelerator engine. In one implementation, the control unit selects a mapping scheme which minimizes the memory bandwidth utilization of the multi-core inference accelerator engine. In one implementation, this mapping scheme involves having one inference core of the multi-core inference accelerator engine fetch given data and broadcast the given data to other inference cores of the inference accelerator engine. Each inference core fetches second data unique to the respective inference core.

Abstract: Disclosed is a processor-implemented data processing method in a neural network. A data processing apparatus includes at least one processor, and at least one memory configured to store instructions to be executed by the processor and a neural network, wherein the processor is configured to, based on the instructions, input an input activation map into a current layer included in the neural network, output an output activation map by performing a convolution operation between the input activation map and a weight quantized with a first representation bit number of the current layer, and output a quantized activation map by quantizing the output activation map with a second representation bit number based on an activation quantization parameter.

Abstract: A system includes inputs, outputs, and nodes between the inputs and the outputs. The nodes include hidden nodes. Connections between the nodes are determined based on a gradient computable using symmetric solution submatrices.

Abstract: A computer-implemented method includes: initializing model parameters for training a neural network; performing a forward pass and backpropagation for a first minibatch of training data; determining a new weight value for each of a plurality of nodes of the neural network using a gradient descent of the first minibatch; for each determined new weight value, determining whether to update a running mean corresponding to a weight of a particular node; based on a determination to update the running mean, calculating a new mean weight value for the particular node using the determined new weight value; updating the weight parameters for all nodes based on the calculated new mean weight values corresponding to each node; assigning the running mean as the weight for the particular node when training on the first minibatch is completed; and reinitializing running means for all nodes at a start of training a second minibatch.

Abstract: Methods and systems include a neural network system that includes a neural network accelerator. The neural network accelerator includes multiple processing engines coupled together to perform arithmetic operations in support of an inference performed using the deep neural network system. The neural network accelerator also includes a schedule-aware tensor data distribution circuitry or software that is configured to load tensor data into the multiple processing engines in a load phase, extract output data from the multiple processing engines in an extraction phase, reorganize the extracted output data, and store the reorganized extracted output data to memory.

Abstract: A computing core circuit, including: an encoding module, a route sending module, and a control module, wherein the control module is configured to control the encoding module to perform encoding processing on a pulse sequence determined by pulses of at least one neuron in a current computing core to be transmitted, so as to obtain an encoded pulse sequence, and control the route sending module to determine a corresponding route packet according to the encoded pulse sequence, so as to send the route packet. The present disclosure further provides a data processing method, a chip, a board, an electronic device, and a computer-readable storage medium.

Abstract: The present disclosure applies trained artificial intelligence (AI) processing adapted to automatically generating transformations of formatted templates. Pre-existing formatted templates (e.g., slide-based presentation templates) are leveraged by the trained AI processing to automatically generate a plurality of high-quality template transformations. In transforming a formatted template, the trained AI processing not only generates feature transformation of objects thereof but may also provide style transformations where attributes associated with a presentation theme may be modified for a formatted template or set of formatted templates. The trained AI processing is novel in that it is tailored for analysis of feature data of a specific type of formatted template.

Abstract: A simulation apparatus includes a machine learning device for learning a change in a machining route in machining of a workpiece. The machine learning device observes data indicating the changed machining route and data indicating a machining condition of the workpiece as a state variable, and also acquires determination data for determining whether or not a cycle time obtained by simulation using the changed machining route is appropriate, and learns by associating the machining condition of the workpiece with the change in the machining route, using the state variable and the determination data.

Abstract: A data processing system is configured to pre-process data for a machine learning classifier. The data processing system includes an input port that receives one or more data items, an extraction engine that extracts a plurality of data signatures and structure data, a logical rule set generation engine configured to generate a data structure, select a particular data signature of the data structure, identify each instance of the particular data signature in the data structure, segment the data structure around instances of the particular data signature, identify one or more sequences of data signatures connected to the particular data signature, and generate a logical ruleset. A classification engine executes one or more classifiers against the logical ruleset to classify the one or more data items received by the input port.

Abstract: A neural network system includes a first neural network configured to predict a mean value output and epistemic uncertainty of the output given input data, and a second neural network configured to predict total uncertainty of the output of the first neural network. The second neural network is trained to predict total uncertainty of the output of the first neural network given the input data through a training process involving minimizing a cost function that involves differences between a predicted mean value of a geophysical property of a geological formation from the first neural network and a ground-truth value of the geophysical property of the geological formation. The neural network system further includes one or more processors configured to run a software module that determines aleatoric uncertainty of the output of the first neural network based on the epistemic uncertainty of the output and the total uncertainty of the output.

Abstract: A neural processing device and method for pruning thereof are provided. The neural processing device includes a processing unit configured to perform calculations, an L0 memory configured to store input and output data of the processing unit, wherein the input and output data include a two-dimensional weight matrix and a weight manipulator configured to receive the two-dimensional weight matrix and partition it into preset sizes to thereby generate partitioned matrices, to generate a pruning matrix by pruning the partitioned matrix, and to transmit the pruning matrix to the processing unit.

Abstract: A system tokenizes raw values and corresponding standardized values into raw token sequences and corresponding standardized token sequences. A machine-learning model learns standardization from token insertions and token substitutions that modify the raw token sequences to match the corresponding standardized token sequences. The system tokenizes an input value into an input token sequence. The machine-learning model determines a probability of inserting an insertion token after an insertion markable token in the input token sequence. If the probability of inserting the insertion token satisfies a threshold, the system inserts the insertion token after the insertion markable token in the input token sequence. The machine-learning model determines a probability of substituting a substitution token for a substitutable token in the input token sequence.

Abstract: Detection systems, methods and computer program products comprising a non-transitory tangible storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method for anomaly detection, a detected anomaly being indicative of an undesirable event. A detection system comprises a computer and an anomaly detection engine executable by the computer, the anomaly detection engine configured to perform a method comprising receiving data comprising a plurality m of multidimensional data points (MDDPs), each data point having n features, constructing a dictionary D based on the received data, embedding dictionary D into a lower dimension embedded space and classifying, based in the lower dimension embedded space, a MDDP as an anomaly or as normal.

Abstract: A facility for estimating a value relating to an occurrence is described. The facility receives a first occurrence specifying a first value for each of a plurality of independent variables that include a distinguished independent variable designated to be monotonically linked to a dependent variable. The facility subjects the first independent variable values specified by the received occurrence to a statistical model to obtain a first value of the dependent variable. The facility receives a second occurrence specifying a second value for each of the plurality of independent variables (the second values varying from the first values in a first direction). The facility subjects the second independent variable values to the statistical model to obtain a second value of the dependent variable, the second value of the dependent variable being guaranteed not to vary from the first value of the dependent variable in a second direction opposite the first direction.

Abstract: In a data processing system, at least one processing node is configured to perform computations for a multi-stage process whilst at least one other processor performs the load/unload operations required to calculate a subsequent stage of the multi stage process. An exchange of data then occurs between the processing nodes. At a later time, at least one processing node performs calculations using the data loaded from storage, whilst at least one other processor performs the load/unload operations required to calculate a subsequent stage of the multi stage process.

Abstract: An acceleration engine with multiple accelerators may share a common set of data that is used by each accelerator to perform computations on input data. The set of shared data can be loaded into the acceleration engine from an external memory. Instead of accessing the external memory multiple times to load the set of shared data into each accelerator, the external memory can be accessed once using direct memory access to load the set of shared data into the first accelerator. The set of shared data can then be serially loaded from one accelerator to the next accelerator in the acceleration engine using direct memory access. To achieve data parallelism and reduce computation time, a runtime driver may split the input data into data batches, and each accelerator can perform computations on a different batch of input data with the common set of shared data.

Abstract: A 2D array-based neuromorphic processor includes: axon circuits each being configured to receive a first input corresponding to one bit from among bits indicating n-bit activation; first direction lines extending in a first direction from the axon circuits; second direction lines intersecting the first direction lines; synapse circuits disposed at intersections of the first direction lines and the second direction lines, and each being configured to store a second input corresponding to one bit from among bits indicating an m-bit weight and to output operation values of the first input and the second input; and neuron circuits connected to the first or second direction lines, each of the neuron circuits being configured to receive an operation value output from at least one of the synapse circuits, based on time information assigned individually to the synapse circuits, and to perform an arithmetic operation by using the operation values.

Abstract: Disclosed is a neuromorphic integrated circuit including, in some embodiments, a multi-layered neural network disposed in an analog multiplier array of two-quadrant multipliers. Each multiplier of the multipliers is wired to ground and draws a negligible amount of current when input signal values for input signals to transistors of the multiplier are approximately zero, weight values of the transistors of the multiplier are approximately zero, or a combination thereof. Also disclosed is a method of the neuromorphic integrated circuit including, in some embodiments, training the neural network; tracking rates of change for the weight values; determining if and how quickly certain weight values are trending toward zero; and driving those weight values toward zero, thereby encouraging sparsity in the neural network. Sparsity in the neural network combined with the multipliers wired to ground minimizes power consumption of the neuromorphic integrated circuit such that battery power is sufficient for power.