Abstract: Some embodiments provide a method for compiling a neural network (NN) program for an NN inference circuit (NNIC) that includes multiple partial dot product computation circuits (PDPCCs) for computing dot products between weight values and input values. The method receives an NN definition with multiple nodes. The method assigns a group of filters to specific PDPCCs. Each filter is assigned to a different set of the PDPCCs. When a filter does not have enough weight values equal to zero for a first set of PDPCCs to which the filter is assigned to compute dot products for nodes that use the filter, the method divides the filter between the first set and a second set of PDPCCs. The method generates program instructions for instructing the NNIC to execute the NN by using the first and second PDPCCs to compute dot products for the nodes that use the filter.

Abstract: Some embodiments provide a novel compressive-sensing image capture device and a method of using data captured by the compressive-sensing image capture device. The novel compressive-sensing image capture device includes an array of sensors for detecting electromagnetic radiation. Each sensor in the sensor array has an associated mask that blocks electromagnetic radiation from portions of the sensor. In some embodiments, an array of passive masks is used to block a particular set of areas of each sensor in the sensor array. In some embodiments, the image capture device also includes an array of lenses corresponding to the sensors of the sensor array such that each sensor receives light that passes through a different lens. Some embodiments of the invention provide a dynamic mask array. In some embodiments, a novel machine trained network is provided that processes image capture data captured by the compressive-sensing image capture device to predict solutions to problems.

Abstract: Some embodiments provide a neural network inference circuit (NNIC) for executing a neural network. The NNIC includes a first circuit that outputs dot products for computation nodes of a first set of neural network layers, that include dot product computations of sets of weight values with sets of input values. The NNIC also includes a second circuit that outputs values for computation nodes of a second set of neural network layers, that apply a set of calculations that do not include dot products to sets of input values. The NNIC also includes a selection circuit that selects a dot product output from the first circuit when a current layer being processed by the NNIC belongs to the first set of layers, and selects a non-dot product output from the second circuit when the current layer belongs to the second set of layers.

Abstract: Some embodiments provide a method for a neural network inference circuit that executes a neural network including multiple computation nodes at multiple layers. Each computation node of a set of the computation nodes includes a dot product of input values and weight values. The method reads a set of encoded weight data for a set of weight values from a memory of the neural network inference circuit. The method decodes the encoded weight data to generate decoded weight data for the set of weight values. The method stores the decoded weight data in a buffer. The method uses the decoded weight data to execute a set of computation nodes. Each computation node of the set of computation nodes includes a dot product between the set of weight values and a different set of input values.

Abstract: Some embodiments provide a neural network inference circuit (NNIC) for implementing a neural network that includes multiple computation nodes at multiple layers. Each of a set of the computation nodes includes a dot product of input values and weight values. The NNIC includes multiple dot product core circuits for computing multiple partial dot products and a set of channel circuits connecting the core circuits. The set of channel circuits includes (i) a dot product bus for aggregating the partial dot products to compute dot products for computation nodes of the neural network, (ii) one or more post-processing circuits for performing additional computation operations on the dot products to compute outputs for the computation nodes, and (iii) an output bus for providing the computed outputs of the computation nodes to the core circuits for the core circuits to use as inputs for subsequent computation nodes.

Abstract: Some embodiments provide a neural network inference circuit (NNIC) for executing a neural network that includes multiple computation nodes, that include dot products, at multiple layers. The NNIC includes multiple dot product core circuits and a bus, including one or more aggregation circuits, that connects the core circuits. Each core circuit includes (i) a set of memories for storing multiple input values and multiple weight values and (ii) a set of adder tree circuits for computing dot products of sets of input values and sets of weight values stored in the set of memories. For a particular computation node, at least two of the core circuits compute partial dot products using input values and weight values stored in the memories of the respective core circuits and at least one of the aggregation circuits of the bus combines the partial dot products to compute the dot product for the computation node.

Abstract: Some embodiments provide a method for training a machine-trained (MT) network that processes inputs using network parameters. The method propagates a set of input training items through the MT network to generate a set of output values. The set of input training items comprises multiple training items for each of multiple categories. The method identifies multiple training item groupings in the set of input training items. Each grouping includes at least two training items in a first category and at least one training item in a second category. The method calculates a value of a loss function as a summation of individual loss functions for each of the identified training item groupings. The individual loss function for each particular training item grouping is based on the output values for the training items of the grouping. The method trains the network parameters using the calculated loss function value.

Abstract: Some embodiments provide a method for processing a video that includes a sequence of images using a neural network. The method receives a set of video images as a set of inputs to successive executions of the neural network. The method executes the neural network for each successive video image of the set of video images to reduce an amount of noise in the video image by (i) identifying spatial features of the video image and (ii) storing a set of state data representing identified spatial features for use in identifying spatial features of subsequent video images in the set of video images. Identifying spatial features of a particular video image includes using the stored sets of spatial features of video images previous to the particular video image.

Abstract: Some embodiments provide a method for configuring a machine-trained (MT) network that includes input nodes, output nodes, and interior nodes between the input and output nodes. Each node produces an output value and each interior node and output node receives as input values a set of output values of other nodes and applies weights to each received input value. The weights are configurable parameters for training. The method propagates a set of inputs through the MT network to generate a set of outputs. Each input has a corresponding expected output. The method calculates a value of a continuously-differentiable augmented loss function that combines a measurement of a difference between each output and its corresponding expected output and a term that biases training of the weights towards a set of discrete values. The method trains the weights by backpropagating a gradient of the continuously-differentiable augmented loss function at the calculated value.

Abstract: Some embodiments provide a novel compressive-sensing image capture device and a method of using data captured by the compressive-sensing image capture device. The novel compressive-sensing image capture device includes an array of sensors for detecting electromagnetic radiation. Each sensor in the sensor array has an associated mask that blocks electromagnetic radiation from portions of the sensor. In some embodiments, a diffractive mask is used to direct incoming light from a same object to different sensors in a sensing array. Some embodiments of the invention provide a dynamic mask array. In some embodiments, a novel machine trained network is provided that processes image capture data captured by the compressive-sensing image capture device to predict solutions to problems.

Abstract: Some embodiments provide a neural network inference circuit for executing a neural network that includes multiple computation nodes at multiple layers. Each of a set of the computation nodes includes a dot product of input values and weight values. The neural network inference circuit includes (i) a first set of memory units allocated to storing input values during execution of the neural network and (ii) a second set of memory units storing encoded weight value data. The weight value data is encoded such that less than one bit of memory is used per weight value of the neural network.

Abstract: Some embodiments of the invention provide a novel multi-layer node network to determine a set of misalignment values for a set of cameras that may be arranged with deviations from an ideal alignment or placement based on images captured by the set of cameras. A set of misalignment values for a set of cameras, in some embodiments, takes the form of translation vectors indicating the offsets between the centers of projection of the cameras relative to some useful coordinate system, and quaternions indicating the orientations of the cameras' optical axes and reference vectors associated with the cameras. Some embodiments train the multi-layer network using a set of inputs generated with random misalignments incorporated into the training set.

Abstract: Some embodiments provide a set of processing units and a set of machine-readable media. The set of machine-readable media stores sets of instructions for applying a network of computation nodes to an input received by the device. The network of computation nodes includes multiple layers of nodes. The set of machine-readable media stores a set of machine-trained weight parameters for configuring the network to perform a specific function. Each layer of nodes has an associated value, and each of the weight parameters is associated with a computation node. Each weight parameter is zero, the associated value for the layer of the computation node with which the weight parameter is associated, or the negative of the associated value for the layer of the computation node with which the weight parameter is associated. Each weight value is stored using two bits or less of data.

Abstract: Some embodiments provide an IC for implementing a machine-trained network with multiple layers. The IC includes a set of circuits to compute a dot product of (i) a first number of input values computed by other circuits of the IC and (ii) a set of predefined weight values, several of which are zero, with a weight value for each of the input values. The set of circuits includes (i) a dot product computation circuit to compute the dot product based on a second number of inputs and (ii) for each input value, at least two sets of wires for providing the input value to at least two of the dot product computation circuit inputs. The second number is less than the first number. Each input value with a corresponding weight value that is not equal to zero is provided to a different one of the dot product computation circuit inputs.

Abstract: Some embodiments provide a method for executing a portion of a node of a machine-trained network. The method receives (i) multiple input values computed by previous nodes of the machine-trained network and (ii) for each of the input values, a corresponding predefined weight value. Each of the weight values is zero, a positive value, or a negation of the positive value. To compute a dot product of the input values with the weight values, the method passes to an adder circuit the input value for each input value with a corresponding positive weight value, the value zero for each input value with a corresponding weight value of zero, and a binary inversion of the input value for each input value with a corresponding negative weight value. After the adder circuit adds the values passed to it, the method adds an additional value based on the number of negative weight values.

Abstract: Some embodiments provide a novel compressive-sensing image capture device and a method of using data captured by the compressive-sensing image capture device. The novel compressive-sensing image capture device includes an array of sensors for detecting electromagnetic radiation. Each sensor in the sensor array has an associated mask that blocks electromagnetic radiation from portions of the sensor. In some embodiments, an array of passive masks is used to block a particular set of areas of each sensor in the sensor array. In some embodiments, the image capture device also includes an array of lenses corresponding to the sensors of the sensor array such that each sensor receives light that passes through a different lens. Some embodiments of the invention provide a dynamic mask array. In some embodiments, a novel machine trained network is provided that processes image capture data captured by the compressive-sensing image capture device to predict solutions to problems.

Abstract: Some embodiments of the invention provide efficient, expressive machine-trained networks for performing machine learning. The machine-trained (MT) networks of some embodiments use novel processing nodes with novel activation functions that allow the MT network to efficiently define with fewer processing node layers a complex mathematical expression that solves a particular problem (e.g., face recognition, speech recognition, etc.). In some embodiments, the same activation function (e.g., a cup function) is used for numerous processing nodes of the MT network, but through the machine learning, this activation function is configured differently for different processing nodes so that different nodes can emulate or implement two or more different functions (e.g., two or more Boolean logical operators, such as XOR and AND). The activation function in some embodiments is a periodic function that can be configured to implement different functions (e.g., different sinusoidal functions).

Abstract: Some embodiments provide a novel compressive-sensing image capture device and a method of using data captured by the compressive-sensing image capture device. The novel compressive-sensing image capture device includes an array of sensors for detecting electromagnetic radiation. Each sensor in the sensor array has an associated mask that blocks electromagnetic radiation from portions of the sensor. In some embodiments, an array of passive masks is used to block a particular set of areas of each sensor in the sensor array. In some embodiments, the image capture device also includes an array of lenses corresponding to the sensors of the sensor array such that each sensor receives light that passes through a different lens. Some embodiments of the invention provide a dynamic mask array. In some embodiments, a novel machine trained network is provided that processes image capture data captured by the compressive-sensing image capture device to predict solutions to problems.

Abstract: Some embodiments provide a novel compressive-sensing image capture device and a method of using data captured by the compressive-sensing image capture device. The novel compressive-sensing image capture device includes an array of sensors for detecting electromagnetic radiation. Each sensor in the sensor array has an associated mask that blocks electromagnetic radiation from portions of the sensor. In some embodiments, an array of passive masks is used to block a particular set of areas of each sensor in the sensor array. In some embodiments, the image capture device also includes an array of lenses corresponding to the sensors of the sensor array such that each sensor receives light that passes through a different lens. Some embodiments of the invention provide a dynamic mask array. In some embodiments, a novel machine trained network is provided that processes image capture data captured by the compressive-sensing image capture device to predict solutions to problems.

Abstract: Some embodiments of the invention provide efficient, expressive machine-trained networks for performing machine learning. The machine-trained (MT) networks of some embodiments use novel processing nodes with novel activation functions that allow the MT network to efficiently define with fewer processing node layers a complex mathematical expression that solves a particular problem (e.g., face recognition, speech recognition, etc.). In some embodiments, the same activation function (e.g., a cup function) is used for numerous processing nodes of the MT network, but through the machine learning, this activation function is configured differently for different processing nodes so that different nodes can emulate or implement two or more different functions (e.g., two or more Boolean logical operators, such as XOR and AND). The activation function in some embodiments is a periodic function that can be configured to implement different functions (e.g., different sinusoidal functions).