Abstract: Some embodiments provide a neural network inference circuit for executing a neural network that includes multiple nodes that use state data from previous executions of the neural network. The neural network inference circuit includes (i) a set of computation circuits configured to execute the nodes of the neural network and (ii) a set of memories configured to implement a set of one or more registers to store, while executing the neural network for a particular input, state data generated during at least two executions of the network for previous inputs. The state data is for use by the set of computation circuits when executing a set of the nodes of the neural network for the particular input.

Abstract: Some embodiments provide a method for training a machine-trained (MT) network that processes inputs using network parameters. The method propagates a set of input training items through the MT network to generate a set of output values. The set of input training items comprises multiple training items for each of multiple categories. The method identifies multiple training item groupings in the set of input training items. Each grouping includes at least two training items in a first category and at least one training item in a second category. The method calculates a value of a loss function as a summation of individual loss functions for each of the identified training item groupings. The individual loss function for each particular training item grouping is based on the output values for the training items of the grouping. The method trains the network parameters using the calculated loss function value.

Abstract: Some embodiments provide a method for training a machine-trained (MT) network that processes inputs using network parameters. The method propagates a set of input training items through the MT network to generate a set of output values. The set of input training items comprises multiple training items for each of multiple categories. The method identifies multiple training item groupings in the set of input training items. Each grouping includes at least two training items in a first category and at least one training item in a second category. The method calculates a value of a loss function as a summation of individual loss functions for each of the identified training item groupings. The individual loss function for each particular training item grouping is based on the output values for the training items of the grouping. The method trains the network parameters using the calculated loss function value.

Abstract: Some embodiments provide a method for executing a neural network that includes multiple nodes. The method receives an input for a particular execution of the neural network. The method receives state data that includes data generated from at least two previous executions of the neural network. The method executes the neural network to generate a set of output data for the received input. A set of the nodes performs computations using (i) data output from other nodes of the particular execution of the neural network and (ii) the received state data generated from at least two previous executions of the neural network.

Abstract: Some embodiments provide a method for training a machine-trained (MT) network that processes input data using network parameters. The method maps input instances to output values by propagating the instances through the network. The input instances include instances for each of multiple categories. For a particular instance selected as an anchor instance, the method identifies each instance in a different category as a negative instance. The method calculates, for each negative instance of the anchor, a surprise function that probabilistically measures a surprise of finding an output value for an instance in the same category as the anchor that is a greater distance from the output value for the anchor instance than output value for the negative instance. The method calculates a loss function that emphasizes a maximum surprise calculated for the anchor. The method trains the network parameters using the calculated loss function value to minimize the maximum surprise.

Abstract: Some embodiments provide a method for configuring a machine-trained (MT) network that includes multiple configurable weights to train. The method propagates a set of inputs through the MT network to generate a set of output probability distributions. Each input has a corresponding expected output probability distribution. The method calculates a value of a continuously-differentiable loss function that includes a term approximating an extremum function of the difference between the expected output probability distributions and generated set of output probability distributions. The method trains the weights by back-propagating the calculated value of the continuously-differentiable loss function.

Abstract: Some embodiments provide a method for configuring a machine-trained (MT) network that includes multiple configurable weights to train. The method propagates a set of inputs through the MT network to generate a set of output probability distributions. Each input has a corresponding expected output probability distribution. The method calculates a value of a continuously-differentiable loss function that includes a term approximating an extremum function of the difference between the expected output probability distributions and generated set of output probability distributions. The method trains the weights by back-propagating the calculated value of the continuously-differentiable loss function.

Abstract: Some embodiments provide a method for training a machine-trained (MT) network that processes inputs using network parameters. The method propagates a set of input training items through the MT network to generate a set of output values. The set of input training items comprises multiple training items for each of multiple categories. The method identifies multiple training item groupings in the set of input training items. Each grouping includes at least two training items in a first category and at least one training item in a second category. The method calculates a value of a loss function as a summation of individual loss functions for each of the identified training item groupings. The individual loss function for each particular training item grouping is based on the output values for the training items of the grouping. The method trains the network parameters using the calculated loss function value.

Abstract: Some embodiments provide a method for training a machine-trained (MT) network that processes inputs using network parameters. The method propagates a set of input training items through the MT network to generate a set of output values. The set of input training items comprises multiple training items for each of multiple categories. The method identifies multiple training item groupings in the set of input training items. Each grouping includes at least two training items in a first category and at least one training item in a second category. The method calculates a value of a loss function as a summation of individual loss functions for each of the identified training item groupings. The individual loss function for each particular training item grouping is based on the output values for the training items of the grouping. The method trains the network parameters using the calculated loss function value.

Abstract: Some embodiments provide a method for processing a video that includes a sequence of images using a neural network. The method receives a set of video images as a set of inputs to successive executions of the neural network. The method executes the neural network for each successive video image of the set of video images to reduce an amount of noise in the video image by (i) identifying spatial features of the video image and (ii) storing a set of state data representing identified spatial features for use in identifying spatial features of subsequent video images in the set of video images. Identifying spatial features of a particular video image includes using the stored sets of spatial features of video images previous to the particular video image.

Abstract: Some embodiments provide a method for training a machine-trained (MT) network that processes inputs using network parameters. The method propagates a set of input training items through the MT network to generate a set of output values. The set of input training items comprises multiple training items for each of multiple categories. The method identifies multiple training item groupings in the set of input training items. Each grouping includes at least two training items in a first category and at least one training item in a second category. The method calculates a value of a loss function as a summation of individual loss functions for each of the identified training item groupings. The individual loss function for each particular training item grouping is based on the output values for the training items of the grouping. The method trains the network parameters using the calculated loss function value.

Abstract: Some embodiments provide a method for training a machine-trained (MT) network that processes inputs using network parameters. The method propagates a set of input training items through the MT network to generate a set of output values. The set of input training items comprises multiple training items for each of multiple categories. The method identifies multiple training item groupings in the set of input training items. Each grouping includes at least two training items in a first category and at least one training item in a second category. The method calculates a value of a loss function as a summation of individual loss functions for each of the identified training item groupings. The individual loss function for each particular training item grouping is based on the output values for the training items of the grouping. The method trains the network parameters using the calculated loss function value.

Abstract: Some embodiments provide a method for training a machine-trained (MT) network that processes images using multiple network parameters. The method propagates a triplet of input images through the MT network to generate an output value for each of the input images. The triplet includes an anchor first image, a second image of a same category as the anchor image, and a third image of a different category as the anchor image. The method calculates a value of a loss function for the triplet that is based on a probabilistic classification of an output value for the anchor image compared to output values for the second and third images. The method uses the calculated loss function value to train the network parameters.

Abstract: A method of detailed placement for ICs is provided. The method receives an initial placement and iteratively builds sets of constraints for placement of different groups of cells in the IC design and uses a satisfiability solver to resolve placement violations. In some embodiments, the constraints include mathematical expressions that express timing requirements. The method in some embodiments converts the mathematical expressions into Boolean clauses and sends the clauses to a satisfiability solver that is only capable of solving Boolean clauses. In some embodiments, the method groups several cells in the user design and several sites on the IC fabric and uses the satisfiability solver to resolve all placement issues in the group. The satisfiability solver informs placer after each cell is moved to a different site. The method then dynamically builds more constraints based on the new cell placement and sends the constraints to the satisfiability solver.

Abstract: A method of detailed placement for ICs is provided. The method receives an initial placement and iteratively builds sets of constraints for placement of different groups of cells in the IC design and uses a satisfiability solver to resolve placement violations. In some embodiments, the constraints include mathematical expressions that express timing requirements. The method in some embodiments converts the mathematical expressions into Boolean clauses and sends the clauses to a satisfiability solver that is only capable of solving Boolean clauses. In some embodiments, the method groups several cells in the user design and several sites on the IC fabric and uses the satisfiability solver to resolve all placement issues in the group. The satisfiability solver informs placer after each cell is moved to a different site. The method then dynamically builds more constraints based on the new cell placement and sends the constraints to the satisfiability solver.

Abstract: A method of detailed placement for ICs is provided. The method receives an initial placement and iteratively builds sets of constraints for placement of different groups of cells in the IC design and uses a satisfiability solver to resolve placement violations. In some embodiments, the constraints include mathematical expressions that express timing requirements. The method in some embodiments converts the mathematical expressions into Boolean clauses and sends the clauses to a satisfiability solver that is only capable of solving Boolean clauses. In some embodiments, the method groups several cells in the user design and several sites on the IC fabric and uses the satisfiability solver to resolve all placement issues in the group. The satisfiability solver informs placer after each cell is moved to a different site. The method then dynamically builds more constraints based on the new cell placement and sends the constraints to the satisfiability solver.