Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sharding dataflow graphs for a device having multiple synchronous tiles. One of the methods includes receiving a representation of a dataflow graph comprising a plurality of nodes that each represent respective matrix operations to be performed by a device having a plurality synchronous tiles. Candidate allocations of respective portions of the dataflow graph to each tile of the plurality of synchronous tiles are evaluated according to one or more resource constraints of the device. One of the candidate allocations is selected based on evaluating each candidate allocation.

Abstract: A method of configuring an integrated circuit including multiple hardware tiles, includes: establishing a data forwarding path through the multiple hardware tiles by configuring each hardware tile, except for a last hardware tile, of the multiple hardware tiles to be in a data forwarding state, in which configuring each hardware tile, except for the last hardware tile, to be in a forwarding state includes installing a respective forwarding state counter specifying a corresponding predefined length of time that the hardware tile is in the data forwarding state; supplying, along the data forwarding path, each hardware tile of the plurality of hardware tiles with a respective program data packet comprising program data for the hardware tile; and installing, for each hardware tile of the multiple hardware tiles, the respective program data.

Abstract: A tile including circuitry for use with machine learning models, the tile including: a first computational array of cells, in which the computational array of cells is a sub-array of a larger second computational array of cells; local memory coupled to the first computational array of cells; and multiple controllable bus lines, in which a first subset of the multiple controllable bus lines include multiple general purpose controllable bus lines couplable to the local memory.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining, for each pair of adjacent chips in a plurality of chips connected in a series-ring arrangement of a semiconductor device, a corresponding loop latency for round trip data transmissions between the pair of chips. Identifying, from among the loop latencies, a maximum loop latency. Determining a ring latency for a data transmission originating from a chip of the plurality chips to be transmitted around the series-ring arrangement and back to the chip. Comparing half of the maximum loop latency to one N-th of the ring latency, where N is the number of chips in the plurality of chips, and storing the greater value as an inter-chip latency of the semiconductor device, the inter-chip latency representing an operational characteristic of the semiconductor device.

Abstract: Aspects of the disclosure include methods, systems, and apparatus, including computer-readable storage media for multi-mode integrated circuits with balanced energy consumption. A method includes determining, by one or more processors and based at least on a maximum energy threshold for planned multi-mode system having one or more processing units, a respective number of operations that can be performed per clock cycle by the processing units for each operating mode. The system is configured to consume the same amount of energy per clock cycle in each operating mode, but perform more operations in operating modes corresponding to operations performed on smaller bit-width operands.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining, for each pair of adjacent chips in a plurality of chips connected in a series-ring arrangement of a semiconductor device, a corresponding loop latency for round trip data transmissions between the pair of chips. Identifying, from among the loop latencies, a maximum loop latency. Determining a ring latency for a data transmission originating from a chip of the plurality chips to be transmitted around the series-ring arrangement and back to the chip. Comparing half of the maximum loop latency to one N-th of the ring latency, where N is the number of chips in the plurality of chips, and storing the greater value as an inter-chip latency of the semiconductor device, the inter-chip latency representing an operational characteristic of the semiconductor device.

Abstract: Methods, systems, and apparatus, including computer-readable media, are described for implementing a one-dimensional computational unit in an integrated circuit for a machine-learning (ML) hardware accelerator.

Abstract: An application specific integrated circuit (ASIC) chip includes: a systolic array of cells; and multiple controllable bus lines configured to convey data among the systolic array of cells, in which the systolic array of cells is arranged in multiple tiles, each tile of the multiple tiles including 1) a corresponding sub array of cells of the systolic array of cells, 2) a corresponding subset of controllable bus lines of the multiple controllable bus lines, and 3) memory coupled to the subarray of cells.

Abstract: A tile including circuitry for use with machine learning models, the tile including: a first computational array of cells, in which the computational array of cells is a sub-array of a larger second computational array of cells; local memory coupled to the first computational array of cells; and multiple controllable bus lines, in which a first subset of the multiple controllable bus lines include multiple general purpose controllable bus lines couplable to the local memory.

Abstract: A method includes: receiving control data at a first data selector of a plurality of data selectors, in which the control data comprises (i) a configuration registry address specifying a location in a configuration state registry and (ii) configuration data specifying a circuit configuration state of a circuit element of a computational circuit; transferring the control data, from the first data selector, to an entry in a trigger table registry; responsive to a first trigger event occurring, transferring the configuration data to the location in the configuration state registry specified by the configuration registry address; and updating a state of the circuit element based on the configuration data.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining, for each pair of adjacent chips in a plurality of chips connected in a series-ring arrangement of a semiconductor device, a corresponding loop latency for round trip data transmissions between the pair of chips. Identifying, from among the loop latencies, a maximum loop latency. Determining a ring latency for a data transmission originating from a chip of the plurality chips to be transmitted around the series-ring arrangement and back to the chip. Comparing half of the maximum loop latency to one N-th of the ring latency, where N is the number of chips in the plurality of chips, and storing the greater value as an inter-chip latency of the semiconductor device, the inter-chip latency representing an operational characteristic of the semiconductor device.

Abstract: Aspects of the disclosure include methods, systems, and apparatus, including computer-readable storage media for multi-mode integrated circuits with balanced energy consumption. A method includes determining, by one or more processors and based at least on a maximum energy threshold for planned multi-mode system having one or more processing units, a respective number of operations that can be performed per clock cycle by the processing units for each operating mode. The system is configured to consume the same amount of energy per clock cycle in each operating mode, but perform more operations in operating modes corresponding to operations performed on smaller bit-width operands.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for obtaining a first schedule, for a first hardware block of an integrated circuit device, where the first schedule identifies a first set of operations to be performed by the first hardware block. Obtaining a second schedule for a second hardware block of the integrated circuit device, where the second schedule identifies a second set of operations to be performed by the second hardware block and where operations of the second schedule are coordinated with operations of the first schedule such that the first schedule triggers the first hardware block to send data to the second block at a first pre-scheduled value of a counter, and the second schedule triggers the second hardware block to accept the data at an input at a second pre-scheduled value of the counter that is after the first pre-scheduled value.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for scheduling operations to reduce propagation latency between tiles of an accelerator. One of the methods includes receiving a request to generate a schedule for a first layer of a program to be executed by an accelerator configured to perform matrix operations at least partially in parallel, wherein the program defines a plurality of layers including the first layer, each layer of the program defining matrix operations to be performed using a respective matrix of values. A plurality of initial blocks of the schedule are assigned according to an initial assignment direction. The assignment direction is switched starting at a particular cycle so that blocks processed after the selected particular cycle are processed along a different second dimension of the first matrix. All remaining unassigned blocks are then assigned according to the switched assignment direction.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sharding dataflow graphs for a device having multiple synchronous tiles. One of the methods includes receiving a representation of a dataflow graph comprising a plurality of nodes that each represent respective matrix operations to be performed by a device having a plurality synchronous tiles. Candidate allocations of respective portions of the dataflow graph to each tile of the plurality of synchronous tiles are evaluated according to one or more resource constraints of the device. One of the candidate allocations is selected based on evaluating each candidate allocation.

Abstract: A tile including circuitry for use with machine learning models, the tile including: a first computational array of cells, in which the computational array of cells is a sub-array of a larger second computational array of cells; local memory coupled to the first computational array of cells; and multiple controllable bus lines, in which a first subset of the multiple controllable bus lines include multiple general purpose controllable bus lines couplable to the local memory.

Abstract: A method of configuring an integrated circuit including multiple hardware tiles, includes: establishing a data forwarding path through the multiple hardware tiles by configuring each hardware tile, except for a last hardware tile, of the multiple hardware tiles to be in a data forwarding state, in which configuring each hardware tile, except for the last hardware tile, to be in a forwarding state includes installing a respective forwarding state counter specifying a corresponding predefined length of time that the hardware tile is in the data forwarding state; supplying, along the data forwarding path, each hardware tile of the plurality of hardware tiles with a respective program data packet comprising program data for the hardware tile; and installing, for each hardware tile of the multiple hardware tiles, the respective program data.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining, for each pair of adjacent chips in a plurality of chips connected in a series-ring arrangement of a semiconductor device, a corresponding loop latency for round trip data transmissions between the pair of chips. Identifying, from among the loop latencies, a maximum loop latency. Determining a ring latency for a data transmission originating from a chip of the plurality chips to be transmitted around the series-ring arrangement and back to the chip. Comparing half of the maximum loop latency to one N-th of the ring latency, where N is the number of chips in the plurality of chips, and storing the greater value as an inter-chip latency of the semiconductor device, the inter-chip latency representing an operational characteristic of the semiconductor device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining, for each pair of adjacent chips in a plurality of chips connected in a series-ring arrangement of a semiconductor device, a corresponding loop latency for round trip data transmissions between the pair of chips. Identifying, from among the loop latencies, a maximum loop latency. Determining a ring latency for a data transmission originating from a chip of the plurality chips to be transmitted around the series-ring arrangement and back to the chip. Comparing half of the maximum loop latency to one N-th of the ring latency, where N is the number of chips in the plurality of chips, and storing the greater value as an inter-chip latency of the semiconductor device, the inter-chip latency representing an operational characteristic of the semiconductor device.

Abstract: An application specific integrated circuit (ASIC) chip includes: a systolic array of cells; and multiple controllable bus lines configured to convey data among the systolic array of cells, in which the systolic array of cells is arranged in multiple tiles, each tile of the multiple tiles including 1) a corresponding subarray of cells of the systolic array of cells, 2) a corresponding subset of controllable bus lines of the multiple controllable bus lines, and 3) memory coupled to the subarray of cells.