Patents by Inventor Xiran Huang
Xiran Huang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20250224959Abstract: Neural network accelerators with one or more neural network accelerator cores. Each neural network accelerator core has hardware accelerators configured to accelerate neural network operations, an embedded processor, a command decoder, and a hardware feedback path between the embedded processor and the command decoder. The command decoder is configured to control the hardware accelerators and the embedded processor of that core in accordance with commands of a command stream, and when the command stream comprises a set of one or more branch commands that indicate a conditional branch is to be performed, cause the embedded processor to determine a next command stream, and in response to receiving information from the embedded processor identifying the next command stream via the hardware feedback path, control the one or more hardware accelerators and the embedded processor in accordance with commands of the next command stream.Type: ApplicationFiled: December 22, 2024Publication date: July 10, 2025Inventor: Xiran Huang
-
Publication number: 20240354560Abstract: A hardware implementation of a neural network and a method of processing data in such a hardware implementation are disclosed. Input data for a plurality of layers of the network is processed in blocks, to generate respective blocks of output data. The processing proceeds depth-wise through the plurality of layers, evaluating all layers of the plurality of layers for a given block, before proceeding to the next block.Type: ApplicationFiled: July 3, 2024Publication date: October 24, 2024Inventors: Xiran Huang, Cagatay Dikici
-
Patent number: 12061972Abstract: A hardware implementation of a neural network and a method of processing data in such a hardware implementation are disclosed. Input data for a plurality of layers of the network is processed in blocks, to generate respective blocks of output data. The processing proceeds depth-wise through the plurality of layers, evaluating all layers of the plurality of layers for a given block, before proceeding to the next block.Type: GrantFiled: November 30, 2020Date of Patent: August 13, 2024Assignee: Imagination Technologies LimitedInventors: Xiran Huang, Cagatay Dikici
-
Publication number: 20240232597Abstract: A neural network is mapped to hardware by defining a plurality of layer groups, each layer group comprising one or more layers of the neural network that are processed in a single pass through the hardware. The layer groups are grouped into tile groups, each tile group comprising a set of layers groups that are evaluated when executing the neural network. Grouping the layer groups into a tile group comprises selecting a layer group that precedes a first layer group in the tile group and determining a number of times that input data to the layer group is read from memory. In response to this number exceeding a threshold, it is determined whether to merger the layer group into the tile group by determining an amount of space in on-chip memory required for storing pre-fetched input data and assessing one or more criteria relating to output data of the layer group.Type: ApplicationFiled: December 21, 2023Publication date: July 11, 2024Inventor: Xiran Huang
-
Publication number: 20240232596Abstract: Methods of implementing a neural network in hardware, the neural network including a plurality of layers and the layers being grouped into a plurality of layer groups, each layer group comprising one or more layers of the neural network that are processed in a single pass through the hardware. The layer groups are grouped into a plurality of tile groups, each tile group comprising a set of layer groups that are evaluated when executing the neural network. The method comprises pre-fetching a portion of the input data for a first layer group in a tile group into a buffer slot in on-chip memory; and subsequently releasing the buffer slot after output data for the first layer group has been written to memory.Type: ApplicationFiled: December 21, 2023Publication date: July 11, 2024Inventor: Xiran Huang
-
Patent number: 11875248Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions that are traversed by strides in at least one layer of a first layer group, and one or more non-traversed dimensions. If a size of the input data in a first dimension is greater than a threshold, the hardware implementation splits the input data for the first layer group into at least a first tile and a second tile, along the first dimension. If the size of the input data in the first dimension is not greater than the threshold, the hardware implementation splits the evaluation of the first layer group into at least a first pass and a second pass, along a dimension other than the first dimension.Type: GrantFiled: October 13, 2021Date of Patent: January 16, 2024Assignee: Imagination Technologies LimitedInventors: Xiran Huang, Fernando Escobar
-
Patent number: 11853866Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions, being dimensions that are traversed by strides in at least one layer of a first layer group. The hardware implementation is configured to split the input data for the first layer group into at least a first tile and a second tile, along at least one of the traversed dimensions, each tile comprising a plurality of data elements in each of the one or more traversed dimensions. A first core is configured to evaluate multiple layer groups, depth-first, based on the first tile. A second core is configured to evaluate multiple layer groups, depth-first, based on the second tile.Type: GrantFiled: October 13, 2021Date of Patent: December 26, 2023Assignee: Imagination Technologies LimitedInventors: Xiran Huang, Fernando Escobar
-
Publication number: 20220147832Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions, being dimensions that are traversed by strides in at least one layer of a first layer group. The hardware implementation is configured to split the input data for the first layer group into at least a first tile and a second tile, along at least one of the traversed dimensions, each tile comprising a plurality of data elements in each of the one or more traversed dimensions. A first core is configured to evaluate multiple layer groups, depth-first, based on the first tile. A second core is configured to evaluate multiple layer groups, depth-first, based on the second tile.Type: ApplicationFiled: October 13, 2021Publication date: May 12, 2022Inventors: Xiran Huang, Fernando Escobar
-
Publication number: 20220129741Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions that are traversed by strides in at least one layer of a first layer group, and one or more non-traversed dimensions. The hardware implementation splits the evaluation of the first layer group into a first pass and a second pass, along one of the traversed dimensions or one of the non-traversed dimensions. A first core evaluates the first layer group for the first pass, to generate a first portion of output data. A second core evaluates the first layer group for the second pass, to generate a second portion of output data. The hardware implementation combines the first portion of output data and the second portion of output data to produce the output data of the first layer group.Type: ApplicationFiled: October 13, 2021Publication date: April 28, 2022Inventors: Xiran Huang, Fernando Escobar
-
Publication number: 20220121914Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions that are traversed by strides in at least one layer of a first layer group, and one or more non-traversed dimensions. If a size of the input data in a first dimension is greater than a threshold, the hardware implementation splits the input data for the first layer group into at least a first tile and a second tile, along the first dimension. If the size of the input data in the first dimension is not greater than the threshold, the hardware implementation splits the evaluation of the first layer group into at least a first pass and a second pass, along a dimension other than the first dimension.Type: ApplicationFiled: October 13, 2021Publication date: April 21, 2022Inventors: Xiran Huang, Fernando Escobar
-
Publication number: 20210174180Abstract: A hardware implementation of a neural network and a method of processing data in such a hardware implementation are disclosed. Input data for a plurality of layers of the network is processed in blocks, to generate respective blocks of output data. The processing proceeds depth-wise through the plurality of layers, evaluating all layers of the plurality of layers for a given block, before proceeding to the next block.Type: ApplicationFiled: November 30, 2020Publication date: June 10, 2021Inventors: Xiran Huang, Cagatay Dikici
-
Publication number: 20210174181Abstract: A hardware implementation of a neural network and a method of processing data in such a hardware implementation are disclosed. Input data for a plurality of layers of the network is processed in blocks, to generate respective blocks of output data. The processing proceeds depth-wise through the plurality of layers, evaluating all layers of the plurality of layers for a given block, before proceeding to the next block.Type: ApplicationFiled: November 30, 2020Publication date: June 10, 2021Inventors: Xiran Huang, Cagatay Dikici
-
Publication number: 20190050199Abstract: System and techniques for a localized grid sort are described herein. Elements that have a first-dimension value and a second-dimension value, and correspond to a cell value, are obtained for sorting. Each element is placed into one of a set of first-in-first-out (FIFO) buffers based on a difference between the first-dimension value for the element and a first coordinate of the cell value in the first-dimension. The set of FIFO buffers are then merged by outputting a lowest value element each comparison. This creates a stream of elements (e.g., elements stream) sorted in the first dimension. The elements from the element stream are placed into a set of window buffers. In response to the next element in the element stream not being in the set of window buffers, a lowest buffer from the set of buffers is flushed to produce an output stream of sorted elements.Type: ApplicationFiled: March 30, 2018Publication date: February 14, 2019Inventors: Richard Allen, Xiran Huang