Patents by Inventor Xiran Huang

Xiran Huang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20250224959
    Abstract: Neural network accelerators with one or more neural network accelerator cores. Each neural network accelerator core has hardware accelerators configured to accelerate neural network operations, an embedded processor, a command decoder, and a hardware feedback path between the embedded processor and the command decoder. The command decoder is configured to control the hardware accelerators and the embedded processor of that core in accordance with commands of a command stream, and when the command stream comprises a set of one or more branch commands that indicate a conditional branch is to be performed, cause the embedded processor to determine a next command stream, and in response to receiving information from the embedded processor identifying the next command stream via the hardware feedback path, control the one or more hardware accelerators and the embedded processor in accordance with commands of the next command stream.
    Type: Application
    Filed: December 22, 2024
    Publication date: July 10, 2025
    Inventor: Xiran Huang
  • Publication number: 20240354560
    Abstract: A hardware implementation of a neural network and a method of processing data in such a hardware implementation are disclosed. Input data for a plurality of layers of the network is processed in blocks, to generate respective blocks of output data. The processing proceeds depth-wise through the plurality of layers, evaluating all layers of the plurality of layers for a given block, before proceeding to the next block.
    Type: Application
    Filed: July 3, 2024
    Publication date: October 24, 2024
    Inventors: Xiran Huang, Cagatay Dikici
  • Patent number: 12061972
    Abstract: A hardware implementation of a neural network and a method of processing data in such a hardware implementation are disclosed. Input data for a plurality of layers of the network is processed in blocks, to generate respective blocks of output data. The processing proceeds depth-wise through the plurality of layers, evaluating all layers of the plurality of layers for a given block, before proceeding to the next block.
    Type: Grant
    Filed: November 30, 2020
    Date of Patent: August 13, 2024
    Assignee: Imagination Technologies Limited
    Inventors: Xiran Huang, Cagatay Dikici
  • Publication number: 20240232597
    Abstract: A neural network is mapped to hardware by defining a plurality of layer groups, each layer group comprising one or more layers of the neural network that are processed in a single pass through the hardware. The layer groups are grouped into tile groups, each tile group comprising a set of layers groups that are evaluated when executing the neural network. Grouping the layer groups into a tile group comprises selecting a layer group that precedes a first layer group in the tile group and determining a number of times that input data to the layer group is read from memory. In response to this number exceeding a threshold, it is determined whether to merger the layer group into the tile group by determining an amount of space in on-chip memory required for storing pre-fetched input data and assessing one or more criteria relating to output data of the layer group.
    Type: Application
    Filed: December 21, 2023
    Publication date: July 11, 2024
    Inventor: Xiran Huang
  • Publication number: 20240232596
    Abstract: Methods of implementing a neural network in hardware, the neural network including a plurality of layers and the layers being grouped into a plurality of layer groups, each layer group comprising one or more layers of the neural network that are processed in a single pass through the hardware. The layer groups are grouped into a plurality of tile groups, each tile group comprising a set of layer groups that are evaluated when executing the neural network. The method comprises pre-fetching a portion of the input data for a first layer group in a tile group into a buffer slot in on-chip memory; and subsequently releasing the buffer slot after output data for the first layer group has been written to memory.
    Type: Application
    Filed: December 21, 2023
    Publication date: July 11, 2024
    Inventor: Xiran Huang
  • Patent number: 11875248
    Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions that are traversed by strides in at least one layer of a first layer group, and one or more non-traversed dimensions. If a size of the input data in a first dimension is greater than a threshold, the hardware implementation splits the input data for the first layer group into at least a first tile and a second tile, along the first dimension. If the size of the input data in the first dimension is not greater than the threshold, the hardware implementation splits the evaluation of the first layer group into at least a first pass and a second pass, along a dimension other than the first dimension.
    Type: Grant
    Filed: October 13, 2021
    Date of Patent: January 16, 2024
    Assignee: Imagination Technologies Limited
    Inventors: Xiran Huang, Fernando Escobar
  • Patent number: 11853866
    Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions, being dimensions that are traversed by strides in at least one layer of a first layer group. The hardware implementation is configured to split the input data for the first layer group into at least a first tile and a second tile, along at least one of the traversed dimensions, each tile comprising a plurality of data elements in each of the one or more traversed dimensions. A first core is configured to evaluate multiple layer groups, depth-first, based on the first tile. A second core is configured to evaluate multiple layer groups, depth-first, based on the second tile.
    Type: Grant
    Filed: October 13, 2021
    Date of Patent: December 26, 2023
    Assignee: Imagination Technologies Limited
    Inventors: Xiran Huang, Fernando Escobar
  • Publication number: 20220147832
    Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions, being dimensions that are traversed by strides in at least one layer of a first layer group. The hardware implementation is configured to split the input data for the first layer group into at least a first tile and a second tile, along at least one of the traversed dimensions, each tile comprising a plurality of data elements in each of the one or more traversed dimensions. A first core is configured to evaluate multiple layer groups, depth-first, based on the first tile. A second core is configured to evaluate multiple layer groups, depth-first, based on the second tile.
    Type: Application
    Filed: October 13, 2021
    Publication date: May 12, 2022
    Inventors: Xiran Huang, Fernando Escobar
  • Publication number: 20220129741
    Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions that are traversed by strides in at least one layer of a first layer group, and one or more non-traversed dimensions. The hardware implementation splits the evaluation of the first layer group into a first pass and a second pass, along one of the traversed dimensions or one of the non-traversed dimensions. A first core evaluates the first layer group for the first pass, to generate a first portion of output data. A second core evaluates the first layer group for the second pass, to generate a second portion of output data. The hardware implementation combines the first portion of output data and the second portion of output data to produce the output data of the first layer group.
    Type: Application
    Filed: October 13, 2021
    Publication date: April 28, 2022
    Inventors: Xiran Huang, Fernando Escobar
  • Publication number: 20220121914
    Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions that are traversed by strides in at least one layer of a first layer group, and one or more non-traversed dimensions. If a size of the input data in a first dimension is greater than a threshold, the hardware implementation splits the input data for the first layer group into at least a first tile and a second tile, along the first dimension. If the size of the input data in the first dimension is not greater than the threshold, the hardware implementation splits the evaluation of the first layer group into at least a first pass and a second pass, along a dimension other than the first dimension.
    Type: Application
    Filed: October 13, 2021
    Publication date: April 21, 2022
    Inventors: Xiran Huang, Fernando Escobar
  • Publication number: 20210174180
    Abstract: A hardware implementation of a neural network and a method of processing data in such a hardware implementation are disclosed. Input data for a plurality of layers of the network is processed in blocks, to generate respective blocks of output data. The processing proceeds depth-wise through the plurality of layers, evaluating all layers of the plurality of layers for a given block, before proceeding to the next block.
    Type: Application
    Filed: November 30, 2020
    Publication date: June 10, 2021
    Inventors: Xiran Huang, Cagatay Dikici
  • Publication number: 20210174181
    Abstract: A hardware implementation of a neural network and a method of processing data in such a hardware implementation are disclosed. Input data for a plurality of layers of the network is processed in blocks, to generate respective blocks of output data. The processing proceeds depth-wise through the plurality of layers, evaluating all layers of the plurality of layers for a given block, before proceeding to the next block.
    Type: Application
    Filed: November 30, 2020
    Publication date: June 10, 2021
    Inventors: Xiran Huang, Cagatay Dikici
  • Publication number: 20190050199
    Abstract: System and techniques for a localized grid sort are described herein. Elements that have a first-dimension value and a second-dimension value, and correspond to a cell value, are obtained for sorting. Each element is placed into one of a set of first-in-first-out (FIFO) buffers based on a difference between the first-dimension value for the element and a first coordinate of the cell value in the first-dimension. The set of FIFO buffers are then merged by outputting a lowest value element each comparison. This creates a stream of elements (e.g., elements stream) sorted in the first dimension. The elements from the element stream are placed into a set of window buffers. In response to the next element in the element stream not being in the set of window buffers, a lowest buffer from the set of buffers is flushed to produce an output stream of sorted elements.
    Type: Application
    Filed: March 30, 2018
    Publication date: February 14, 2019
    Inventors: Richard Allen, Xiran Huang