Patents by Inventor Norman Paul Jouppi

Norman Paul Jouppi has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10261786
    Abstract: A vector processing unit is described, and includes processor units that each include multiple processing resources. The processor units are each configured to perform arithmetic operations associated with vectorized computations. The vector processing unit includes a vector memory in data communication with each of the processor units and their respective processing resources. The vector memory includes memory banks configured to store data used by each of the processor units to perform the arithmetic operations. The processor units and the vector memory are tightly coupled within an area of the vector processing unit such that data communications are exchanged at a high bandwidth based on the placement of respective processor units relative to one another, and based on the placement of the vector memory relative to each processor unit.
    Type: Grant
    Filed: March 9, 2017
    Date of Patent: April 16, 2019
    Assignee: Google LLC
    Inventors: William Lacy, Gregory Michael Thorson, Christopher Aaron Clark, Norman Paul Jouppi, Thomas Norrie, Andrew Everett Phelps
  • Publication number: 20180336163
    Abstract: Methods, systems, and apparatus for a matrix multiply unit implemented as a systolic array of cells are disclosed. Each cell of the matrix multiply includes: a weight matrix register configured to receive a weight input from either a transposed or a non-transposed weight shift register; a transposed weight shift register configured to receive a weight input from a horizontal direction to be stored in the weight matrix register; a non-transposed weight shift register configured to receive a weight input from a vertical direction to be stored in the weight matrix register; and a multiply unit that is coupled to the weight matrix register and configured to multiply the weight input of the weight matrix register with a vector data input in order to obtain a multiplication result.
    Type: Application
    Filed: May 17, 2018
    Publication date: November 22, 2018
    Inventors: Andrew Everett Phelps, Norman Paul Jouppi
  • Publication number: 20180336456
    Abstract: Methods, systems, and apparatus including a special purpose hardware chip for training neural networks are described. The special-purpose hardware chip may include a scalar processor configured to control computational operation of the special-purpose hardware chip. The chip may also include a vector processor configured to have a 2-dimensional array of vector processing units which all execute the same instruction in a single instruction, multiple-data manner and communicate with each other through load and store instructions of the vector processor. The chip may additionally include a matrix multiply unit that is coupled to the vector processor configured to multiply at least one two-dimensional matrix with a second one-dimensional vector or two-dimensional matrix in order to obtain a multiplication result.
    Type: Application
    Filed: May 17, 2018
    Publication date: November 22, 2018
    Inventors: Thomas Norrie, Olivier Temam, Andrew Everett Phelps, Norman Paul Jouppi
  • Publication number: 20180336164
    Abstract: Methods, systems, and apparatus for a matrix multiply unit implemented as a systolic array of cells are disclosed. The matrix multiply unit may include cells arranged in columns of the systolic array. Two chains of weight shift registers per column of the systolic array are in the matrix multiply unit. Each weight shift register is connected to only one chain and each cell is connected to only one weight shift register. A weight matrix register per cell is configured to store a weight input received from a weight shift register. A multiply unit is coupled to the weight matrix register and configured to multiply the weight input of the weight matrix register with a vector data input in order to obtain a multiplication result.
    Type: Application
    Filed: May 17, 2018
    Publication date: November 22, 2018
    Inventors: Andrew Everett Phelps, Norman Paul Jouppi
  • Publication number: 20180336165
    Abstract: Methods, systems, and apparatus for performing a matrix multiplication using a hardware circuit are described. An example method begins by obtaining an input activation value and a weight input value in a first floating point format. The input activation value and the weight input value are multiplied to generate a product value in a second floating point format that has higher precision than the first floating point format. A partial sum value is obtained in a third floating point format that has a higher precision than the first floating point format. The partial sum value and the product value are combined to generate an updated partial sum value that has the third floating point format.
    Type: Application
    Filed: May 17, 2018
    Publication date: November 22, 2018
    Inventors: Andrew Everett Phelps, Norman Paul Jouppi
  • Patent number: 10127154
    Abstract: A memory system includes a plurality of memory nodes provided at different hierarchical levels of the memory system, each of the memory nodes including a corresponding memory storage and a cache. A memory node at a first of the different hierarchical levels is coupled to a processor with lower communication latency than a memory node at a second of the different hierarchical levels. The memory nodes are to cooperate to decide which of the memory nodes is to cache data of a given one of the memory nodes.
    Type: Grant
    Filed: March 20, 2013
    Date of Patent: November 13, 2018
    Assignee: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP
    Inventors: Norman Paul Jouppi, Sheng Li, Ke Chen
  • Publication number: 20180260220
    Abstract: A vector processing unit is described, and includes processor units that each include multiple processing resources. The processor units are each configured to perform arithmetic operations associated with vectorized computations. The vector processing unit includes a vector memory in data communication with each of the processor units and their respective processing resources. The vector memory includes memory banks configured to store data used by each of the processor units to perform the arithmetic operations. The processor units and the vector memory are tightly coupled within an area of the vector processing unit such that data communications are exchanged at a high bandwidth based on the placement of respective processor units relative to one another, and based on the placement of the vector memory relative to each processor unit.
    Type: Application
    Filed: March 9, 2017
    Publication date: September 13, 2018
    Inventors: William Lacy, Gregory Michael Thorson, Christopher Aaron Clark, Norman Paul Jouppi, Thomas Norrie, Andrew Everett Phelps
  • Publication number: 20180240039
    Abstract: Methods, systems, and apparatus, including instructions encoded on storage media, for performing reduction of gradient vectors and similarly structured data that are generated in parallel, for example, on nodes organized in a mesh or torus topology defined by connections in at least two dimension between the nodes. The methods provide parallel computation and communication between nodes in the topology.
    Type: Application
    Filed: September 18, 2017
    Publication date: August 23, 2018
    Inventors: Ian Moray Mclaren, Norman Paul Jouppi, Clifford Hsiang Chao, Gregory Michael Thorson, Bjarke Hammersholt Roune
  • Patent number: 10055692
    Abstract: Methods, systems, and apparatus, including instructions encoded on storage media, for performing reduction of gradient vectors and similarly structured data that are generated in parallel, for example, on nodes organized in a mesh or torus topology defined by connections in at least two dimension between the nodes. The methods provide parallel computation and communication between nodes in the topology.
    Type: Grant
    Filed: September 18, 2017
    Date of Patent: August 21, 2018
    Assignee: Google LLC
    Inventors: Ian Moray Mclaren, Norman Paul Jouppi, Clifford Hsiang Chao, Gregory Michael Thorson, Bjarke Hammersholt Roune
  • Patent number: 10032695
    Abstract: A semiconductor package includes a substrate, an integrated circuit disposed on the substrate, a memory support disposed on the integrated circuit, stacked memory disposed on the memory support and in communication with the integrated circuit, and a lid connected to the substrate. The integrated circuit has a low power region and a high power region. The memory support is disposed on the low power region of the integrated circuit and is configured to allow a flow of fluid therethrough to conduct heat away from the low power region of the integrated circuit. The lid defines a first port, a second port, and a lid volume fluidly connecting the first port and the second port. The lid volume is configured to house the integrated circuit, the memory support, and the stacked memory, while directing the flow of fluid to flow over the integrated circuit, the memory support, and the stacked memory.
    Type: Grant
    Filed: February 19, 2016
    Date of Patent: July 24, 2018
    Assignee: Google LLC
    Inventors: Madhu Krishnan Iyengar, Teck-Gyu Kang, Christopher Gregory Malone, Norman Paul Jouppi
  • Patent number: 9934085
    Abstract: A detector detects, using an error code, an error in data stored in a memory. The detector determines whether the error is uncorrectable using the error code. In response to determining that the error is uncorrectable, an error handler associated with an application is invoked to handle the error in the data by recovering the data to an application-wide consistent state.
    Type: Grant
    Filed: May 29, 2013
    Date of Patent: April 3, 2018
    Assignee: Hewlett Packard Enterprise Development LP
    Inventors: Doe-Hyun Yoon, Jichuan Chang, Naveen Muralimanohar, Parthasarathy Ranganathan, Robert Schreiber, Norman Paul Jouppi
  • Publication number: 20180046907
    Abstract: A circuit for performing neural network computations for a neural network comprising a plurality of neural network layers, the circuit comprising: a matrix computation unit configured to, for each of the plurality of neural network layers: receive a plurality of weight inputs and a plurality of activation inputs for the neural network layer, and generate a plurality of accumulated values based on the plurality of weight inputs and the plurality of activation inputs; and a vector computation unit communicatively coupled to the matrix computation unit and configured to, for each of the plurality of neural network layers: apply an activation function to each accumulated value generated by the matrix computation unit to generate a plurality of activated values for the neural network layer.
    Type: Application
    Filed: August 25, 2017
    Publication date: February 15, 2018
    Inventors: Jonathan Ross, Norman Paul Jouppi, Andrew Everett Phelps, Reginald Clifford Young, Thomas Norrie, Gregory Michael Thorson, Dan Luu
  • Patent number: 9852792
    Abstract: A non-volatile multi-level cell (“MLC”) memory device is disclosed. The memory device has an array of non-volatile memory cells, an array of non-volatile memory cells, with each non-volatile memory cell storing multiple groups of bits. A row buffer in the memory device has multiple buffer portions, each buffer portion storing one or more bits from the memory cells and having different read and write latencies and energies.
    Type: Grant
    Filed: January 31, 2013
    Date of Patent: December 26, 2017
    Assignee: Hewlett Packard Enterprise Development LP
    Inventors: Naveen Muralimanohar, Han Bin Yoon, Norman Paul Jouppi
  • Patent number: 9846550
    Abstract: A disclosed example apparatus includes a row address register (412) to store a row address corresponding to a row (608) in a memory array (602). The example apparatus also includes a row decoder (604) coupled to the row address register to assert a signal on a wordline (704) of the row after the memory receives a column address. In addition, the example apparatus includes a column decoder (606) to selectively activate a portion of the row based on the column address and the signal asserted on the wordline.
    Type: Grant
    Filed: April 4, 2016
    Date of Patent: December 19, 2017
    Assignees: Hewlett Packard Enterprise Development LP, University of Utah
    Inventors: Naveen Muralimanohar, Aniruddha Nagendran Udipi, Niladrish Chatterjee, Rajeev Balasubramonian, Alan Lynn Davis, Norman Paul Jouppi
  • Patent number: 9832550
    Abstract: A system can include an optical multiplexer to combine a plurality of optical input signals having respective wavelengths into a wide-channel optical input signal that is provided to an input channel. The system also includes a photonic packet switch comprising a switch core and a plurality of ports defining a switch radix of the photonic packet switch. The input channel and an output channel can be associated with one of the plurality of ports. The photonic packet switch can process the wide-channel optical input signal and can generate a wide-channel optical output signal that is provided to the output channel. The system further includes an optical demultiplexer to separate the wide-channel optical output signal into a plurality of optical output signals having respective wavelengths. The optical multiplexer and the optical demultiplexer can collectively provide the system with a radix greater than the switch radix.
    Type: Grant
    Filed: January 31, 2013
    Date of Patent: November 28, 2017
    Assignee: Hewlett Packard Enterprise Development LP
    Inventors: Moray McLaren, Raymond G Beausoleil, Norman Paul Jouppi, Marco Fiorentino, Alan Lynn Davis, Naveen Muralimanohar, Sheng Li
  • Patent number: 9823986
    Abstract: According to an example, a resiliency group for a memory node in a memory network can provide error correction for a memory access in the memory node. The memory access may be received from a main memory controller of a processor connected to the memory network. The memory access may be executed by a memory controller of the memory node.
    Type: Grant
    Filed: April 30, 2013
    Date of Patent: November 21, 2017
    Assignee: Hewlett Packard Enterprise Development LP
    Inventors: Sheng Li, Norman Paul Jouppi, Paolo Faraboschi, Doe Hyun Yoon, Dwight L. Barron
  • Patent number: 9747546
    Abstract: A circuit for performing neural network computations for a neural network comprising a plurality of neural network layers, the circuit comprising: a matrix computation unit configured to, for each of the plurality of neural network layers: receive a plurality of weight inputs and a plurality of activation inputs for the neural network layer, and generate a plurality of accumulated values based on the plurality of weight inputs and the plurality of activation inputs; and a vector computation unit communicatively coupled to the matrix computation unit and configured to, for each of the plurality of neural network layers: apply an activation function to each accumulated value generated by the matrix computation unit to generate a plurality of activated values for the neural network layer.
    Type: Grant
    Filed: September 3, 2015
    Date of Patent: August 29, 2017
    Assignee: Google Inc.
    Inventors: Jonathan Ross, Norman Paul Jouppi, Andrew Everett Phelps, Reginald Clifford Young, Thomas Norrie, Gregory Michael Thorson, Dan Luu
  • Publication number: 20170243806
    Abstract: A semiconductor package includes a substrate, an integrated circuit disposed on the substrate, a memory support disposed on the integrated circuit, stacked memory disposed on the memory support and in communication with the integrated circuit, and a lid connected to the substrate. The integrated circuit has a low power region and a high power region. The memory support is disposed on the low power region of the integrated circuit and is configured to allow a flow of fluid therethrough to conduct heat away from the low power region of the integrated circuit. The lid defines a first port, a second port, and a lid volume fluidly connecting the first port and the second port. The lid volume is configured to house the integrated circuit, the memory support, and the stacked memory, while directing the flow of fluid to flow over the integrated circuit, the memory support, and the stacked memory.
    Type: Application
    Filed: February 19, 2016
    Publication date: August 24, 2017
    Applicant: Google Inc.
    Inventors: Madhu Krishnan Iyengar, Teck-Gyu Kang, Christopher Gregory Malone, Norman Paul Jouppi
  • Patent number: 9710748
    Abstract: A circuit for performing neural network computations for a neural network comprising a plurality of neural network layers, the circuit comprising: a matrix computation unit configured to, for each of the plurality of neural network layers: receive a plurality of weight inputs and a plurality of activation inputs for the neural network layer, and generate a plurality of accumulated values based on the plurality of weight inputs and the plurality of activation inputs; and a vector computation unit communicatively coupled to the matrix computation unit and configured to, for each of the plurality of neural network layers: apply an activation function to each accumulated value generated by the matrix computation unit to generate a plurality of activated values for the neural network layer.
    Type: Grant
    Filed: December 22, 2016
    Date of Patent: July 18, 2017
    Assignee: Google Inc.
    Inventors: Jonathan Ross, Norman Paul Jouppi, Andrew Everett Phelps, Reginald Clifford Young, Thomas Norrie, Gregory Michael Thorson, Dan Luu
  • Publication number: 20170103313
    Abstract: A circuit for performing neural network computations for a neural network comprising a plurality of neural network layers, the circuit comprising: a matrix computation unit configured to, for each of the plurality of neural network layers: receive a plurality of weight inputs and a plurality of activation inputs for the neural network layer, and generate a plurality of accumulated values based on the plurality of weight inputs and the plurality of activation inputs; and a vector computation unit communicatively coupled to the matrix computation unit and configured to, for each of the plurality of neural network layers: apply an activation function to each accumulated value generated by the matrix computation unit to generate a plurality of activated values for the neural network layer.
    Type: Application
    Filed: December 22, 2016
    Publication date: April 13, 2017
    Inventors: Jonathan Ross, Norman Paul Jouppi, Andrew Everett Phelps, Reginald Clifford Young, Thomas Norrie, Gregory Michael Thorson, Dan Luu