Patents by Inventor Torsten HOEFLER
Torsten HOEFLER has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20250047603Abstract: A computing network implements a congestion control mechanism and a load balancing mechanism. The load balancing mechanism is run at the packet level. A connection-level measure is generated for congestion in the computing network. The connection-level measure is accumulated at the packet level. Activation of the congestion control mechanism is limited until the accumulated connection-level measure reaches a threshold.Type: ApplicationFiled: November 13, 2023Publication date: February 6, 2025Inventors: Abdul KABBANI, Torsten HOEFLER
-
Publication number: 20250047604Abstract: A computing network implements a congestion control mechanism and a load balancing mechanism. It is determined which available routes in the network are congested. Activation of the congestion control mechanism is limited until a threshold number of the available routes are determined to be congested, which prevents over-attenuation of an overall sending rate or window by the congestion control mechanism.Type: ApplicationFiled: November 13, 2023Publication date: February 6, 2025Inventors: Abdul KABBANI, Torsten HOEFLER
-
Publication number: 20250047598Abstract: It is determined that a computing node is contributing to a network congestion event. A congestion notification message is generated. A timing profile is determined for sending the congestion notification message based on the level of the network congestion event. Based on the timing profile, the congestion notification message is forwarded to the computing node determined to be contributing to the network congestion event.Type: ApplicationFiled: November 13, 2023Publication date: February 6, 2025Inventors: Abdul KABBANI, Torsten HOEFLER
-
Publication number: 20250047600Abstract: A first ratio of a sending rate limit to a full line rate for a link in the computing network is accessed. A second ratio of a sending window size to W_max for the link is accessed. W_max is the maximum allowed window size or the window size that utilizes an end-to-end path for the link. One or more of the first or second ratio is used to determine an amount to reduce the sending rate or window for the link in response to an indication of network congestion in the link.Type: ApplicationFiled: November 13, 2023Publication date: February 6, 2025Inventors: Abdul KABBANI, Torsten HOEFLER
-
Publication number: 20250047610Abstract: An entropy value is generated for a data packet to be transmitted on a computing network. The entropy value is usable to select or change a network path for the data packet. In response to receiving an acknowledgement message for the data packet, the entropy value is saved in a storage structure if the entropy value is acknowledged as not congested. When transmitting an additional data packet, the oldest saved entropy from the storage structure is reused and the oldest saved entropy value is invalidated.Type: ApplicationFiled: November 13, 2023Publication date: February 6, 2025Inventors: Abdul KABBANI, Torsten HOEFLER
-
Publication number: 20250047613Abstract: Acknowledgement messages for a link in a computing network are accessed. Round trip time (RTT) measurements for the link are accessed. In response to determining that none of the acknowledgement messages are Explicit Congestion Notification (ECN)-marked and none of the accessed RTT measurements exceeded a minimum expected latency threshold, it is determined that an end-to-end path of the link is not congested.Type: ApplicationFiled: November 13, 2023Publication date: February 6, 2025Inventors: Abdul KABBANI, Torsten HOEFLER
-
Publication number: 20250047616Abstract: A match table is used to match a concatenation of a source address of a data packet and a consecutive message sequence number (MSN) to a match data structure. The matched concatenation is inserted into the match data structure based on a source and a running counter for a MSN to each source. The currently active PDC associated with the source is attached or an empty PDC is atomically created and attached. In response to a packet arriving before the semantic layer has posted a recv( ), A canary value “failed match” is atomically inserted into the match data structure and a request to wait (RTW) message is sent to the source. In response to the semantic layer posting the recv( ), the “failed match” entry in the match data structure is identified, the “failed match” entry is atomically updated with match information, and a request to send (RTS) message is sent to the source.Type: ApplicationFiled: November 13, 2023Publication date: February 6, 2025Inventors: Torsten HOEFLER, Abdul KABBANI
-
Publication number: 20240314073Abstract: A computer implemented method includes encoding a packet in a source endpoint of a multi-path communication network, the packet having a hash seed for use by routers to route the packet through the multi-path communication network to a destination endpoint. Network performance is tracked for the packet at the source endpoint. The hash seed is modified as a function of the network performance. The packet is re-sent such that the modified hash seed is used to route the packet to the destination endpoint.Type: ApplicationFiled: March 16, 2023Publication date: September 19, 2024Inventors: Abdul KABBANI, Torsten HOEFLER
-
Publication number: 20240303117Abstract: Operations of a workload are assigned to physical resources of a physical device array. The workload includes a graph of operations to be performed on a physical device array. The graph of operations is partitioned into subgraphs. Partitioning includes at least minimizing the quantity of subgraphs and maximizing resource utilization per subgraph. A logical mapping of the subgraph to logical processing engine (PE) units is generated using features of the subgraph and tiling factors of the logical PE units. The logical mapping is assigned to physical PE units of the physical device array at least by minimizing network traffic across the physical PE units. The operations of the subgraph are performed using the physical PE units to which the logical mapping is assigned. This process enhances the computational efficiency of the array when executing the workload.Type: ApplicationFiled: February 24, 2023Publication date: September 12, 2024Inventors: Fanny NINA PARAVECINO, Michael Eric DAVIES, Abhishek Dilip KULKARNI, Md Aamir RAIHAN, Ankit MORE, Aayush ANKIT, Torsten HOEFLER, Douglas Christopher BURGER
-
Patent number: 11989414Abstract: Embodiments of the present disclosure include a digital circuit and method for multi-stage compression. Digital data values are compressed using a multi-stage compression algorithm and stored in a memory. A decompression circuit receives the values and performs a partial decompression. The partially compressed values are provided to a processor, which performs the final decompression. In one embodiment, a vector of N length compressed values are decompressed using a first bit mask into two N length sets having non-zero values. The two N length sets are further decompressed using two M length bit masks into M length sparse vectors, each having non-zero values.Type: GrantFiled: June 23, 2023Date of Patent: May 21, 2024Assignee: Microsoft Technology Licensing, LLCInventors: Mattheus C. Heddes, Ankit More, Nishit Shah, Torsten Hoefler
-
Patent number: 11886938Abstract: One example provides an integrated computing device, comprising one or more computing clusters, and one or more network controllers, each network controller comprising a local data notification queue to queue send message notifications originating from the computing clusters on the integrated computing device, a remote data notification queue to queue receive message notifications originating from network controllers on remote integrated computing devices, a local no-data notification queue to queue receive message notifications originating from computing clusters on the integrated computing device, and a connection scheduler configured to schedule sending of data from memory on the integrated computing device when a send message notification in the local data notification queue is matched with a receive message notification in the remote data notification queue, and to schedule sending of receive message notifications from the local no-data notification queue.Type: GrantFiled: March 11, 2021Date of Patent: January 30, 2024Assignee: Microsoft Technology Licensing, LLCInventors: Deepak Goel, Mattheus C Heddes, Torsten Hoefler, Xiaoling Xu
-
Publication number: 20230333739Abstract: Embodiments of the present disclosure include a digital circuit and method for multi-stage compression. Digital data values are compressed using a multi-stage compression algorithm and stored in a memory. A decompression circuit receives the values and performs a partial decompression. The partially compressed values are provided to a processor, which performs the final decompression. In one embodiment, a vector of N length compressed values are decompressed using a first bit mask into two N length sets having non-zero values. The two N length sets are further decompressed using two M length bit masks into M length sparse vectors, each having non-zero values.Type: ApplicationFiled: June 23, 2023Publication date: October 19, 2023Inventors: Mattheus C. HEDDES, Ankit MORE, Nishit SHAH, Torsten HOEFLER
-
Patent number: 11720252Abstract: Embodiments of the present disclosure include a digital circuit and method for multi-stage compression. Digital data values are compressed using a multi-stage compression algorithm and stored in a memory. A decompression circuit receives the values and performs a partial decompression. The partially compressed values are provided to a processor, which performs the final decompression. In one embodiment, a vector of N length compressed values are decompressed using a first bit mask into two N length sets having non-zero values. The two N length sets are further decompressed using two M length bit masks into M length sparse vectors, each having non-zero values.Type: GrantFiled: March 4, 2022Date of Patent: August 8, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Mattheus C. Heddes, Ankit More, Nishit Shah, Torsten Hoefler
-
Patent number: 11580388Abstract: Embodiments of the present disclosure include techniques for processing neural networks. Various forms of parallelism may be implemented using topology that combines sequences of processors. In one embodiment, the present disclosure includes a computer system comprising a plurality of processor groups, the processor groups each comprising a plurality of processors. A plurality of network switches are coupled to subsets of the plurality of processor groups. A subset of the processors in the processor groups may be configurable to form sequences, and the network switches are configurable to form at least one sequence across one or more of the plurality of processor groups to perform neural network computations. Various alternative configurations for creating Hamiltonian cycles are disclosed to support data parallelism, pipeline parallelism, layer parallelism, or combinations thereof.Type: GrantFiled: January 3, 2020Date of Patent: February 14, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Torsten Hoefler, Mattheus C. Heddes, Deepak Goel, Jonathan R Belk
-
Publication number: 20220291976Abstract: One example provides an integrated computing device, comprising one or more computing clusters, and one or more network controllers, each network controller comprising a local data notification queue to queue send message notifications originating from the computing clusters on the integrated computing device, a remote data notification queue to queue receive message notifications originating from network controllers on remote integrated computing devices, a local no-data notification queue to queue receive message notifications originating from computing clusters on the integrated computing device, and a connection scheduler configured to schedule sending of data from memory on the integrated computing device when a send message notification in the local data notification queue is matched with a receive message notification in the remote data notification queue, and to schedule sending of receive message notifications from the local no-data notification queue.Type: ApplicationFiled: March 11, 2021Publication date: September 15, 2022Applicant: Microsoft Technology Licensing, LLCInventors: Deepak GOEL, Mattheus C. HEDDES, Torsten HOEFLER, Xiaoling XU
-
Publication number: 20220244911Abstract: The present disclosure includes digital circuits that generate values of a power of two (2) raised to an input value. For example, a digital circuit may include combinational logic that receives first digital bits representing an input mantissa of an input value and second digital bits representing an input exponent of the input value. The combinational logic generates a plurality of output mantissas and plurality of output exponents corresponding to an approximate value of a power of two (2) raised to a power of the input value when the input value is positive and negative and when the input exponent is above and below a first value. Selection circuits are configured to receive output mantissas and output exponents. The selection circuits include selection control inputs coupled to the input exponent and an input sign bit of the input value to select one of the output mantissas and one output exponents.Type: ApplicationFiled: January 29, 2021Publication date: August 4, 2022Inventors: Torsten Hoefler, Mattheus C Heddes
-
Publication number: 20220138524Abstract: Embodiments of the present disclosure include systems and methods for training neural networks based on dual pipeline architectures. In some embodiments, a first set of compute elements are configured to implement a first set of layers of a first instance of a neural network. A second set of compute elements are configured to implement a second set of layers of the first instance of the neural network. The second set of compute elements are further configured to implement a first set of layers of a second instance of the neural network. The first set of compute elements are further configured to implement a second set of layers of the second instance of the neural network. The first set of layers of the first instance of the neural network and the first set of layers of the second instance of the neural network are each configured to receive training data.Type: ApplicationFiled: January 15, 2021Publication date: May 5, 2022Inventors: Mattheus HEDDES, Torsten HOEFLER, Kenneth Andrew COLWELL, Amar PHANISHAYEE
-
Patent number: 11076210Abstract: Embodiments of the present disclosure include techniques for processing neural networks. Various forms of parallelism may be implemented using topology that combines sequences of processors. In one embodiment, the present disclosure includes a computer system comprising one or more processor groups, the processor groups each comprising a plurality of processors. A plurality of network switches are coupled to subsets of the plurality of processor groups. In one embodiment, the switches may be optical network switches. Processors in the processor groups may be configurable to form sequences, and the network switches are configurable to form at least one sequence across one or more of the plurality of processor groups to perform neural network computations. Various alternative configurations for creating Hamiltonian cycles are disclosed to support data parallelism, pipeline parallelism, layer parallelism, or combinations thereof.Type: GrantFiled: May 26, 2020Date of Patent: July 27, 2021Assignee: Microsoft Technology Licensing, LLCInventors: Torsten Hoefler, Mattheus C. Heddes, Jonathan R. Belk
-
Publication number: 20210211787Abstract: Embodiments of the present disclosure include techniques for processing neural networks. Various forms of parallelism may be implemented using topology that combines sequences of processors. In one embodiment, the present disclosure includes a computer system comprising one or more processor groups, the processor groups each comprising a plurality of processors. A plurality of network switches are coupled to subsets of the plurality of processor groups. In one embodiment, the switches may be optical network switches. Processors in the processor groups may be configurable to form sequences, and the network switches are configurable to form at least one sequence across one or more of the plurality of processor groups to perform neural network computations. Various alternative configurations for creating Hamiltonian cycles are disclosed to support data parallelism, pipeline parallelism, layer parallelism, or combinations thereof.Type: ApplicationFiled: May 26, 2020Publication date: July 8, 2021Inventors: Torsten HOEFLER, Mattheus C. HEDDES, Jonathan R. BELK
-
Publication number: 20210209460Abstract: Embodiments of the present disclosure include techniques for processing neural networks. Various forms of parallelism may be implemented using topology that combines sequences of processors. In one embodiment, the present disclosure includes a computer system comprising a plurality of processor groups, the processor groups each comprising a plurality of processors. A plurality of network switches are coupled to subsets of the plurality of processor groups. A subset of the processors in the processor groups may be configurable to form sequences, and the network switches are configurable to form at least one sequence across one or more of the plurality of processor groups to perform neural network computations. Various alternative configurations for creating Hamiltonian cycles are disclosed to support data parallelism, pipeline parallelism, layer parallelism, or combinations thereof.Type: ApplicationFiled: January 3, 2020Publication date: July 8, 2021Inventors: Torsten HOEFLER, Mattheus C. HEDDES, Deepak GOEL, Jonathan R. BELK