Abstract: This disclosure describes systems, methods, and computer-readable media for initializing and training differentiable logic gate networks for machine-learning inference. In various embodiments, parameters of learnable logic gate operators are initialized using a residual initialization that biases the initial probability distribution of discrete gate choices toward a feedforward operation, such as a wire or an inverter, thereby preserving information flow and mitigating vanishing gradients in deep networks. During training, the gate parameters are updated so that gates can remain as feedforward connections or transition to other logic operations as needed. The residual initialization can be applied to convolutional logic gate-tree architectures and pooling operations, and trained networks can be discretized and synthesized into efficient logic circuits in hardware or software.

Abstract: Systems and methods are described for training and using logic gate tree networks and convolutional logic gate tree networks. A computing system may apply a convolution operation to an input tensor using one or more kernels, each kernel comprising a tree of logic gate nodes. For each kernel placement, leaves of the tree select input activations from a receptive field, and internal nodes generate a kernel output by applying logic operations. During training, each node may be parameterized with differentiable parameters that define a probability distribution over candidate logic operators. The parameters may be shared across kernel placements to provide equivariance. Gradient-based optimization may be used to update the differentiable parameters during training. A fixed tree kernel may be defined by selecting a logic operator for each node, which can be executed by a processor or synthesized for use in programmable logic and/or in an application-specific integrated circuit.

Abstract: Systems and methods are described for training a logic gate network using student-teacher learning. In various embodiments, a system obtains a teacher model that produces a target output distribution for an input and records a training dataset of inputs and teacher outputs. The system instantiates a differentiable logic-gate network with nodes parameterized over candidate logic gate operators. The system trains the differentiable logic gate network by minimizing the divergence between its outputs and the teacher outputs. After training, a logic gate operator may be selected for each node. The logic gate network may be implemented as logic circuitry in a programmable device or as a fixed-in-silicon application-specific integrated circuit.

Abstract: Systems and methods are described for training and using logic gate tree networks and convolutional logic gate tree networks. A computing system may apply a convolution operation to an input tensor using one or more kernels, each kernel comprising a tree of logic gate nodes. For each kernel placement, leaves of the tree select input activations from a receptive field, and internal nodes generate a kernel output by applying logic operations. During training, each node may be parameterized with differentiable parameters that define a probability distribution over candidate logic operators. The parameters may be shared across kernel placements to provide equivariance. Gradient-based optimization may be used to update the differentiable parameters during training. A fixed tree kernel may be defined by selecting a logic operator for each node, which can be executed by a processor or synthesized for use in programmable logic and/or in an application-specific integrated circuit.

Abstract: Systems and methods are disclosed for constructing and implementing feature extractors for inference on structured inputs. A computing system determines feature directions from a dataset, quantizes them into a restricted set of value representations, and iteratively updates the dataset. The iterative process removes contributions corresponding to the quantized feature directions while retaining the residual quantization error, thereby generating additional quantized feature directions. In some embodiments, shared arithmetic structures are constructed across multiple feature extractors by identifying channel tuples having common or proportional coefficients and creating merged channels. The resulting feature extractors may be synthesized into hardware representations, including FPGA configurations and ASIC implementations, and may generate Boolean, multi-bit, or integer-valued feature outputs.

Abstract: Systems and methods are disclosed for implementing and training differentiable logic-gate neural networks. A network includes hyper-logic gate nodes configured as differentiable lookup tables that receive selector signals and LUT entry signals and generate outputs via differentiable selection, enabling LUT-style multiplexing during discretization while remaining trainable by gradient-based optimization. LUT entry signals may be learned parameters, outputs of other nodes, external inputs, or internal memory-state values, enabling aggregation, switching, skip connections, and memory read/write operations. Training may employ constrained or reduced parameterizations that transform unconstrained trainable parameters via elementwise bounding functions and coefficient-mapping matrices to produce coefficient vectors for k-input gate functions, reducing storage and computation relative to enumerating discrete gates.

Abstract: Systems, methods, and computer-readable media are described for training and implementing logic gate networks for inference tasks. An untrained node network includes a first set of nodes parameterized by differentiable parameters, each associated with a predefined set of potential logic gate operators, and a second set of nodes corresponding to predefined logic gate operators. During training, node outputs of the first set are computed using a first relaxation, and node outputs of the second set are computed using a second relaxation different from the first relaxation. Updated differentiable parameters are applied over multiple training iterations, and a fixed logic-gate network is generated by selecting logic gate operators for at least some nodes in the first set while retaining the predefined logic gate operators for the second set. In some embodiments, the second relaxation imposes structural inductive bias. ASICs and FPGAs implementing fixed logic gate networks are also described.

Abstract: Systems, methods, and computer-readable media are described for training and implementing logic gate networks for inference tasks. An untrained node network includes a first set of nodes parameterized by differentiable parameters, each associated with a predefined set of potential logic gate operators, and a second set of nodes corresponding to predefined logic gate operators. During training, node outputs of the first set are computed using a first relaxation, and node outputs of the second set are computed using a second relaxation different from the first relaxation. Updated differentiable parameters are applied over multiple training iterations, and a fixed logic-gate network is generated by selecting logic gate operators for at least some nodes in the first set while retaining the predefined logic gate operators for the second set. In some embodiments, the second relaxation imposes structural inductive bias. ASICs and FPGAs implementing fixed logic gate networks are also described.

Abstract: The disclosure describes integrated circuits and training apparatuses for efficiently training learnable logic networks. One or more hardware-implemented learnable logic engines execute forward and backward propagation through differentiable relaxations of Boolean logic gates arranged in configurable clusters. Each engine processes multiple inputs and outputs with shared parameter sets, supports variable numbers of gate inputs, and may reuse locally stored parameters across batched samples to reduce memory bandwidth. Engines and associated cores employ mixed-precision arithmetic, including low-precision activations and higher-precision gradients with on-chip gradient accumulation. Configurable interconnects route activations between engines, and topology bits select among logic operations and wiring options. Systems including a host processor orchestrate the use of the learnable logic engines to design fixed logic gate networks for efficient inference.