Abstract: A system for performing a machine learning task on a network input is described. The system includes one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to implement (i) multiple sorting networks in which each sorting network is configured to sort vector blocks in a sequence of vector blocks to generate a sorted sequence of vector blocks; and (ii) a sorting attention neural network configured to perform the machine learning task on the input sequence by executing multiple sorting attention mechanisms using the sorting networks.

Abstract: A multiplexer and an antenna array module applying the multiplexer are provided, the array antenna module includes an array antenna and a beamforming module, the multiplexer includes a first end and at least two second ends extending towards the first end. One of the first end or the at least two second ends is connected to the array antenna, the other one of the first end or the at least two second ends is connected to the beamforming module, the first end and the at least two second ends are configured to conduct signals between the array antenna and the beamforming module.

Abstract: A multiplexer and an antenna array module applying the multiplexer are provided, the array antenna module includes a circuit board having a plurality of layers and an array antenna, the multiplexer includes a first end, at least two second ends, a connecting portion, and a plurality of conductive portions. The second ends and the first end are arranged on a same layer of the circuit board. The connecting portion is connected between the first end and the second ends, the connecting portion arranged on a different layer of the circuit board to the layer which the first end and the at least two second ends are located. The connecting portion is connected to the first end and the second ends through the conductive portions. One of the first end or the at least two second ends is connected to the array antenna for conducting signals of the array antenna.

Abstract: Provided are systems and methods for training and/or use of a machine learning model that can directly predict one or more resources that are responsive to a query as an output of the model. In particular, the present disclosure demonstrates that information retrieval can be accomplished with a single machine learning model (e.g., that has a neural network architecture such as, for example, a Transformer architecture) in which all information about the corpus is encoded in the parameters of the model. To this end, the present disclosure introduces the Differentiable Search Index (DSI), a new paradigm that learns a query-to-result (e.g., in text-to-text format) model that will map queries (e.g., text strings) directly to relevant resource identifiers (“docids”) (e.g.

Abstract: An example method for pretraining a machine-learned model is provided. The example method includes obtaining a plurality of different combinations of configuration parameters of a pretraining objective framework. The example method includes generating, using the pretraining objective framework, a plurality of corrupted training examples from one or more training examples, wherein the plurality of corrupted training examples are respectively generated according to the plurality of different combinations. The example method includes inputting the plurality of corrupted training examples into the machine-learned model, wherein the machine-learned model is configured to generate uncorrupted subportions corresponding to corrupted subportions of the corrupted training examples. The example method includes obtaining, from the machine-learned model, a plurality of outputs respectively generated by the machine-learned model based on the plurality of corrupted training examples.

Abstract: A printing device and the ribbon mounting mechanism thereof are disclosed. The ribbon mounting mechanism is used for mounting a first ribbon comprising a first core and a second ribbon comprising a second core having a diameter smaller than that of the first core. The ribbon mounting mechanism includes a first shaft supporting the first core of the first ribbon; a second shaft supporting the second core of the second ribbon, wherein the second shaft and the first shaft are concentrically disposed; a torque generator comprising a third shaft; a first gear set connecting the first shaft and the third shaft, and torque generated by the torque generator is transmitted to the first shaft through the first gear set; and a second gear set connecting the second shaft and the third shaft, and torque generated by the torque generator is transmitted to the second shaft through the second gear set.

Abstract: A substrate comprising at least one dielectric layer, and a plurality of interconnects located at least partially in the at least one dielectric layer, wherein the plurality of interconnects include a plurality of via interconnects, and wherein the plurality of via interconnects include a first via interconnect comprising a first via wall that is approximately vertical.

Abstract: Implantable depots for delivering therapeutic agents and associated systems and methods are provided. In some embodiments, an implantable depot for treating pain in a subject after a surgical procedure includes a therapeutic region having a first polymer and an analgesic agent, a first control region including a second polymer, and a second control region including a third polymer. The first and second control regions can cover first and second surfaces of the therapeutic region to inhibit release of the analgesic agent therefrom. The depot can include one or more holes extending through the first and second control regions and the therapeutic region to form one or more exposed portions. When implanted in the subject, the implantable depot can release the analgesic agent from a lateral surface of the therapeutic region between the first and second surfaces, and from the exposed portions of the therapeutic region.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for executing and training a multi-modal, multi-task self-attention neural network.

Abstract: A printing device and the ribbon mounting mechanism thereof are disclosed. The ribbon mounting mechanism is used for mounting a first ribbon comprising a first core and a second ribbon comprising a second core having a diameter smaller than that of the first core. The ribbon mounting mechanism includes a first shaft supporting the first core of the first ribbon; a second shaft supporting the second core of the second ribbon, wherein the second shaft and the first shaft are concentrically disposed; a torque generator comprising a third shaft; a first gear set connecting the first shaft and the third shaft, and torque generated by the torque generator is transmitted to the first shaft through the first gear set; and a second gear set connecting the second shaft and the third shaft, and torque generated by the torque generator is transmitted to the second shaft through the second gear set.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a machine learning task on an input sequence of characters that has a respective character at each of a plurality of character positions to generate a network output. One of the systems includes a neural network configured to perform the machine learning task, the neural network comprising a gradient-based sub-word tokenizer and an output neural network. The gradient-based sub-word tokenizer is configured to apply a learned, i.e., flexible, sub-word tokenization strategy to the input sequence of characters to generate a sequence of latent sub-word representations. The output neural network is configured to process the latent sub-word representation to generate the network output for the task.

Abstract: An example method for training a machine-learned sequence processing model includes obtaining a plurality of training examples for training the machine-learned sequence processing model. For each respective training example of the plurality of training examples, the example method includes: obtaining a respective query associated with the respective training example; inputting the respective query to the machine-learned sequence processing model; obtaining, from the machine-learned sequence processing model a response to the respective query and a trace of intermediate states from the respective query to the response; evaluating the response using a ground truth response associated with the respective training example; evaluating the trace using a ground truth trace associated with the respective training example; and updating one or more parameters of the machine-learned sequence processing model based on the evaluation of the response and based on the evaluation of the trace.

Abstract: An example method includes obtaining a pretrained machine-learned model that was initially pretrained using a pretraining dataset and further pretraining the model by generating, using a pretraining objective framework, a plurality of corrupted training examples from one or more training examples obtained from the pretraining dataset. A first set of one or more training examples can be corrupted according to a first set of configuration parameters of the pretraining objective framework. A second set can be corrupted according to a second set of configuration parameters of the pretraining objective framework. The example method includes inputting the plurality of corrupted training examples into model; obtaining from the model, a plurality of outputs respectively generated by model based on the plurality of corrupted training examples; and updating one or more parameters of model based on an evaluation of the plurality of outputs.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output sequences using auto-regressive decoder neural networks. In particular, during generation, adaptive early exiting is used to reduce the time required to generate the output sequence.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output sequences using auto-regressive decoder neural networks. In particular, during generation, adaptive early exiting is used to reduce the time required to generate the output sequence.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output sequences using auto-regressive decoder neural networks. In particular, during generation, adaptive early exiting is used to reduce the time required to generate the output sequence.

Abstract: An example method for pretraining a machine-learned model is provided. The example method includes obtaining a plurality of different combinations of configuration parameters of a pretraining objective framework. The example method includes generating, using the pretraining objective framework, a plurality of corrupted training examples from one or more training examples, wherein the plurality of corrupted training examples are respectively generated according to the plurality of different combinations. The example method includes inputting the plurality of corrupted training examples into the machine-learned model, wherein the machine-learned model is configured to generate uncorrupted subportions corresponding to corrupted subportions of the corrupted training examples. The example method includes obtaining, from the machine-learned model, a plurality of outputs respectively generated by the machine-learned model based on the plurality of corrupted training examples.

Abstract: Some embodiments relate to a memory device. The memory device includes a substrate comprising an inter-metal dielectric layer having a metal line, a dielectric layer over the substrate, a bottom electrode via through the dielectric layer and in contact with the metal line, a bottom electrode over the bottom electrode via, a magnetic tunneling junction (MTJ) element over the bottom electrode, and a top electrode over the MTJ element. A center portion of the bottom electrode directly above the bottom electrode via is thicker than an edge portion of the bottom electrode.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network having a plurality of network parameters. One of the methods includes obtaining an unlabeled training input from a set of unlabeled training data; processing the unlabeled training input to generate a first embedding; generating a corrupted version of the unlabeled training input, comprising determining a proper subset of the feature dimensions and, for each feature dimension that is in the proper subset of feature dimensions, applying a corruption to the respective feature in the feature dimension using one or more feature values sampled from a marginal distribution of the feature dimension as specified in the set of unlabeled training data; processing the corrupted version of the unlabeled training input to generate a second embedding; and determining an update to the current values of the plurality of network parameters.

Abstract: A test tube preparation device uses a labeling device to include a linking module to link a positioning unit, and the positioning unit can be used to place a tube body and finely adjust a position of the tube body relative to the label generating module of the surface treating device to complete the label generation, label conveyance, tube labeling and tube delivery, and provide a preparation device for effectively integrating the label and applying to the tube body before and after the labeling of the tube body, so as to improve the quality of the generation of the tube body and the label. Further, the present invention further provides a test tube preparation method.