Abstract: A method includes receiving a sequence of audio features characterizing an utterance and processing, using an encoder neural network, the sequence of audio features to generate a sequence of encodings. At each of a plurality of output steps, the method also includes determining a corresponding hard monotonic attention output to select an encoding from the sequence of encodings, identifying a proper subset of the sequence of encodings based on a position of the selected encoding in the sequence of encodings, and performing soft attention over the proper subset of the sequence of encodings to generate a context vector at the corresponding output step. The method also includes processing, using a decoder neural network, the context vector generated at the corresponding output step to predict a probability distribution over possible output labels at the corresponding output step.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for increasing the security of neural network by discretizing neural network inputs. One of the methods includes receiving a network input for a neural network; processing the network input using a discretization layer, wherein the discretization layer is configured to generate a discretized network input comprising a respective discretized vector for each of the numeric values in the network input; and processing the discretized network input using the plurality of additional neural network layers to generate a network output for the network input.

Abstract: A method includes receiving a sequence of audio features characterizing an utterance and processing, using an encoder neural network, the sequence of audio features to generate a sequence of encodings. At each of a plurality of output steps, the method also includes determining a corresponding hard monotonic attention output to select an encoding from the sequence of encodings, identifying a proper subset of the sequence of encodings based on a position of the selected encoding in the sequence of encodings, and performing soft attention over the proper subset of the sequence of encodings to generate a context vector at the corresponding output step. The method also includes processing, using a decoder neural network, the context vector generated at the corresponding output step to predict a probability distribution over possible output labels at the corresponding output step.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for enhanced attention mechanisms. In some implementations, data indicating an input sequence is received. The data is processed using an encoder neural network to generate a sequence of encodings. A series of attention outputs is determined using one or more attender modules. Determining each attention output can include (i) selecting an encoding from the sequence of encodings and (ii) determining attention over a proper subset of the sequence of encodings, where the proper subset of encodings is determined based on a position of the selected encoding in the sequence of encodings. The selections of encodings are also monotonic through the sequence of encodings. An output sequence is generated by processing the attention outputs using a decoder neural network. An output is provided that indicates a language sequence determined from the output sequence.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a target sequence including a respective output at each of multiple output time steps from respective encoded representations of inputs in an input sequence. The method includes, for each output time step, starting from the position, in the input order, of the encoded representation that was selected as a preceding context vector at a preceding output time step, traversing the encoded representations until an encoded representation is selected as a current context vector at the output time step. A decoder neural network processes the current context vector and a preceding output at the preceding output time step to generate a respective output score for each possible output and to update the hidden state of the decoder recurrent neural network. An output is selected for the output time step using the output scores.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for increasing the security of neural network by discretizing neural network inputs. One of the methods includes receiving a network input for a neural network; processing the network input using a discretization layer, wherein the discretization layer is configured to generate a discretized network input comprising a respective discretized vector for each of the numeric values in the network input; and processing the discretized network input using the plurality of additional neural network layers to generate a network output for the network input.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for enhanced attention mechanisms. In some implementations, data indicating an input sequence is received. The data is processed using an encoder neural network to generate a sequence of encodings. A series of attention outputs is determined using one or more attender modules. Determining each attention output can include (i) selecting an encoding from the sequence of encodings and (ii) determining attention over a proper subset of the sequence of encodings, where the proper subset of encodings is determined based on a position of the selected encoding in the sequence of encodings. The selections of encodings are also monotonic through the sequence of encodings. An output sequence is generated by processing the attention outputs using a decoder neural network. An output is provided that indicates a language sequence determined from the output sequence.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a target sequence including a respective output at each of multiple output time steps from respective encoded representations of inputs in an input sequence. The method includes, for each output time step, starting from the position, in the input order, of the encoded representation that was selected as a preceding context vector at a preceding output time step, traversing the encoded representations until an encoded representation is selected as a current context vector at the output time step. A decoder neural network processes the current context vector and a preceding output at the preceding output time step to generate a respective output score for each possible output and to update the hidden state of the decoder recurrent neural network. An output is selected for the output time step using the output scores.

Abstract: Connector systems may include a connector receptacle and connector plug or insert. The connector receptacle may include a tongue. A first plurality of contacts may be formed on a top surface of the tongue. A first ground pad may be located on a top surface of tongue, and a shield may be formed around the tongue. The connector insert may include a housing and a conductive shield around the housing behind a leading edge of the connector insert. A front edge of the shield may be folded into an opening at the leading edge. In other examples, the receptacle shield may include one or more fingers. These fingers may contact the connector insert shield to form a ground path. One or more of these fingers may engage openings in the insert shield to provide a retention force between the connector insert and receptacle.

Abstract: Connector systems may include a connector receptacle and connector plug or insert. The connector receptacle may include a tongue. A first plurality of contacts may be formed on a top surface of the tongue. A first ground pad may be located on a top surface of tongue, and a shield may be formed around the tongue. The connector insert may include a housing and a conductive shield around the housing behind a leading edge of the connector insert. A front edge of the shield may be folded into an opening at the leading edge. In other examples, the receptacle shield may include one or more fingers. These fingers may contact the connector insert shield to form a ground path. One or more of these fingers may engage openings in the insert shield to provide a retention force between the connector insert and receptacle.

Abstract: Low cost ground connections for standard connectors and connectors having reduced pin counts. One example may provide a connector system including a connector insert having a plurality of contacts along a top or bottom, or both top and bottom, of a connector insert, where first contacts in the plurality of contacts may be used to convey power, ground, or data and where second contacts in the plurality of contacts are used for ground. The second contacts may be arranged to have contacting portions that are positioned in the insert at different heights relative to the top or bottom of the connector insert and at different depths relative to a front opening of the connector insert.

Abstract: Connector systems may include a connector receptacle and connector plug or insert. The connector receptacle may include a tongue. A first plurality of contacts may be formed on a top surface of the tongue. A first ground pad may be located on a top surface of tongue, and a shield may be formed around the tongue. The connector insert may include a housing and a conductive shield around the housing behind a leading edge of the connector insert. A front edge of the shield may be folded into an opening at the leading edge. In other examples, the receptacle shield may include one or more fingers. These fingers may contact the connector insert shield to form a ground path. One or more of these fingers may engage openings in the insert shield to provide a retention force between the connector insert and receptacle.

Abstract: Low cost ground connections for standard connectors and connectors having reduced pin counts. One example may provide a connector system including a connector insert having a plurality of contacts along a top or bottom, or both top and bottom, of a connector insert, where first contacts in the plurality of contacts may be used to convey power, ground, or data and where second contacts in the plurality of contacts are used for ground. The second contacts may be arranged to have contacting portions that are positioned in the insert at different heights relative to the top or bottom of the connector insert and at different depths relative to a front opening of the connector insert.

Abstract: Connector systems may include a connector receptacle and connector plug or insert. The connector receptacle may include a tongue. A first plurality of contacts may be formed on a top surface of the tongue. A first ground pad may be located on a top surface of tongue, and a shield may be formed around the tongue. The connector insert may include a housing and a conductive shield around the housing behind a leading edge of the connector insert. A front edge of the shield may be folded into an opening at the leading edge. In other examples, the receptacle shield may include one or more fingers. These fingers may contact the connector insert shield to form a ground path. One or more of these fingers may engage openings in the insert shield to provide a retention force between the connector insert and receptacle.