Abstract: An example method includes receiving, by a computational assistant executing at one or more processors, a representation of an utterance spoken at a computing device; identifying, based on the utterance, a task to be performed by the computational assistant; responsive to determining, by the computational assistant, that complete performance of the task will take more than a threshold amount of time, outputting, for playback by one or more speakers operably connected to the computing device, synthesized voice data that informs a user of the computing device that complete performance of the task will not be immediate; and performing, by the computational assistant, the task.

Abstract: A number of circuits for use in an output block coupled to a non-volatile memory array in a neural network are disclosed. The embodiments include a circuit for converting an output current from a neuron in a neural network into an output voltage, a circuit for converting a voltage received on an input node into an output current, a circuit for summing current received from a plurality of neurons in a neural network, and a circuit for summing current received from a plurality of neurons in a neural network.

Abstract: Methods, and systems, including computer programs encoded on computer storage media for generating data items. A method includes reading a glimpse from a data item using a decoder hidden state vector of a decoder for a preceding time step, providing, as input to a encoder, the glimpse and decoder hidden state vector for the preceding time step for processing, receiving, as output from the encoder, a generated encoder hidden state vector for the time step, generating a decoder input from the generated encoder hidden state vector, providing the decoder input to the decoder for processing, receiving, as output from the decoder, a generated a decoder hidden state vector for the time step, generating a neural network output update from the decoder hidden state vector for the time step, and combining the neural network output update with a current neural network output to generate an updated neural network output.

Abstract: Systems and techniques are described for improving the evaluation of unstructured transaction data to, for example, recognize reoccurring data patterns or patterns of interest, predict future outcomes using historical indicators, identify attributes of interest, or evaluate likelihoods of certain conditions occurring. For example, a system can transform unstructured public record data obtained from multiple independent public data sources according to a hierarchical data model. The hierarchical data model can specify nodes within different data layers of a data hierarchy and classification labels corresponding to each of the nodes. In this way, the system can utilize data transformation techniques to permit the processing of information within unstructured transaction data that would have otherwise been impossible to perform without initially structuring the data according to the hierarchical data model.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for adjusting neural network resource usage. One of the methods includes receiving a network input for processing by a task neural network, the task neural network comprising a plurality of neural network layers; receiving a usage input specifying a respective weight for each of one or more usage factors, wherein each usage factor impacts how many computational resources are used by the task neural network during the processing of the network input; and processing the network input using the task neural network in accordance with the usage input to generate a network output for the network input, comprising: selecting, based at least on the usage input, a proper subset of the plurality of neural network layers to be active while processing the network input, and processing the network input using only the selected neural network layers.

Abstract: Quantization-aware neural architecture search (“QNAS”) can be utilized to learn optimal hyperparameters for configuring an artificial neural network (“ANN”) that quantizes activation values and/or weights. The hyperparameters can include model topology parameters, quantization parameters, and hardware architecture parameters. Model topology parameters specify the structure and connectivity of an ANN. Quantization parameters can define a quantization configuration for an ANN such as, for example, a bit width for a mantissa for storing activation values or weights generated by the layers of an ANN. The activation values and weights can be represented using a quantized-precision floating-point format, such as a block floating-point format (“BFP”) having a mantissa that has fewer bits than a mantissa in a normal-precision floating-point representation and a shared exponent.

Abstract: Techniques are disclosed for identifying multimodal subevents within an event having spatially-related and temporally-related features. In one example, a system receives a Spatio-Temporal Graph (STG) comprising (1) a plurality of nodes, each node having a feature descriptor that describes a feature present in the event, (2) a plurality of spatial edges, each spatial edge describing a spatial relationship between two of the plurality of nodes, and (3) a plurality of temporal edges, each temporal edge describing a temporal relationship between two of the plurality of nodes. Furthermore, the STG comprises at least one of: (1) variable-length descriptors for the feature descriptors or (2) temporal edges that span multiple time steps for the event. A machine learning system processes the STG to identify the multimodal subevents for the event. In some examples, the machine learning system comprises stacked Spatio-Temporal Graph Convolutional Networks (STGCNs), each comprising a plurality of STGCN layers.

Abstract: A system includes a neural network that includes a Mixture of Experts (MoE) subnetwork between a first neural network layer and a second neural network layer. The MoE subnetwork includes multiple expert neural networks. Each expert neural network is configured to process a first layer output generated by the first neural network layer to generate a respective expert output. The MoE subnetwork further includes a gating subsystem that selects, based on the first layer output, one or more of the expert neural networks and determine a respective weight for each selected expert neural network, provides the first layer output as input to each of the selected expert neural networks, combines the expert outputs generated by the selected expert neural networks in accordance with the weights for the selected expert neural networks to generate an MoE output, and provides the MoE output as input to the second neural network layer.

Abstract: Systems and methods for extracting data values from electronic documents using neural networks. The method includes receiving an electronic document having data values and associated field identifiers, determining pixel coordinates corresponding to the field identifiers using a first neural network, and extracting the field identifiers located at the pixel coordinates using a second neural network. The method also includes, for each of the field identifiers, calculating pixel coordinates on the electronic document corresponding to a data value associated with the field identifier using a third neural network and extracting the data value located at the calculated pixel coordinates using the second neural network. The method further includes, for each of the data values, generating a record in a data structure, the record including the extracted value and the extracted field identifier. The method also includes storing the data structure including the records in a database.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting likelihoods of conditions being satisfied using recurrent neural networks. One of the systems is configured to process a temporal sequence comprising a respective input at each of a plurality of time steps and comprises: one or more recurrent neural network layers; one or more logistic regression nodes, wherein each of the logistic regression nodes corresponds to a respective condition from a predetermined set of conditions, and wherein each of the logistic regression nodes is configured to, for each of the plurality of time steps: receive the network internal state for the time step; and process the network internal state for the time step in accordance with current values of a set of parameters of the logistic regression node to generate a future condition score for the corresponding condition for the time step.

Abstract: An apparatus is described. The apparatus includes a long short term memory (LSTM) circuit having a multiply accumulate circuit (MAC). The MAC circuit has circuitry to rely on a stored product term rather than explicitly perform a multiplication operation to determine the product term if an accumulation of differences between consecutive, preceding input values has not reached a threshold.

Abstract: A machine learning system, including at least one temporal filter. An input variable, encompassing a chronological sequence of images, is processed with the aid of the machine learning system, using the filter. The machine learning system is configured to use the filter on a sequence of pixels, which are all situated at identical coordinates of the images, or at identical coordinates of intermediate results. Filter coefficients of the filter are quantized. A method, a computer program, and a device for creating the machine learning system are also described.

Abstract: Some embodiments provide a three-dimensional (3D) circuit structure that has two or more vertically stacked bonded layers with a machine-trained network on at least one bonded layer. As described above, each bonded layer can be an IC die or an IC wafer in some embodiments with different embodiments encompassing different combinations of wafers and dies for the different bonded layers. The machine-trained network in some embodiments includes several stages of machine-trained processing nodes with routing fabric that supplies the outputs of earlier stage nodes to drive the inputs of later stage nodes. In some embodiments, the machine-trained network is a neural network and the processing nodes are neurons of the neural network. In some embodiments, one or more parameters associated with each processing node (e.g., each neuron) is defined through machine-trained processes that define the values of these parameters in order to allow the machine-trained network (e.g.

Abstract: The present disclosure relates to a neuron for an artificial neural network. The neuron comprises a dot product engine operative to: receive a set of weights; receive a set of data inputs based on a set of input data signals; and calculate the dot product of the set of data inputs and the set of weights to generate a dot product engine output. The neuron further comprises an activation function module arranged to apply an activation function to a signal indicative of the dot product engine output to generate a neuron output; and gain control circuitry. The gain control circuitry is operative to control: an input gain applied to the input data signals to generate the set of data inputs; and an output gain applied to the dot product engine output or by the activation function module. The output gain is selected to compensate for the applied input gain.

Abstract: Many of the features of neural networks for machine learning can naturally be mapped into the quantum optical domain by introducing the quantum optical neural network (QONN). A QONN can be performed to perform a range of quantum information processing tasks, including newly developed protocols for quantum optical state compression, reinforcement learning, black-box quantum simulation and one way quantum repeaters. A QONN can generalize from only a small set of training data onto previously unseen inputs. Simulations indicate that QONNs are a powerful design tool for quantum optical systems and, leveraging advances in integrated quantum photonics, a promising architecture for next generation quantum processors.

Abstract: The present invention concerns a method for learning the parameters of a convolutional neural network, CNN, for data classification, the method comprising the implementation of steps by data processing means (11a, 11 b, 11c) of at least one server (1a, 1b, 1c), of: (a1) Learning, from a base of already-classified confidential learning data, the parameters of a first CNN; (a2) Learning, from a base of already-classified public learning data, the parameters of a last fully-connected layer (FC) of a second CNN corresponding to the first CNN to which said fully-connected layer (FC) has been added. The present invention also concerns a method for classifying an input datum.

Abstract: Methods and systems are disclosed for boosting deep neural networks for deep learning. In one example, in a deep neural network including a first shallow network and a second shallow network, a first training sample is processed by the first shallow network using equal weights. A loss for the first shallow network is determined based on the processed training sample using equal weights. Weights for the second shallow network are adjusted based on the determined loss for the first shallow network. A second training sample is processed by the second shallow network using the adjusted weights. In another example, in a deep neural network including a first weak network and a second weak network, a first subset of training samples is processed by the first weak network using initialized weights. A classification error for the first weak network on the first subset of training samples is determined.

Abstract: A method of generating and using a metadata files for integration flows may include analyzing definition files of integration flows to generate metadata files that include summary information for each of the integration flows. The method also includes extracting source-target relationships from the summary information for each of the integration flows and training a model using the plurality of source-target relationships. The method also includes receiving a source element from a current integration flow; providing the source element and characteristics of the current integration flow to the model; and receiving, from the model, recommended target elements to be connected to the source element in the current integration flow.

Abstract: A data processing apparatus configured to execute hierarchical calculation processing corresponding to a neural network on input data includes a storage unit configured to store a plurality of sets of control data each for use for one of a plurality of processing units into which the hierarchical calculation processing corresponding to the neural network is divided, a transfer unit configured to sequentially transfer the plurality of sets of control data from the storage unit, and a calculation processing unit configured to perform calculation processing of the processing unit corresponding to one set of control data transferred by the transfer unit using the one set of control data.

Abstract: The performance of a neural network (NN) and/or deep neural network (DNN) can limited by the number of operations being performed as well as management of data among the various memory components of the NN/DNN. A sparsity-inducing regularization optimization process is performed on a machine learning model to generate a compressed machine learning model. A machine learning model is trained using a first set of training data. A sparsity-inducing regularization optimization process is executed on the machine learning model. Based on the sparsity-inducing regularization optimization process, a compressed machine learning model is received. The compressed machine learning model is executed to generate one or more outputs.

Abstract: Systems and methods are disclosed for automatically scoring a constructed response using a neural network. In embodiments, a constructed response received by a processing system may be processed to divide the constructed response into multiple series of word tokens, wherein each word token includes a sequence of characters. The constructed response may be further processed to correct one or more spelling errors. The word tokens may be encoded to generate representation vectors for the constructed response. A set of nonlinear operations may be applied to the plurality of representation vectors in a neural network to generate a single vector output. A set of predetermined network weights may be applied to the vector output of the neural network to generate a scalar output for scoring the constructed response.

Abstract: A learning-based model is trained using a plurality of attributes of media. Depth estimation is performed using the learning-based model. The depth estimation supports performing a computer vision task on the media. Attributes used in the depth estimation include scene understanding, depth correctness, and processing of sharp edges and gaps. The media may be processed to perform media restoration or the media quality enhancement. A computer vision task may include semantic segmentation.

Abstract: The present disclosure provides systems and methods for synthetic data generation. A recurrent neural network can be trained for synthetic data generation by obtaining a sequence of elements and determining, using a classifier, that the sequence corresponds to a token. In response to the determination, a recurrent neural network configured to use a first vocabulary including the elements can be modified to use a second vocabulary, the second vocabulary including the token and the first vocabulary. The modified recurrent neural network can be trained using the token and the sequence of elements. The trained recurrent neural network can be used to generate synthetic data. A classifier can detect sequences of elements in the synthetic data corresponding to tokens. The tokens can replace the sequences of elements in the generated synthetic data and can be provided to the trained recurrent neural network to continue synthetic data generation.

Abstract: In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.

Abstract: A computer-implemented method according to one embodiment includes applying a predetermined augmentation to the sample set of training data to create an augmented sample set, training a model with the augmented sample set, determining a performance of the trained model, and assigning a weight to the predetermined augmentation for the training data set based on the determined performance. A determination is made as to whether to apply the predetermined augmentation to a larger training data set before the training data set is applied to the model, based on the weight assigned to the predetermined augmentation.

Abstract: A neural network deep learning data control apparatus includes: a memory; an encoding circuit configured to receive a data sequence, generate a compressed data sequence in which consecutive invalid bits in a bit string of the data sequence are compressed into a single bit of the compressed data sequence, generate a validity determination sequence indicating a valid bit and an invalid bit in a bit string of the compressed data sequence, and write the compressed data sequence and the validity determination sequence to the memory; and a decoding circuit configured to read the compressed data sequence and the validity determination sequence from the memory, and determine a bit in the bit string of the compressed data sequence set for transmission to a neural network circuit, based on the validity determination sequence, such that the neural network circuit omits an operation with respect to non-consecutive invalid bits.

Abstract: The specification describes methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a larger neural network from a smaller neural network. One of the described methods includes obtaining data specifying an original neural network and generating a larger neural network from the original neural network. The larger neural network has a larger neural network structure than the original neural network structure. The values of the parameters of the original neural network units and the additional neural network units are initialized so that the larger neural network generates the same outputs from the same inputs as the original neural network, and the larger neural network is trained to determine trained values of the parameters of the original neural network units and the additional neural network units from the initialized values.

Abstract: In implementations of resource-aware training for neural network, one or more computing devices of a system implement an architecture optimization module for monitoring parameter utilization while training a neural network. Dead neurons of the neural network are identified as having activation scales less than a threshold. Neurons with activation scales greater than or equal to the threshold are identified as survived neurons. The dead neurons are converted to reborn neurons by adding the dead neurons to layers of the neural network having the survived neurons. The reborn neurons are prevented from connecting to the survived neurons for training the reborn neurons.

Abstract: Computer systems and methods modify a base deep neural network (DNN). The method comprises replacing the target node of the base DNN with a compound node to thereby create a modified base DNN. The compound node comprises at least first and second nodes. The first node is trained to detect target node patterns in inputs to the first node and the second node is trained to detect an absence of the target node patterns in inputs to the second node, and the first and second nodes are trained to be non-complementary.

Abstract: Systems and methods according to the present disclosure can employ a computer-implemented method for inference using a machine-learned model. The method can be implemented by a computing system having one or more computing devices. The method can include obtaining data descriptive of a neural network including one or more network units and one or more gating paths, wherein each of the gating path(s) includes one or more gating units. The method can include obtaining data descriptive of one or more input features. The method can include determining one or more network unit outputs from the network unit(s) based at least in part on the input feature(s). The method can include determining one or more gating values from the gating path(s). The method can include determining one or more gated network unit outputs based at least in part on a combination of the network unit output(s) and the gating value(s).

Abstract: Methods, apparatus, systems and articles of manufacture to defend against adversarial machine learning are disclosed. An example apparatus includes memory; computer readable instructions; and processor circuitry to execute the computer readable instructions to: generate a first output indicating a feature that contributed to the generation of a classification by a machine learning model; compare the first output with a second output generated by a server that trained the machine learning model; and flag the machine learning model as corresponding to at least one of model drift or an adversarial attack when first output differs from the second output by more than a threshold.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using multi-task neural networks. One of the methods includes receiving a first network input and data identifying a first machine learning task to be performed on the first network input; selecting a path through the plurality of layers in a super neural network that is specific to the first machine learning task, the path specifying, for each of the layers, a proper subset of the modular neural networks in the layer that are designated as active when performing the first machine learning task; and causing the super neural network to process the first network input using (i) for each layer, the modular neural networks in the layer that are designated as active by the selected path and (ii) the set of one or more output layers corresponding to the identified first machine learning task.

Abstract: A specification of a property required to be upheld by a computerized machine learning system is obtained. A training data set corresponding to the property and inputs and outputs of the system is built. The system is trained on the training data set. Activity of the system is monitored before, during, and after the training. Based on the monitoring, performance of the system is evaluated to determine whether the system, once trained on the training data set, upholds the property.

Abstract: This disclosure relates to artificial intelligence (AI) and machine learning networks for predicting or determining demand metrics across multiple channels. An analytics platform can receive channel events from multiple channels corresponding to geographic areas, and channel features related to demand conditions in the channels can be extracted from the channel events. During a training phase, the channel features can be accumulated into one or more training datasets for training one or more demand prediction models. The one or more demand prediction models can be trained to predict or determine demand metrics for each of the channels. The demand metrics can indicate or predict demand conditions based on the current conditions in the channels and/or based on future, predicted conditions in the channels. Other embodiments are disclosed herein as well.

Abstract: In one embodiment, a method of simulating an operation of an artificial neural network on a binary neural network processor includes receiving a binary input vector for a layer including a probabilistic binary weight matrix and performing vector-matrix multiplication of the input vector with the probabilistic binary weight matrix, wherein the multiplication results are modified by simulated binary-neural-processing hardware noise, to generate a binary output vector, where the simulation is performed in the forward pass of a training algorithm for a neural network model for the binary-neural-processing hardware.

Abstract: Techniques are described for generating and applying mini-machine learning variants of machine learning algorithms to save computational resources in tuning and selection of machine learning algorithms. In an embodiment, at least one of the hyper-parameter values for a reference variant is modified to a new hyper-parameter value thereby generating a new variant of machine learning algorithm from the reference variant of machine learning algorithm. A performance score is determined for the new variant of machine learning algorithm using a training dataset, the performance score representing the accuracy of the new machine learning model for the training dataset. By performing training of the new variant of machine learning algorithm with the training data set, a cost metric of the new variant of machine learning algorithm is measured by measuring usage the used computing resources for the training.

Abstract: A unit structure of non-volatile memory is provided. The unit structure includes a substrate, an n-type ferroelectric field effect transistor (FeFET) and a p-type FeFET disposed on the substrate, first circuitry by which sources of the n-type FeFET and the p-type FeFET are electrically coupled in parallel downstream from a common terminal and second circuitry by which top electrodes of the n-type FeFET and the p-type FeFET are electrically coupled in parallel upstream of a common terminal.

Abstract: A system and method for generating a predictive model include accessing data from a dynamic dataset and a static dataset, generating a unique individual profile for each of a plurality of subjects within a population, assigning a class attribute to each subject, and based on the unique individual profile and the class attribute for each subject, developing a classification model based on the unique individual profile and the class attribute for each of the plurality of subject, and generating an individual score for each of the plurality of subjects using the classification model. The unique individual profile is generated from the data in the dynamic and the static datasets.

Abstract: Methods, computer program products, and systems are presented. The methods include, for instance: generating a semantic network cell for a component of a semantic expression in a semantic network. The semantic network includes multiple semantic network cells. Each semantic network cell has attributes of a weight, an access count, and a latest time of access. A machine learning process reinforces the semantic network cell by access and deteriorates the semantic network cell over time based on semantic network cell weight rules, while the semantic network is servicing searches.

Abstract: Methods, computer program products, and systems are presented. The methods include, for instance: obtaining prediction data that are indicative of media use behaviors of a user over a period of time. Prediction on a pattern of media use behaviors of the user including media content, hours spent on the media content, and the period of time is made. Base on the predicted media use pattern corresponding to a current stage in the period of time, a media use control rule for the user on a controlled device is generated.

Abstract: A computer-implemented method is provided. The computer-implemented includes a data-driven model and a robust closure model stored in a memory by using a processor for controlling a system. The computer-implemented method includes steps of acquiring sensor signals from at least one sensor of the system via an interface, computing a state of the system based on the sensor signals, determining a gain of the robust closure model based on the state of the system, reproducing a state of the system based on the determined gain, estimating a physics-based model of the system by combining the data-driven model and the robust closure model, and generating control commands by mapping the state of the system using the estimated physics-based model.

Abstract: A transposable identity enchainment system for diffuse identity management processing entities for each of users, data, and processes equivalently and having a recombinant access mediation system that mediates association among entities, an associational process management system that creates entity-defining indices, and a multi-dimensional enchainment system that enchains aspects of entity identities via mediated association certificates including at least one root certificate for at least one of the entities.

Abstract: The disclosed technology includes systems and methods for automatically generating a dynamic system context diagram based on machine-readable code. A method can include receiving, at a rules engine, machine-readable code describing interactions among a plurality of applications in software architecture, evaluating, with the rules engine in communication with a system of record (SoR), compliance of the interactions among the plurality of applications according to the SoR, identifying, with the rules engine, and based on compliance evaluation, one or more dependencies among the plurality of applications, generating, with an output engine, a system context diagram image comprising a graphical representation of the plurality of applications with associated interactions and dependencies, and outputting, for display, the system context diagram image.

Abstract: Various embodiments are generally directed to an apparatus, system, and other techniques for dynamic and intelligent deployment of a neural network or any inference model on a hardware executor or a combination of hardware executors. Computational costs for one or more operations involved in executing the neural network or inference model may be determined. Based on the computational costs, an optimal distribution of the computational workload involved in running the one or more operations among multiple hardware executors may be determined.

Abstract: Various embodiments described herein relate to a machine-learning based electronic media analysis software system. The system is configured to detect anomalous and predictive patterns associated with an event. The system is configured to use feature extraction techniques and semi-supervised machine-learning to detect the patterns associated with the event in the electronic media messages, which may indicate a synthetic driven behavior and conversation corresponding to the event.

Abstract: According to one embodiment, An apparatus for preprocessing a security log includes a field divider configured to divide a character string of a security log into a plurality of fields on the basis of a structure of the security log, an ASCII code converter configured to convert a character string included in each of the plurality of divided fields into ASCII codes, and a vector data generator configured to generate vector data for each of the plurality of divided fields using the converted ASCII codes.

Abstract: Method and system for modeling of complex systems using a two-sorted reasoning system. Information is received by Distributed Feature Extraction Processors. A first level of reasoning is performed on the information by Distributed Regular Reasoning Processors. A second reasoning process is performed on the information by Distributed Situation Reasoning Processors, which use a Functional Fabric configured to analyze the information received and use functions to modify previous inferences. Client applications allow for viewing and manipulating both reasoning systems and their associated information.

Abstract: The present invention provides a computerized method and system for detecting modeling content within a model file without rendering the model file, the method and system including loading the model file into a detection engine, the model file including software code therein. In the method and system, the detection engine scans the model file, detecting descriptor terms within software code. The method and system includes generating a description list for the model file based on the plurality of descriptor terms and executing a conversion engine to review the description list relative to a relational database, the conversion engine electronically generating file terms describing the modeling content within the model file based on input from the relational database. Whereby, the method and system generates a content list for the model file based on the file terms, the content list thereby associated with the model file.

Abstract: Various systems and methods for modeling a manufacturing assembly line are disclosed herein. Some embodiments relate to operating a processor to receive cell data, extract feature data from the cell data, determine a plurality of faults, determine a priority level for each fault by applying the extracted feature data to a predictive model, determine at least one high priority fault, and generate at least one operator alert based on the at least one high priority fault.

Abstract: A computer-implemented method, system and computer program product for analyzing test result failures using artificial intelligence models. A first machine learning model is trained to differentiate between a bug failure and a test failure within the test failures based on the failure attributes and historical failures. The failure type for each failed test in test failure groups is then determined using the first machine learning model. The failed tests in the test failure groups are then clustered into a set of clusters according to the failure attributes and the determined failure type for each failed test. A root cause failure for each cluster is identified based on the set of clusters and the failure attributes. The root cause of an unclassified failure is predicted using a second machine learning model trained to predict a root cause of the unclassified failure based on identifying the root cause failure for each cluster.