Abstract: A set of nodes organized in a logical tree structure is obtained, where the set of nodes represent objects in a user interface. A first set of rankings is generated for the set of nodes, where the first set of rankings indicate likelihoods of nodes of the set of nodes corresponding to a first classification. A first node from the set of nodes that corresponds to the first classification is identified based at least in part on the first set of rankings. A second set of rankings that indicate likelihoods of descendent nodes of the first node corresponding to a second classification different from the first classification is determined. A second node from the descendent nodes that corresponds to the second classification is identified based at least in part on the second set of rankings. Data from an object in the user interface that corresponds to the second node is obtained.

Abstract: A plurality of HyperText Markup Language (HTML) strings corresponding to a dataset of document object model (DOM) nodes is tokenized according to a tokenization scheme to produce a dictionary of tokens that occur in the dataset. A condensed dictionary of tokens in produced by removing low-value tokens from the vocabulary of tokens. An information matrix is computed based on the condensed dictionary of tokens, the information matrix being a set of values, a value of the set of values corresponding to a frequency of co-occurrence of a pair of tokens. A library of word vectors is derived from the information matrix. A feature vector of an HTML element is generated based at least in part on the library of word vectors, and a classification for the HTML element is obtained from a machine learning model as a result of inputting the feature vector into the machine learning model.

Abstract: A first set of objects is obtained, where an object of the first set of objects is assigned a classification. A first dataset is generated based at least in part on the first set of objects, where the first dataset includes a value corresponding to at least one characteristic of the object and a label corresponding to the classification. A machine learning model is trained to classify objects using the first dataset as training input. A set of predictions that includes incorrect predictions for a second set of objects is generated using the machine learning model. A second dataset that includes negative-examples that correspond to the incorrect predictions is generated. The machine learning model is retrained using the second dataset as training input.

Abstract: A dataset of classification orderings is created based on previously observed interface elements in interfaces of third-party interface providers. A request to evaluate a sequence of predicted classifications is received. The dataset is queried to determine a value derived from a frequency of the sequence of predicted classifications occurring in the dataset. A client device is caused, by responding to the request with the value, to autocomplete input to a plurality of elements corresponding to the sequence of predicted classifications if the value reaches a value relative to a threshold cause.

Abstract: A sequence of interface elements in an interface is determined, where the sequence includes a first element that immediately precedes a second element in the sequence. A first set of potential classifications for the first element is obtained. A set of local confidence scores for a second set of potential classifications of the second element is obtained. A set of sequence confidence scores is obtained by obtaining, for each second potential classification of the second set of potential classifications, a set of scores indicating probability of the second potential classification being immediately preceded in sequence by each first potential classification of the first set of potential classifications. A classification assignment for the second element is determined based on the set of local confidence scores of the first element and the set of sequence confidence scores. An operation is performed with the second element in accordance with the classification assignment.

Abstract: Source code of a form element of a web form and a predetermined data classification of the form element is obtained. A vector is generated based at least in part on a transformation of a set of keywords derived from the source code. A machine learning model is trained to predict data categories of form elements by providing, to the machine learning model, the predetermined data classification and the vector.

Abstract: A baseline request produced from an annotated node of a document object model (DOM) tree and a label assigned to the annotated node are obtained. The label is assigned to a set of neighboring nodes of the DOM that perform a same function by recursively causing the system to, for each neighboring node to the annotated node in the DOM tree an additional request produced in response to performance of simulated human interaction with the neighboring node is identified, if the additional request matches the baseline request, the label is assigned to the neighboring node, and the neighboring node is selected to be the annotated node.

Abstract: Source information for a set of interfaces from a service provider is collected. A generative adversarial network (GAN) is trained using the source information and the set of interfaces. The source information is provided to a generative network of the GAN. The generative network is caused to generate a simulated interface. A discriminative network of the GAN is caused, by providing the simulated interface to the discriminative network, to output an estimate as to the authenticity of the simulated interface. The generative network is trained, based on the estimate, to produce a trained generative network. The trained generative network is caused to generate a plurality of simulated interfaces. A machine learning model is trained, using the plurality of simulated interfaces, to determine how to interact with different types of interfaces.