Abstract: Caption association techniques as part of digital content creation by a computing device are described. The computing device is configured to extract text features and bounding boxes from an input document. These text features and bounding boxes are processed to reduce a number of possible search spaces. The processing may involve generating and utilizing a language model that captures the semantic meaning of the text features to identify and filter static text, and may involve identifying and filtering inline captions. A number of bounding boxes are identified for a potential caption. The potential caption and corresponding identified bounding boxes are concatenated into a vector. The concatenated vector is used to identify relationships among the bounding boxes to determine a single bounding box associated with the caption. The determined association is utilized to generate an output digital document that includes a structured association between the caption and a data entry field.

Abstract: A standardized data model (“SDM”) includes standardized data types that indicate classifications of data elements. In a data service platform, such as a marketing data platform, a data standardization module classifies received data elements. One or more components included in the data standardization module are trained using supervised or unsupervised learning techniques to classify received data elements into a standardized data type included in the SDM. In some cases, an output of an unsupervised learning phase is provided as an input to a supervised learning phase. In some cases, a classified data element is modified by the data standardization module to indicate the standardized data type into which the data element is classified.

Abstract: A conversational agent facilitates conversational searches for users. The conversational agent is a reinforcement learning (RL) agent trained using a user model generated from existing session logs from a search engine. The user model is generated from the session logs by mapping entries from the session logs to user actions understandable by the RL agent and computing conditional probabilities of user actions occurring given previous user actions in the session logs. The RL agent is trained by conducting conversations with the user model in which the RL agent selects agent actions in response to user actions sampled using the conditional probabilities from the user model.

Abstract: In implementations of digital image search based on arbitrary image features, a server computing device maintains an images database of digital images, and includes an image search system that receives a search input as a digital image depicting image features, and receives search criteria of depicted image features in the digital image. The image search system can then determine similar images to the received digital image based on similarity criterion corresponding to the search criteria. A trained image model of the image search system is applied to determine an image feature representation of the received digital image. A feature mask model of the image search system is applied to the image feature representation to determine a masked feature representation of the received digital image. The masked feature representation of the received digital image is compared to a masked feature representation of each respective database image to identify the similar images.

Abstract: Some implementations provide a feature recommendation system that receives sequences from user sessions with applications, where each sequence is of features of the applications in an order the features were used by a user. The sequences are applied to a feature embedding model that learns semantic similarities between the features based on occurrences of the features in the sequences in a same user session. A request is received for a feature recommendation that identifies a feature of an application used by a given user in a user session. A recommended feature for the feature recommendation is determined from a set of the semantic similarities that are between the identified feature and others of the features. The feature recommendation is presented on a user device associated with the given user.

Abstract: Techniques for determining reading order in a document. A current labeled text run (R1), RIGHT text run (R1) and DOWN text run (R3) are generated. The R1 labeled text run is processed by a first LSTM, the R2 labeled text run is processed by a second LSTM, and the R3 labeled text run is processed by a third LSTM, wherein each of the LSTMs generates a respective internal representation (R1?, R2? and R3?). Deep learning tools other than LSTMs can be used, as will be appreciated. The respective internal representations R1?, R2? and R3? are concatenated or otherwise combined into a vector or tensor representation and provided to a classifier network that generates a predicted label for a next text run as RIGHT, DOWN or EOS in the reading order of the document.

Abstract: Caption association techniques as part of digital content creation by a computing device are described. The computing device is configured to extract text features and bounding boxes from an input document. These text features and bounding boxes are processed to reduce a number of possible search spaces. The processing may involve generating and utilizing a language model that captures the semantic meaning of the text features to identify and filter static text, and may involve identifying and filtering inline captions. A number of bounding boxes are identified for a potential caption. The potential caption and corresponding identified bounding boxes are concatenated into a vector. The concatenated vector is used to identify relationships among the bounding boxes to determine a single bounding box associated with the caption. The determined association is utilized to generate an output digital document that includes a structured association between the caption and a data entry field.

Abstract: Systems and techniques map an input field from a data schema to a hierarchical standard data model (XDM). The XDM includes a tree of single XDM fields and each of the single XDM fields includes a composition of single level XDM fields. An input field from a data schema is processed by an unsupervised learning algorithm to obtain a sequence of vectors representing the input field and a sequence of vectors representing single level hierarchical standard data model (XDM) fields. These vectors are processed by a neural network to obtain a similarity score between the input field and each of the single level XDM fields. A probability of a match is determined using the similarity score between the input field and each of the single level XDM fields. The input field is mapped to the XDM field having the probability of the match with a highest score.

Abstract: Techniques for determining reading order in a document. A current labeled text run (R1), RIGHT text run (R1) and DOWN text run (R3) are generated. The R1 labeled text run is processed by a first LSTM, the R2 labeled text run is processed by a second LSTM, and the R3 labeled text run is processed by a third LSTM, wherein each of the LSTMs generates a respective internal representation (R1?, R2? and R3?). Deep learning tools other than LSTMs can be used, as will be appreciated. The respective internal representations R1?, R2? and R3? are concatenated or otherwise combined into a vector or tensor representation and provided to a classifier network that generates a predicted label for a next text run as RIGHT, DOWN or EOS in the reading order of the document.

Abstract: In implementations of digital image search based on arbitrary image features, a server computing device maintains an images database of digital images, and includes an image search system that receives a search input as a digital image depicting image features, and receives search criteria of depicted image features in the digital image. The image search system can then determine similar images to the received digital image based on similarity criterion corresponding to the search criteria. A trained image model of the image search system is applied to determine an image feature representation of the received digital image. A feature mask model of the image search system is applied to the image feature representation to determine a masked feature representation of the received digital image. The masked feature representation of the received digital image is compared to a masked feature representation of each respective database image to identify the similar images.

Abstract: A standardized data model (“SDM”) includes standardized data types that indicate classifications of data elements. In a data service platform, such as a marketing data platform, a data standardization module classifies received data elements. One or more components included in the data standardization module are trained using supervised or unsupervised learning techniques to classify received data elements into a standardized data type included in the SDM. In some cases, an output of an unsupervised learning phase is provided as an input to a supervised learning phase. In some cases, a classified data element is modified by the data standardization module to indicate the standardized data type into which the data element is classified.

Abstract: Some implementations provide a feature recommendation system that receives sequences from user sessions with applications, where each sequence is of features of the applications in an order the features were used by a user. The sequences are applied to a feature embedding model that learns semantic similarities between the features based on occurrences of the features in the sequences in a same user session. A request is received for a feature recommendation that identifies a feature of an application used by a given user in a user session. A recommended feature for the feature recommendation is determined from a set of the semantic similarities that are between the identified feature and others of the features. The feature recommendation is presented on a user device associated with the given user.

Abstract: In some embodiments, a processor accesses a metrics dataset, which includes metrics whose values indicate data network activity. The metrics dataset has segments. Each segment is a respective subset of the data items having a common feature. The processor identifies anomalous segments in the metrics dataset. Each anomalous segment has a segment trend that is different from a trend associated with the larger metrics dataset. The processor generates a data graph that includes nodes, which represent anomalous segments, and edges connecting the nodes. The processor applies weights to the edges. Each weight indicates (i) a similarity between a pair of anomalous segments represented by the nodes connected by the weighted edge and (ii) a relationship between the anomalous segments and the metrics dataset. The processor ranks the anomalous segments based on the applied weights and selects one or more segments with sufficiently high ranks.

Abstract: A conversational agent facilitates conversational searches for users. The conversational agent is a reinforcement learning (RL) agent trained using a user model generated from existing session logs from a search engine. The user model is generated from the session logs by mapping entries from the session logs to user actions understandable by the RL agent and computing conditional probabilities of user actions occurring given previous user actions in the session logs. The RL agent is trained by conducting conversations with the user model in which the RL agent selects agent actions in response to user actions sampled using the conditional probabilities from the user model.

Abstract: In some embodiments, a processor accesses a metrics dataset, which includes metrics whose values indicate data network activity. The metrics dataset has segments. Each segment is a respective subset of the data items having a common feature. The processor identifies anomalous segments in the metrics dataset. Each anomalous segment has a segment trend that is different from a trend associated with the larger metrics dataset. The processor generates a data graph that includes nodes, which represent anomalous segments, and edges connecting the nodes. The processor applies weights to the edges. Each weight indicates (i) a similarity between a pair of anomalous segments represented by the nodes connected by the weighted edge and (ii) a relationship between the anomalous segments and the metrics dataset. The processor ranks the anomalous segments based on the applied weights and selects one or more segments with sufficiently high ranks.