Abstract: Adapting a machine learning model to process data that differs from training data used to configure the model for a specified objective is described. A domain adaptation system trains the model to process new domain data that differs from a training data domain by using the model to generate a feature representation for the new domain data, which describes different content types included in the new domain data. The domain adaptation system then generates a probability distribution for each discrete region of the new domain data, which describes a likelihood of the region including different content described by the feature representation. The probability distribution is compared to ground truth information for the new domain data to determine a loss function, which is used to refine model parameters. After determining that model outputs achieve a threshold similarity to the ground truth information, the model is output as a domain-agnostic model.

Abstract: Techniques are provided for generating a digital image of simulated handwriting using an encoder-decoder neural network trained on images of natural handwriting samples. The simulated handwriting image can be generated based on a style of a handwriting sample and a variable length coded text input. The style represents visually distinctive characteristics of the handwriting sample, such as the shape, size, slope, and spacing of the letters, characters, or other markings in the handwriting sample. The resulting simulated handwriting image can include the text input rendered in the style of the handwriting sample. The distinctive visual appearance of the letters or words in the simulated handwriting image mimics the visual appearance of the letters or words in the handwriting sample image, whether the letters or words in the simulated handwriting image are the same as in the handwriting sample image or different from those in the handwriting sample image.

Abstract: Systems and methods for machine learning based multipage scanning are provided. In one embodiment, one or more processing devices perform operations that include receiving a video stream that includes image frames that capture a plurality of pages of a document. The operations further include detection, via a machine learning model that is trained to infer events from the video stream detects, a new page event. Detection of the new page event indicates that a page of the plurality of pages available for scanning has changed from a first page to a second page. Based on the detection of the new page event, the one or more processing devices capture an image frame of the page from the video stream. In some embodiments, the machine learning model detects events based on a weighted use of video data, inertial data, audio samples, image depth information, image statistics and/or other information.

Abstract: Systems and methods for performing Document Visual Question Answering tasks are described. A document and query are received. The document encodes document tokens and the query encodes query tokens. The document is segmented into nested document sections, lines, and tokens. A nested structure of tokens is generated based on the segmented document. A feature vector for each token is generated. A graph structure is generated based on the nested structure of tokens. Each graph node corresponds to the query, a document section, a line, or a token. The node connections correspond to the nested structure. Each node is associated with the feature vector for the corresponding object. A graph attention network is employed to generate another embedding for each node. These embeddings are employed to identify a portion of the document that includes a response to the query. An indication of the identified portion of the document is be provided.

Abstract: Various disclosed embodiments can resolve output inaccuracies produced by many machine learning models. Embodiments use content order as input to machine learning model systems so that they can process documents according to the position or rank of instances in a document or image. In this way, the model is less likely to misclassify or incorrectly detect instances or the ordering between predicted instances. The content order in various embodiments can be used as an additional signal to classify or make predictions.

Abstract: Adapting a machine learning model to process data that differs from training data used to configure the model for a specified objective is described. A domain adaptation system trains the model to process new domain data that differs from a training data domain by using the model to generate a feature representation for the new domain data, which describes different content types included in the new domain data. The domain adaptation system then generates a probability distribution for each discrete region of the new domain data, which describes a likelihood of the region including different content described by the feature representation. The probability distribution is compared to ground truth information for the new domain data to determine a loss function, which is used to refine model parameters. After determining that model outputs achieve a threshold similarity to the ground truth information, the model is output as a domain-agnostic model.

Abstract: Techniques are disclosed for sharing user markings between digital documents and corresponding physically printed documents. The sharing is facilitated using an Augmented Reality (AR) device, such as a smartphone or a tablet. The device streams images of a page of a book on a display. The device accesses a corresponding digital document that is a digital version of content printed on the book. In an example, the digital document has a digital user marking, e.g., a comment associated with a paragraph of the digital document, wherein a corresponding paragraph of the physical book lacks any such comment. When the device streams the images of the page of the book on the display, the device appends the digital comment on the paragraph of the page of the book within the image stream. Thus, the user can view the digital comment in the AR environment, while reading the physical book.

Abstract: Various disclosed embodiments can resolve output inaccuracies produced by many machine learning models. Embodiments use content order as input to machine learning model systems so that they can process documents according to the position or rank of instances in a document or image. In this way, the model is less likely to misclassify or incorrectly detect instances or the ordering between predicted instances. The content order in various embodiments can be used as an additional signal to classify or make predictions.

Abstract: Adapting a machine learning model to process data that differs from training data used to configure the model for a specified objective is described. A domain adaptation system trains the model to process new domain data that differs from a training data domain by using the model to generate a feature representation for the new domain data, which describes different content types included in the new domain data. The domain adaptation system then generates a probability distribution for each discrete region of the new domain data, which describes a likelihood of the region including different content described by the feature representation. The probability distribution is compared to ground truth information for the new domain data to determine a loss function, which is used to refine model parameters. After determining that model outputs achieve a threshold similarity to the ground truth information, the model is output as a domain-agnostic model.

Abstract: Techniques are disclosed for neural network based windowed contextual pooling. A methodology implementing the techniques according to an embodiment includes segmenting input feature channels into first and second groups of feature channels. The method also includes applying a first windowed pooling process to the first group of feature channels to generate a first group of pooled feature channels and applying a second windowed pooling process to the second group of feature channels to generate a second group of pooled feature channels. The method further includes performing a weighted merging of the first group of pooled feature channels and the second group of pooled feature channels to generate merged pooled feature channels.

Abstract: Techniques are provided for generating a digital image of simulated handwriting using an encoder-decoder neural network trained on images of natural handwriting samples. The simulated handwriting image can be generated based on a style of a handwriting sample and a variable length coded text input. The style represents visually distinctive characteristics of the handwriting sample, such as the shape, size, slope, and spacing of the letters, characters, or other markings in the handwriting sample. The resulting simulated handwriting image can include the text input rendered in the style of the handwriting sample. The distinctive visual appearance of the letters or words in the simulated handwriting image mimics the visual appearance of the letters or words in the handwriting sample image, whether the letters or words in the simulated handwriting image are the same as in the handwriting sample image or different from those in the handwriting sample image.

Abstract: Techniques are provided for generating a digital image of simulated handwriting using an encoder-decoder neural network trained on images of natural handwriting samples. The simulated handwriting image can be generated based on a style of a handwriting sample and a variable length coded text input. The style represents visually distinctive characteristics of the handwriting sample, such as the shape, size, slope, and spacing of the letters, characters, or other markings in the handwriting sample. The resulting simulated handwriting image can include the text input rendered in the style of the handwriting sample. The distinctive visual appearance of the letters or words in the simulated handwriting image mimics the visual appearance of the letters or words in the handwriting sample image, whether the letters or words in the simulated handwriting image are the same as in the handwriting sample image or different from those in the handwriting sample image.

Abstract: Adapting a machine learning model to process data that differs from training data used to configure the model for a specified objective is described. A domain adaptation system trains the model to process new domain data that differs from a training data domain by using the model to generate a feature representation for the new domain data, which describes different content types included in the new domain data. The domain adaptation system then generates a probability distribution for each discrete region of the new domain data, which describes a likelihood of the region including different content described by the feature representation. The probability distribution is compared to ground truth information for the new domain data to determine a loss function, which is used to refine model parameters. After determining that model outputs achieve a threshold similarity to the ground truth information, the model is output as a domain-agnostic model.

Abstract: Techniques are disclosed for sharing user markings between digital documents and corresponding physically printed documents. The sharing is facilitated using an Augmented Reality (AR) device, such as a smartphone or a tablet. The device streams images of a page of a book on a display. The device accesses a corresponding digital document that is a digital version of content printed on the book. In an example, the digital document has a digital user marking, e.g., a comment associated with a paragraph of the digital document, wherein a corresponding paragraph of the physical book lacks any such comment. When the device streams the images of the page of the book on the display, the device appends the digital comment on the paragraph of the page of the book within the image stream. Thus, the user can view the digital comment in the AR environment, while reading the physical book.

Abstract: Techniques are provided for generating a digital image of simulated handwriting using an encoder-decoder neural network trained on images of natural handwriting samples. The simulated handwriting image can be generated based on a style of a handwriting sample and a variable length coded text input. The style represents visually distinctive characteristics of the handwriting sample, such as the shape, size, slope, and spacing of the letters, characters, or other markings in the handwriting sample. The resulting simulated handwriting image can include the text input rendered in the style of the handwriting sample. The distinctive visual appearance of the letters or words in the simulated handwriting image mimics the visual appearance of the letters or words in the handwriting sample image, whether the letters or words in the simulated handwriting image are the same as in the handwriting sample image or different from those in the handwriting sample image.

Abstract: Various disclosed embodiments can resolve output inaccuracies produced by many machine learning models. Embodiments use content order as input to machine learning model systems so that they can process documents according to the position or rank of instances in a document or image. In this way, the model is less likely to misclassify or incorrectly detect instances or the ordering between predicted instances. The content order in various embodiments can be used as an additional signal to classify or make predictions.