Abstract: A system and method for estimating vascular flow using CT imaging include a computer readable storage medium having stored thereon a computer program comprising instructions, which, when executed by a computer, cause the computer to acquire a first set of data comprising anatomical information of an imaging subject, the anatomical information comprises information of at least one vessel. The instructions further cause the computer to process the anatomical information to generate an image volume comprising the at least one vessel, generate hemodynamic information based on the image volume, and acquire a second set of data of the imaging subject. The computer is also caused to generate an image comprising the hemodynamic information in combination with a visualization based on the second set of data.

Abstract: Techniques for creating machine learning models for direct homography regression for image rectification are described. In certain embodiments, a training service trains an algorithm on a source view of a training image and a homography matrix of the training image into a machine learning model that generates a normalized homography matrix for an input of the source view. The normalized homography matrix may then be utilized to generate a target view of an image input into the machine learning model. The target view of the image may be used in a document processing pipeline for document images captured using cameras.

Abstract: This disclosure describes automatically selecting and training one or more models for image recognition based upon training and testing (validation) data provided by a user. A service provider network includes a recognition service that may use models to process images and videos to recognize objects in the images and videos, features on the objects in the images and videos, and/or locate objects in the images and videos. The service provider network also includes a model selection and training service that may select one or more modeling techniques based on the objectives of the user and/or the amount of data provided by the user. Based on the selected modeling technique, the model selection and training service selects and trains one or more models for use by the recognition service to process images and videos using the training data. The trained model may be tested and validated using the testing data.

Abstract: Various approaches discussed herein enable browsing groups of visually similar items to an item of interest, wherein the item of interest may be identified in a query image, for example. One or more visual attributes associated with the item of interest are identified, and the visually similar items matching at least one of the visual attributes are grouped together, wherein the group is ranked according to the visually similar items' overall visual similarity to the item of interest, for example by using a visual similarity score and/or metric.

Abstract: Methods, systems, and computer-readable media for content moderation using object detection and image classification are disclosed. A content moderation system performs object detection on an input image using one or more object detectors. The object detection finds one or more elements in the input image. The content moderation system performs classification based at least in part on the input image using one or more image classifiers. The classification determines one or more values indicative of one or more content types in the input image. The content moderation system determines one or more scores for one or more content labels corresponding to the one or more content types. At least one of the scores represents a finding of one or more of the content types in the input image. The content moderation system generates output indicating the finding of the one or more of the content types.

Abstract: Techniques for feedback-based training are described.

Abstract: Techniques for anomaly detection are described. An exemplary method includes receiving one or more requests to train an anomaly detection machine learning model using feedback-based training, the request to indicate one or more of a type of analysis to perform, a model selection indication, and a configuration for a training dataset; training the anomaly detection machine learning model according to the one or more requests using the training data; performing feedback-based training on the trained anomaly detection machine learning model; and using the retrained anomaly detection machine learning model.

Abstract: Techniques for document rectification via homography recovery using machine learning are described. An image rectification system can intelligently make use of multiple pipelines for rectifying document images based on the detected type of device that generated the images. The image rectification system can provide high-quality rectifications without requiring human cooperation, multiple views of the document in multiple images, and/or without being constrained to only be able to process images from one source context.

Abstract: Techniques for auto-generation of annotated real-world training data are described. An electronic document is analyzed to determine text represented in the document and corresponding locations of the text. A representation of the electronic document is modified to include markers and printed. The printed document is photographed in real-world environments, and the markers within the digital photographs are analyzed to allow for the depiction of the document within the photographs to be rectified. The text and location data are used to annotate the rectified images.

Abstract: Techniques for grammar-based automated generation of annotated synthetic form training data for machine learning are described. A training data generation engine utilizes a defined grammar to construct a layout for a form, select key-value units to place within the layout, and select attribute variants for the key-value units. The form is rendered and stored at a storage location, where it can be provided along with other similarly-generated forms to be used as training data for a machine learning model.

Abstract: Techniques for training an embedding using a limited training set are described. In some examples, the embedding is trained by generating a plurality of vectors from a random sample of the limited set of training data classes using a layer of the particular machine learning classification model, randomly selecting samples from the plurality of vectors into a set of samples, computing at least one distance for each sampled class from a center parameter for the class using the set of samples, generating a discrete probability distribution over the classes for a query point based on distances to a center parameter for each of the classes in the embedding space, calculating a loss value for the modified prototypical network, the calculation of the loss value being for a fixed geometry of the embedding space and including a measure of the difference between distributions, and back propagating.

Abstract: Techniques for layout-agnostic complex document processing are described. A document processing service can analyze documents that do not adhere to defined layout rules in an automated manner to determine the content and meaning of a variety of types of segments within the documents. The service may chunk a document into multiple chunks, and operate upon the chunks in parallel by identifying segments within each chunk, classifying the segments into segment types, and processing the segments using special-purpose analysis engines adapted for the analysis of particular segment types to generate results that can be aggregated into an overall output for the entire document that captures the meaning and context of the document text.

Abstract: Techniques for keypoint-based multi-label word segmentation and localization are described. A machine learning model identifies bounding regions of text within an image, and then generates multiple channel matrices representing predicted keypoints of the text within the bounding regions. The keypoints can be used to rectify the corresponding graphical content from the image including the text to improve the ability to perform optical character recognition and identify the text. Line and word segmentation and localization can be performed together.

Abstract: Techniques for automated form understanding via layout-agnostic identification of keys and corresponding values are described. An embedding generator creates embeddings of pixels from an image including a representation of a form. The generated embeddings are similar for pixels within a same key-value unit, and far apart for pixels not in a same key-value unit. A weighted bipartite graph is constructed including a first set of nodes corresponding to keys of the form and a second set of nodes corresponding to values of the form. Weights for the edges are determined based on an analysis of distances between ones of the embeddings. The graph is partitioned according to a scheme to identify pairings between the first set of nodes and the second set of nodes that produces a minimum overall edge weight. The pairings indicate keys and values that are associated within the form.

Abstract: Techniques for layout-agnostic clustering-based classification of document keys and values are described. A key-value differentiation unit generates feature vectors corresponding to text elements of a form represented within an electronic image using a machine learning (ML) model. The ML model was trained utilizing a loss function that separates keys from values. The feature vectors are clustered into at least two clusters, and a cluster is determined to include either keys of the form or values of the form via identifying neighbors between feature vectors of the cluster(s) with labeled feature vectors.

Abstract: A structured document analyzer that associates keys and values in structured documents based on key, value, and key-value container bounding boxes. A trained machine learning model analyzes images of structured documents to determine bounding boxes for keys, values, and key-value containers in the images with confidence scores for the classifications. For each image, duplicate bounding boxes are removed, and then a set of key-value containers are selected and sorted based on the confidence scores. For each key-value container, a best key and value are determined for the container based on overlap of the key and value bounding boxes with the container bounding box and the confidence scores. Optical character recognition may be performed on the image to determine text for the keys and values.

Abstract: Embodiments described herein are directed to allowing manipulation of visual attributes of a query image while preserving the visual attributes of a query image. A query image can be received and analyzed using a trained network to determine a set of items whose images demonstrate visual similarity to the query image across a plurality of visual attributes. Visual attributes of the query image may be manipulated to allow a user to search for items that incorporate the desired manipulated visual attributes while preserving the visual attributes of the query image. Content for at least a determined number of highest ranked, or most similar, items related to the modified visual attributes can then be provided.

Abstract: Various embodiments provide for visual similarity based search techniques that certain desirable visual attributes of one or more items to search for items having similar visual attributes. In order to create an electronic catalog of items that is searchable by parts-based visual attributes, the visual attributes are identified and corresponding feature vectors are extracted from the image data of each item. Thus, feature values of parts-based visual attributes of items in the electronic catalog can be determined and used to select or rank the items in response to a search query based on desirable visual attributes. To conduct a search, a user may define desirable visual attributes of one or more items. The feature vectors of the desirable visual attributes are determined and used to query the electronic catalog of items, in which items having visual attributes of similar feature vectors are selected and returned as search results.

Abstract: Techniques for layout-agnostic complex document processing are described. A document processing service can analyze documents that do not adhere to defined layout rules in an automated manner to determine the content and meaning of a variety of types of segments within the documents. The service may chunk a document into multiple chunks, and operate upon the chunks in parallel by identifying segments within each chunk, classifying the segments into segment types, and processing the segments using special-purpose analysis engines adapted for the analysis of particular segment types to generate results that can be aggregated into an overall output for the entire document that captures the meaning and context of the document text.

Abstract: Approaches introduce a pre-processing and post-processing framework to a neural network-based approach to identify items represented in an image. For example, a classifier that is trained on several categories can be provided. An image that includes a representation of an item of interest is obtained. Rotated versions of the image are generated and each of a subset of the rotated images is analyzed to determine a probability that a respective image includes an instance of a particular category. The probabilities can be used to determine a probability distribution of output category data, and the data can be analyzed to select an image of the rotated versions of the image. Thereafter, a categorization tree can then be utilized, whereby for the item of interest represented the image, the category of the item can be determined. The determined category can be provided to an item retrieval algorithm to determine primary content for the item of interest.