Abstract: Techniques for training a machine-learning (ML) model for captioning images are disclosed. A plurality of feature vectors and a plurality of visual attention maps are generated by a visual model of the ML model based on an input image. Each of the plurality of feature vectors correspond to different regions of the input image. A plurality of caption attention maps are generated by an attention model of the ML model based on the plurality of feature vectors. An attention penalty is calculcated based on a comparison between the caption attention maps and the visual attention maps. A loss function is calculcated based on the attention penalty. One or both of the visual model and the attention model are trained using the loss function.

Abstract: Systems and methods for identifying and segmenting objects from images include a preprocessing module configured to adjust a size of a source image; a region-proposal module configured to propose one or more regions of interest in the size-adjusted source image; and a prediction module configured to predict a classification, bounding box coordinates, and mask. Such systems and methods may utilize end-to-end training of the modules using adversarial loss, facilitating the use of a small training set, and can be configured to process historical documents, such as large images comprising text. The preprocessing module within said systems and methods can utilize a conventional image scaler in tandem with a custom image scaler to provide a resized image suitable for GPU processing, and the region-proposal module can utilize a region-proposal network from a single-stage detection model in tandem with a two-stage detection model paradigm to capture substantially all particles in an image.

Abstract: Systems and methods for determining whether two tree persons in a genealogical database correspond to the same real-life individual. Embodiments include identifying two tree persons in a genealogical database and extracting a plurality of features from both tree persons to generate two vectors. Embodiments also include calculating a plurality of metrics between the two vectors to generate a metric function. Embodiments further include generating feature weights using a recursive process based on training data input by external users, and generating a score by calculating a weighted sum of the metric function being weighted by the feature weights. The generated score may then be compared to a threshold value.

Abstract: Techniques for training a machine-learning (ML) model for captioning images are disclosed. A plurality of feature vectors and a plurality of visual attention maps are generated by a visual model of the ML model based on an input image. Each of the plurality of feature vectors correspond to different regions of the input image. A plurality of caption attention maps are generated by an attention model of the ML model based on the plurality of feature vectors. An attention penalty is calculated based on a comparison between the caption attention maps and the visual attention maps. A loss function is calculated based on the attention penalty. One or both of the visual model and the attention model are trained using the loss function.

Abstract: Described herein are systems, methods, and other techniques for segmenting an input text. A set of tokens are extracted from the input text. Token representations are computed for the set of tokens. The token representations are provided to a machine learning model that generates a set of label predictions corresponding to the set of tokens. The machine learning model was previously trained to generate label predictions in response to being provided input token representations. Each of the set of label predictions indicates a position of a particular token of the set of tokens with respect to a particular segment. One or more segments within the input text are determined based on the set of label predictions.

Abstract: Systems, methods, and other techniques for genealogical entity resolution. In some embodiments, first tree data and second tree data are obtained, the first tree data corresponding to a first tree person and the second tree data corresponding to a second tree person. A set of features is extracted from the first tree data and the second tree data. An individual-level similarity score for each possible pairing of tree persons is generated based on the set of features. A set of most-similar tree persons is identified based on the individual-level similarity score for each possible pairing. A plurality of individual-level similarity scores for the set of most-similar tree persons are provided as input to a family-level ML model to determine that the first tree person and the second tree person correspond to a same individual.

Abstract: Systems and methods for determining whether two tree persons in a genealogical database correspond to the same real-life individual. Embodiments include identifying two tree persons in a genealogical database and extracting a plurality of features from both tree persons to generate two vectors. Embodiments also include calculating a plurality of metrics between the two vectors to generate a metric function. Embodiments further include generating feature weights using a recursive process based on training data input by external users, and generating a score by calculating a weighted sum of the metric function being weighted by the feature weights. The generated score may then be compared to a threshold value.

Abstract: Augmented reality is used to display graphical elements overlaid on a continually updating image of an area around an augmented reality device (e.g., a mobile device). The graphical element may contain geographical location information about a grave of an ancestor and/or biographical information about the ancestor. The continually updating image is captured by a camera of the augmented reality device and updates in response to time and motion of the augmented reality device. Based on orientation data and geographical location data collected by the augmented reality device, the graphical element is updated and displayed on the mobile device.

Abstract: Described herein are systems, methods, and other techniques for training a machine learning (ML) model to jointly perform named entity recognition (NER) and relation extraction (RE) on an input text. A set of hyperparameters for the ML model are set to a first set of values. The ML model is trained using a training dataset and is evaluated to produce a first result. The set of hyperparameters are modified from the first set of values to a second set of values. The ML model is trained using the training dataset and is evaluated to produce a second result. Either the first set of values or the second set of values are selected and used for the set of hyperparameters for the ML model based on a comparison between the first result and the second result.

Abstract: Embodiments described herein relate generally to a methodology of efficient object classification within a visual medium. The methodology utilizes a first neural network to perform an attention based object localization within a visual medium to generate a visual mask. The visual mask is applied to the visual medium to generate a masked visual medium. The masked visual medium may be then fed into a second neural network to detect and classify objects within the visual medium.

Abstract: Systems and methods for classifying historical images. A feature extractor may create feature vectors corresponding to a plurality of images. A first classification of the plurality of images may be performed based on the plurality of feature vectors, which may include assigning a label to each of the plurality of images and assigning a probability for each of the assigned labels. The assigned probability for each of the assigned labels may be related to a statistical confidence that a particular assigned label is correctly assigned to a particular image. A subset of the plurality of images may be displayed to a display device. An input corresponding to replacement of an incorrect label with a corrected label for a certain image may be received from a user. A second classification of the plurality of images based on the input from the user may be performed.

Abstract: Described herein are systems, methods, and other techniques for training a generative adversarial network (GAN) to perform an image-to-image transformation for recognizing text. A pair of training images are provided to the GAN. The pair of training images include a training image containing a set of characters in handwritten form and a reference training image containing the set of characters in machine-recognizable form. The GAN includes a generator and a discriminator. The generated image is generated using the generator based on the training image. Update data is generated using the discriminator based on the generated image and the reference training image. The GAN is trained by modifying one or both of the generator and the discriminator using the update data.

Abstract: Embodiments described herein relate generally to a methodology of efficient object classification within a visual medium. The methodology utilizes a first neural network to perform an attention based object localization within a visual medium to generate a visual mask. The visual mask is applied to the visual medium to generate a masked visual medium. The masked visual medium may be then fed into a second neural network to detect and classify objects within the visual medium.

Abstract: Systems and methods for classifying historical images. A feature extractor may create feature vectors corresponding to a plurality of images. A first classification of the plurality of images may be performed based on the plurality of feature vectors, which may include assigning a label to each of the plurality of images and assigning a probability for each of the assigned labels. The assigned probability for each of the assigned labels may be related to a statistical confidence that a particular assigned label is correctly assigned to a particular image. A subset of the plurality of images may be displayed to a display device. An input corresponding to replacement of an incorrect label with a corrected label for a certain image may be received from a user. A second classification of the plurality of images based on the input from the user may be performed.

Abstract: Systems, methods, and other techniques for extracting data from obituaries are provided. In some embodiments, an obituary containing a plurality of words is received. Using a machine learning model, an entity tag from a set of entity tags may be assigned to each of one or more words of the plurality of words. Each particular tag from the set of entity tags may include a relationship component and a category component. The relationship component may indicate a relationship between a particular word and the deceased individual. The category component may indicate a categorization of the particular word to a particular category from a set of categories. The extracted data may be stored in a genealogical database.

Abstract: An information entropy-based metric is used to represent a degree of diversity of a search result of genealogical records. In response to a query, a data query server locates a set of multiple records that match the query. The records are classified into different record types based on the records' attributes. One or more distributions of numbers of records classified into each record type are determined. Each distribution corresponds to one of the subsets the records. For each distribution, an entropy value is determined. A cumulative entropy that corresponds to a sum of the entropy values of those distributions is then determined. The cumulative entropy may serve as the entropy-based metric of the search result. The cumulative entropy may also be normalized by an ideal cumulative entropy. The normalized metric allows the diversity of different search results to be compared across different queries that may generate different numbers of records.

Abstract: Embodiments described herein relate generally to a methodology of efficient object classification within a visual medium. The methodology utilizes a first neural network to perform an attention based object localization within a visual medium to generate a visual mask. The visual mask is applied to the visual medium to generate a masked visual medium. The masked visual medium may be then fed into a second neural network to detect and classify objects within the visual medium.

Abstract: Techniques for generating genealogical stories by tracing genealogical trees are disclosed. An input tree person within a genealogical tree is identified. The genealogical tree may include a plurality of tree persons corresponding to individuals. The plurality of tree persons may be interconnected through a plurality of connections. The plurality of connections in the genealogical tree may be traced based on one or more story tracing filters to identify one or more tree persons from the plurality of tree persons that are related to the input tree person and to identify data elements associated with the one or more tree persons. The data elements associated with the one or more tree persons may be filtered based on one or more story data filters. Filtering the data elements may cause a subset of the data elements to be selected. A genealogical story may be generated by aggregating the filtered data elements.

Abstract: Embodiments described herein relate generally to a methodology of efficient object classification within a visual medium. The methodology utilizes a first neural network to perform an attention based object localization within a visual medium to generate a visual mask. The visual mask is applied to the visual medium to generate a masked visual medium. The masked visual medium may be then fed into a second neural network to detect and classify objects within the visual medium.

Abstract: Systems and methods for determining whether two tree persons in a genealogical database correspond to the same real-life individual. Embodiments include obtaining, from a tree database, a first tree person from a first genealogical tree and a second tree person from a second genealogical tree. Embodiments also include identifying a plurality of familial categories. Embodiments My further include, for each familial category of the plurality of familial categories, extracting a first quantity of features for each of the tree persons in the familial category, generating a first similarity score for each possible pairing of tree persons, identifying a representative pairing based on a maximum first similarity score, and extracting a second quantity of features for each of the tree persons in the representative pairing. Embodiments may also include generating a second similarity score based on the second quantity of features.