Abstract: A methodology for auditing the visual diversity of unlabeled human face image datasets uses a set of core human interpretable dimensions derived from human similarity judgments. Given a face image, a model can output dimensional values aligned with the human mental representational space of faces, where values not only express the presence of a feature, but also its extent. Since the model can be learned entirely from human behavior, the learned dimensions are not biased toward features that are easier to verbalize or quantify.

Abstract: A multi-attribute bias amplification metric illustrates the need to consider multiple attributes when measuring bias amplification. For datasets that are perfectly balanced with respect to single attributes, bias amplification can still occur with respect to multi-attributes, regardless of whether raw or absolute differences are used. The metric can be used to show that methods used to mitigate single attribute bias can inadvertently increase multi-attribute bias amplification. Accordingly, the methods for determining bias amplification can provide a better understanding of the extent of bias a model introduces from training to prediction. Further, counterfactuals can be generated to decorrelate the co-occurrences of protected attributes with all background objects, both labeled and unlabeled. These generated counterfactuals can be used for both augmenting training and testing datasets.

Abstract: Systems and methods are used to mitigate societal bias in image-text datasets by removing spurious correlations between protected groups and image attributes. Using text-guided inpainting models, the methods ensures protected group independence from all attributes and mitigates inpainting biases through data filtering. Evaluations on multi-label image classification and image captioning tasks show that the methods effectively reduce bias without compromising performance across various models.

Abstract: A diverse dataset of human images can be created by collecting a plurality of images from a plurality of diverse people. A first graphical user interface requires a user to provide subject data, instrument data and environment data as metadata for each of the plurality of images. A second graphical user interface requires a user to form a bounding box about a face of a subject in each of the plurality of images. A third graphical user interface requires annotators to provide annotations for each of the plurality of images. The dataset may be used for training or evaluating machine learning or artificial intelligence systems, such as systems for body and face detection, body and face landmark detection, body and face parsing, face alignment, face recognition, face verification, image editing and image synthesis.

Abstract: A methodology for auditing the visual diversity of unlabeled human face image datasets uses a set of core human interpretable dimensions derived from human similarity judgments. Given a face image, a model can output dimensional values aligned with the human mental representational space of faces, where values not only express the presence of a feature, but also its extent. Since the model can be learned entirely from human behavior, the learned dimensions are not biased toward features that are easier to verbalize or quantify.

Abstract: A multi-attribute bias amplification metric illustrates the need to consider multiple attributes when measuring bias amplification. For datasets that are perfectly balanced with respect to single attributes, bias amplification can still occur with respect to multi-attributes, regardless of whether raw or absolute differences are used. The metric can be used to show that methods used to mitigate single attribute bias can inadvertently increase multi-attribute bias amplification. Accordingly, the methods for determining bias amplification can provide a better understanding of the extent of bias a model introduces from training to prediction. Further, counterfactuals can be generated to decorrelate the co-occurrences of protected attributes with all background objects, both labeled and unlabeled. These generated counterfactuals can be used for both augmenting training and testing datasets.