Abstract: Approaches that intend to reduce disparate impact, particularly those for providing equal coverage sets in conformal prediction, can in fact increase disparate impact for human-in-the-loop systems. To improve these systems, rather than optimizing selection processes for a class prediction set for a confidence level (e.g., a percentage confidence that the correct class is in the class prediction set), a selection process is determined that reduces the set size difference across groups.