Abstract: A model visualization system analyzes model behavior to identify clusters of data instances with similar behavior. For a selected feature, data instances are modified to set the selected feature to different values evaluated by a model to determine corresponding model outputs. The feature values and outputs may be visualized in an instance-feature variation plot. The instance-feature variation plots for the different data instances may be clustered to identify latent differences in behavior of the model with respect to different data instances when varying the selected feature. The number of clusters for the clustering may be automatically determined, and the clusters may be further explored by identifying another feature which may explain the different behavior of the model for the clusters, or by identifying outlier data instances in the clusters.