Abstract: A novel, linear modeling method to model a face recognition algorithm based on the match scores produced by the algorithm. Starting with a distance matrix representing the pair-wise match scores between face images, an iterative stress minimization algorithm is used to obtain an embedding of the distance matrix in a low-dimensional space. A linear transformation used to project new face images into the model space is divided into two sub-transformations: a rigid transformation of face images obtained through principal component analysis of face images and a non-rigid transformation responsible for preserving pair-wise distance relationships between face images. Also provided is a linear indexing method using the linear modeling method to perform the binning or algorithm-specific indexing task with little overhead.