Abstract: An information retrieval training system includes: a training dataset including training data having a feature space; the training data including multiple different types of elements, wherein no labels are provided with the training data; a training module configured to: maintain fixed a pre-trained model configured to receive features of queries; learn sets of pseudo-labels based on the training data; train parameters of adaptor modules for each of the sets of pseudo-labels, respectively, the adaptor modules configured to receive outputs of the pre-trained model, respectively; and train parameters of fusion modules based on neighboring pairs of the training data, the fusion modules configured to fuse together outputs of the adaptor modules, respectively.