Abstract: A method, computer system, and a computer program product for data-free knowledge amalgamation are provided. Multiple pre-trained teacher machine learning models are obtained. Each is trained on a respective different set of training data. Pseudo-data samples that mimic original training data of the teacher models are generated. A block-wise amalgamation with a self-regulative strategy to integrate knowledge from the multiple teacher models is implemented by inputting the pseudo-data samples into the teacher models and into a student machine learning model. The implementing also includes aligning intermediate representations of the student model with a unified representation capturing relevant features from the teacher models.

Abstract: Data-free knowledge distillation for text classification can include generating, by a knowledge transfer system, a knowledge transfer dataset comprising a set of synthesized data samples adapted for a text classification task. A large language model is guided by a teacher model in generating the set of synthesized data samples. The knowledge distillation also includes training, by the knowledge transfer system using the teacher model, a student model by using the knowledge transfer dataset.

Abstract: Generator neural networks are trained to produce synthetic samples that mimic training data used to train pretrained neural network models, respectively. Teaching assistant neural networks that learn from the pretrained neural network models are trained through knowledge distillation using samples produced by the generator neural networks. A student neural network that learns from the pretrained neural network models is trained through knowledge distillation using the synthetic samples produced by the trained generator neural networks.

Abstract: One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to a process for privacy-enhanced machine learning and inference. A system can comprise a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory, wherein the computer executable components can comprise a processing component that generates an access rule that modifies access to first data of a graph database, wherein the first data comprises first party information identified as private, a sampling component that executes a random walk for sampling a first graph of the graph database while employing the access rule, wherein the first graph comprises the first data, and an inference component that, based on the sampling, generates a prediction in response to a query, wherein the inference component avoids directly exposing the first party information in the prediction.