Abstract: The disclosed method of training a language model includes generating, based on corpus data, clustered data, performing, based on the clustered data, a plurality of bootstrapping operations to train a first predictor model and generate a mixture configuration, generating, based on the mixture configuration and the clustered data, mixture data, and performing, based on the mixture data, one or more training operations to train a first untrained language model to generate a first trained language model, where the first trained language model processes first input data to generate first output data.

Abstract: In machine learning, some models, such as language models, are trained to perform language processing tasks. These language processing tasks can include text generation, machine translation, sentiment analysis, question answering, content generation, code development, etc. However, these models are inherently unreliable, especially when prompted with complex (e.g. composite) instructions. For example, they are prone to miscarrying basic operations, ignoring explicitly stated formatting and behavioral constraints, and hallucinating new steps or step outcomes. The present disclosure provides a mechanism that preprocesses an instruction to a machine learning model to generate a sequence of instructions via which flow control and data control in the model may be enforced, thereby enhancing reliability of the output generated by the model.