Abstract: Methods and systems are disclosed herein for preventing hallucinations in machine learning models by enabling query responses based on a predefined ground truth defined by a corpus of information. The system may use multiple machine learning models. In particular, the system may receive a user query and use a machine learning model to split the user query into multiple sub-queries that would ask component questions for the query. The component questions may then be used to get accurate information for responding to the query. Once the information is identified, the hallucination prevention system may input that information into another machine learning model (for example, a large language model) with instructions to deliver the response to the query based on the identified information, but put it into a specific, desired form. In some embodiments, another machine learning model may be used to identify undesired responses based on policy and/or other requirements.