Abstract: A computer-implemented chatbot including a prompt generation module, a large language model (LLM) module, and an answer generation module. The prompt generation modules generates an initial prompt based on a prompt template combined with a received user query. The initial prompt includes information source data specifying sources of factual information, conversation history data, and failed response data. The initial prompt is input to the LLM module, which is configured to generate an output and communicate the output to the answer generation module. The answer generation modules determines if the output is a plan to answer a user query. If so, relevant data is retrieved from external database or more APIs specified in the information source data. A further prompt is generated for answering the user query and input to the LLM module. The answer generation module repeats its tasks until a suitable answer to the user query is output.

Abstract: A computer implemented method of detecting hallucination in a large language model (LLM) output. A message is received. A prompt is generated for an LLM including the message and an instruction to generate an output identifying predetermined content in the message. The prompt is passed through an LLM to generate the output. The output is processed in accordance with a hallucination detection process to identify if any predetermined content identified by the LLM in the output is potentially hallucinated.

Abstract: A system and computer implemented method for detecting hallucination in output of a generative AI system. User input is received specifying a query or task relating to information contained in a data object. A first vector representation of the user input and a second vector representation of the data object are generated. The first and second vector representations are compared to identify parts of the data object which match the query or task. An input is generated for a generative AI system with the user input and the parts of the data object. The input is input to a generative AI system. An output produced by the generative AI system is analysed to determine if the output contains information also present in the data object. If not, an error process is initiated. If so, the output is produced by the generative AI system.

Abstract: A machine learning model may be trained using annotated communications data. Each communication (e.g., a short messaging system (SMS) message or email) is annotated with a measure of user interaction. The machine learning model is thus trained to predict a measure of user interaction for future communications. Before sending future communications, at least a portion of the communication is provided to the trained machine learning model to predict the expected measure of user interaction with the communication. In response to the prediction, the sender of the communication may alter the communication. The system may automatically send the communication if the predicted measure of user interaction exceeds a predetermined threshold and only prompt the user if the predicted measure of user interaction does not exceed the predetermined threshold.

Abstract: A machine learning model may be trained using annotated communications data. Each communication (e.g., a short messaging system (SMS) message or email) is annotated with a measure of user interaction. The machine learning model is thus trained to predict a measure of user interaction for future communications. Before sending future communications, at least a portion of the communication is provided to the trained machine learning model to predict the expected measure of user interaction with the communication. In response to the prediction, the sender of the communication may alter the communication. The system may automatically send the communication if the predicted measure of user interaction exceeds a predetermined threshold and only prompt the user if the predicted measure of user interaction does not exceed the predetermined threshold.

Abstract: A machine learning model may be trained using annotated communications data. Each communication (e.g., a short messaging system (SMS) message or email) is annotated with a measure of user interaction. The machine learning model is thus trained to predict a measure of user interaction for future communications. Before sending future communications, at least a portion of the communication is provided to the trained machine learning model to predict the expected measure of user interaction with the communication. In response to the prediction, the sender of the communication may alter the communication. The system may automatically send the communication if the predicted measure of user interaction exceeds a predetermined threshold and only prompt the user if the predicted measure of user interaction does not exceed the predetermined threshold.