Abstract: Various methods and processes, apparatuses/systems, and media for multimodal retrieval augmented generation for visually rich documents are disclosed. A processor implements a page-wise chunking algorithm to chunk a visual document into a plurality of page chunks; inputs the plurality of page chunks onto a trained embedding model; generates one vector for each page chunk. Each vector represents corresponding text, spatial and visual feature of each page of the visual document; inputs, in response to receiving a prompt of a query corresponding to the visual document, the vectors of the embedded chucks of pages retrieved from the database; identifies most relevant chunks of pages based on the similarities between the query prompt and chucks of pages; and generates, in response to inputting the most relevant chunks of pages onto a generative model, a response to the prompt corresponding to the visual document based on the identified most relevant chunks of pages.

Abstract: Various methods and processes, apparatuses/systems, and media for performing spatial-aware reading for visual documents are disclosed. A processor implements a language model; modifies existing parameters and architecture of the language model to incorporate new parameters in its architecture by integrating a disentangled spatial attention process to the language model; pretrains the modified model by performing an autoregressive block infilling process on a plurality of document pages thereby training the new parameters and further adjusting the existing parameters; instruction-tunes the pretrained model on data derived from a plurality of visually rich document understanding datasets to teach the pretrained model to follow document-oriented instructions or answer questions about documents by leveraging their content and their spatial layout and outputting a trained model; and performs spatial-aware reading for visual documents by utilizing the trained model.

Abstract: A method for automating a process of generating messages that are responsive to client inquiries by using an AI algorithm that implements a machine learning technique to ensure accuracy and timeliness in the responses is provided. The method includes: receiving a first message that includes an inquiry that relates to an account associated with a user; applying an AI algorithm for analyzing the first message in order to extract information that relates to the inquiry; determining, based on a result of the analysis, whether generating a response to the inquiry requires human intervention; when human intervention is not required, retrieving information that is responsive to the inquiry from a memory; generating a second message that includes the information that is responsive to the inquiry; and transmitting the second message to the user.

Abstract: Various methods and processes, apparatuses/systems, and media for performing spatial-aware reading for visual documents are disclosed. A processor implements a language model; modifies existing parameters and architecture of the language model to incorporate new parameters in its architecture by integrating a disentangled spatial attention process to the language model; pretrains the modified model by performing an autoregressive block infilling process on a plurality of document pages thereby training the new parameters and further adjusting the existing parameters; instruction-tunes the pretrained model on data derived from a plurality of visually rich document understanding datasets to teach the pretrained model to follow document-oriented instructions or answer questions about documents by leveraging their content and their spatial layout and outputting a trained model; and performs spatial-aware reading for visual documents by utilizing the trained model.

Abstract: A method for automating a process of generating messages that are responsive to client inquiries by using an AI algorithm that implements a machine learning technique to ensure accuracy and timeliness in the responses is provided. The method includes: receiving a first message that includes an inquiry that relates to an account associated with a user; applying an AI algorithm for analyzing the first message in order to extract information that relates to the inquiry; determining, based on a result of the analysis, whether generating a response to the inquiry requires human intervention; when human intervention is not required, retrieving information that is responsive to the inquiry from a memory; generating a second message that includes the information that is responsive to the inquiry; and transmitting the second message to the user.