Abstract: The present disclosure relates to techniques for enabling dynamic memory evolution in large language models (LLMs) by selectively integrating user-provided facts or information during real-time interactive sessions. The disclosed techniques may include, upon receiving an input prompt from a user, generating a set of memory-weighted tokens using multiple transformer layers, each coupled to a dedicated memory pool comprising memory tokens that represent the model's internal knowledge. These memory-weighted tokens may be analyzed to identify salient, new, or distinct information in the input prompt relative to the existing memory content. The identified knowledge may then be incorporated into one or more memory pools prior to generating a response.

Abstract: Techniques are provided for producing an answer to a question regarding a domain. A natural-language textual sequence representing the question is received. From a knowledge graph associated with the domain, first and second textual passages are received using rankings corresponding to the natural-language textual sequence, a first textual summary is received summarizing textual information in a first vicinity of the first textual passage, and a second textual summary summarizing textual information in a vicinity of the second textual passage is received. An answer to the question is obtained using a language model by encoding a first intermediate output based on the natural-language textual sequence, the first textual passage, and the first textual summary, encoding a second intermediate output based on the natural language textual sequence, the second textual passage, and the second textual summary, and decoding a concatenation of the first and second intermediate outputs. An output is provided.

Abstract: Some techniques relate to generating a knowledge graph including textual passages and textual summaries usable for producing an answer to a question relating to a domain. A corpus of textual information is received. Textual passages and descriptions of associations between textual passages from a first language model are obtained by providing at least a portion of the corpus of textual information to the first language model. Textual summaries corresponding to textual passages are obtained from a second language model. A knowledge graph is generated based on the textual passages, descriptions of associations between the textual passages, and textual summaries. The knowledge graph is stored in a non-transitory computer readable medium.

Abstract: A natural-language textual sequence representing a question is received from a user. From a knowledge graph domain data is retrieved that includes: a first textual passage based on a first ranking with respect to the natural-language textual sequence, a first textual summary associated with the first textual passage, a second textual passage based on a first ranking with respect to a comparison of the second textual passage and the natural-language textual sequence, a second textual summary associated with the second textual passage, and a third textual summary based on a second ranking with respect to the natural-language textual sequence. An answer to the question from a language model is obtained by providing input to the language model based on the natural-language textual sequence, the first textual passage, the first textual summary, the second textual passage, the second textual summary, and the third textual summary.

Abstract: The present disclosure relates to a method and system for enhancing inference in large language models (LLMs) over long input sequences. A segmented inference strategy may be employed, wherein the long context can be divided and sequentially processed through a key-value (KV) cache of the LLM. At each step, the model may generate auxiliary outputs (or margins), which may include extractive summaries or intermediate signals based on the segment's relevance to an instruction. These margins may then be classified and selectively retained to guide final inference on the instruction. The retained margins may be prepended to the instruction to facilitate improved generation without modifying the model's internal weights. The disclosed approach provides efficient localization of relevant content, improves comprehension of extended contexts, and reduces computational overhead.

Abstract: The present disclosure relates to techniques for improving inference efficiency and memory utilization in transformer-based neural networks. A modified architectural design is introduced that decouples key and value matrix generation from inter-layer dependencies, enabling statically computed or parallelizable projections across layers. The disclosed approach may eliminate the need for layer-wise prefilling, support linear-time inference, and substantially reduce the memory footprint associated with key-value (KV) caching. The disclosed architecture may use shared or layer-specific projections, with a single KV-cache serving all or subsets of layers. In some embodiments, a non-linear transformation (e.g., implemented via a feed-forward network), may preprocess input embeddings prior to query, key, and value generation. A lookup table of transformed embeddings may be precomputed to further accelerate inference.

Abstract: Some techniques relate to generating a knowledge graph including textual passages and textual summaries usable for producing an answer to a question relating to a domain. A corpus of textual information is received. Textual passages and descriptions of associations between textual passages from a first language model are obtained by providing at least a portion of the corpus of textual information to the first language model. Textual summaries corresponding to textual passages are obtained from a second language model. A knowledge graph is generated based on the textual passages, descriptions of associations between the textual passages, and textual summaries. The knowledge graph is stored in a non-transitory computer readable medium.

Abstract: Techniques are provided for producing an answer to a question regarding a domain. A natural-language textual sequence representing the question is received. From a knowledge graph associated with the domain, first and second textual passages are received using rankings corresponding to the natural-language textual sequence, a first textual summary is received summarizing textual information in a first vicinity of the first textual passage, and a second textual summary summarizing textual information in a vicinity of the second textual passage is received. An answer to the question is obtained using a language model by encoding a first intermediate output based on the natural-language textual sequence, the first textual passage, and the first textual summary, encoding a second intermediate output based on the natural language textual sequence, the second textual passage, and the second textual summary, and decoding a concatenation of the first and second intermediate outputs. An output is provided.

Abstract: A natural-language textual sequence representing a question is received from a user. From a knowledge graph domain data is retrieved that includes: a first textual passage based on a first ranking with respect to the natural-language textual sequence, a first textual summary associated with the first textual passage, a second textual passage based on a first ranking with respect to a comparison of the second textual passage and the natural-language textual sequence, a second textual summary associated with the second textual passage, and a third textual summary based on a second ranking with respect to the natural-language textual sequence. An answer to the question from a language model is obtained by providing input to the language model based on the natural-language textual sequence, the first textual passage, the first textual summary, the second textual passage, the second textual summary, and the third textual summary.

Abstract: Techniques relate to receiving a natural-language textual sequence representing the textual input; retrieving, from a knowledge graph associated with the domain: a first textual passage based on a first ranking regarding the natural-language textual sequence, a first textual summary associated with the first textual passage, a second textual passage based on a first ranking with respect to a comparison of the second textual passage and the natural-language textual sequence a second textual summary associated with the second textual passage, and a third textual summary based on a second ranking regarding the natural-language textual sequence; obtaining the textual output in response to the textual input from a language model by providing input to the language model using the natural-language textual sequence, the first textual passage, the first textual summary, the second textual passage, the second textual summary, and the third textual summary; and providing an output based on the textual output.

Abstract: Some embodiments relate to receiving a natural-language textual sequence representing; retrieving, from a knowledge graph, a first textual passage and a second textual passage based on rankings with respect to the natural-language textual sequence, a first textual summary summarizing textual information in a first vicinity of the first textual passage, and a second textual summary summarizing textual information in a vicinity of the second textual passage; obtaining the textual output in response to the textual input using a language model by encoding a first intermediate output based on the natural-language textual sequence, the first textual passage, and the first textual summary, encoding a second intermediate output based on the natural language textual sequence, the second textual passage, and the second textual summary, and decoding a concatenation of the first intermediate output and the second intermediate output; and providing an output to a user based on the textual output.

Abstract: A training data system enables the generation of training data based on video content received from one or more outside video sources. For example, the generated training data can include a transcript of a word or phrase alongside emotion, language style, and brand perception data associated with that word or phrase. To generate the training data from a video, the subtitles, video frame, metadata, and audio levels of the video can be analyzed by the training data system. The generated training data (potentially from a plurality of videos) can then be grouped into a set of training data and used to train machine learning modules for Natural Language Processing (NLP) techniques.