Abstract: Simultaneous Weighted Preference Optimization (SWEPO) is a method for enhancing machine learning model alignment by addressing alignment biases. This approach involves calculating mean reward scores for multiple responses to a query, computing deviations, and assigning weights based on these deviations. The method partitions responses into positive and negative sets, generating a weighted contrastive loss function to optimize model parameters. This process prioritizes responses with significant deviations, improving model performance by focusing on the most informative examples. The system can be implemented on a single or distributed computing architecture, facilitating efficient training and inference processes.

Abstract: An intelligent router for generative artificial intelligence (GAI) model instances optimizes request routing to reduce latency. The system predicts output lengths using a trained response-length predictor and assesses the state of multiple GAI instances, including prompt and decode distributions. It estimates the workload mixing impact of routing requests to each instance and determines selection probabilities using a machine-learning routing model. The router either assigns the request to the most suitable instance or delays routing if conditions are suboptimal. This approach improves end-to-end latency, Time-To-First-Token (TTFT), and Time-Between-Tokens (TBT) by considering the distinct characteristics of GAI workload phases.

Abstract: Techniques for generating optimized input prompts for generative language models combines automated prompt refinement with intelligent context retrieval. A prompt optimization service automatically refines initial prompt instructions through controlled mutation and evaluation cycles using multiple generative language models. A context management service maintains a vector database of historical examples pre-encoded as embedding vectors, enabling efficient retrieval of semantically similar examples for new tasks. A dynamic prompt generator assembles optimized prompts by combining refined instructions with relevant historical examples and task context. The system supports deployment with both large language models and smaller fine-tuned models, enabling efficient operation across diverse applications. Configurable parameters allow balancing optimization quality against computational costs.

Abstract: Described herein are techniques for evaluating large language model outputs through autonomous generation of context-aware evaluation criteria without relying on static human-defined standards. The approach enables dynamic generation of evaluation criteria tailored to specific instructions and responses, while incorporating context-specific knowledge crucial for accurate assessment. A framework implements both absolute evaluation against reference answers and relative comparison between multiple responses. Knowledge distillation techniques create efficient smaller models capable of criteria generation and evaluation with performance comparable to larger models. The technique demonstrates significant improvements in evaluation accuracy across diverse tasks while reducing computational costs through optimized model architectures. Additionally, the approach enhances preference-based learning through dynamically generated evaluation criteria, improving model alignment with human judgment.

Abstract: Described herein are techniques for improving language model performance through collaborative interaction between specialized and general-purpose models. A specialized model with fewer than ten billion parameters undergoes supervised fine-tuning on domain-specific data and generates initial outputs. These outputs are refined by a general-purpose model having over one hundred billion parameters and advanced reasoning capabilities. The framework implements preference tuning where outputs from both models are evaluated to generate preference triplets that optimize the performance of the specialized model. This approach achieves significant accuracy improvements while maintaining data privacy and computational efficiency.

Abstract: A technique for optimizing attention mechanism computations in transformer-based language models improves computational efficiency during both prefill and decode phases. The approach unequally partitions attention operations across multiple streaming multiprocessors of a hardware processing unit (e.g., such as a graphics processing unit, or GPU) to maximize hardware utilization. By leveraging the associative property of online softmax calculation as a reduction operation and employing stream-K style decomposition, the technique enables parallelization across all modes of the attention matrix, including the context length dimension. This allows for efficient distribution of computational workload across available GPU resources while ensuring equal total work allocation. The approach delivers significant speedup over existing methods, particularly for long context lengths, by maintaining near 100% GPU occupancy through optimal workload distribution and single-kernel execution.

Abstract: A computer-implemented method includes obtaining a training data set for multiple monitors for various services, which includes service properties and monitor metadata. The metadata for a given monitor defines resources utilized by a corresponding service. The method determines N feature vectors and a target resource class for each service based on the training data set. A machine learning model is trained in multiple training iterations using the training data set. In a given training iteration, N feature vectors of a selected service are provided to the machine learning model, which predicts a resource class of the selected service. A difference between the predicted resource class and the target resource class for the selected service is determined, based on which one or more parameters of the machine learning model can be updated. The trained machine learning model can be used to recommend a new monitor for a new service.

Abstract: Systems and methods are provided for generating and updating a dependency graph that is used in combination with textual information about incidents to improve incident-linking suggestions. Systems and methods are also provided for generating, training, and using a machine learning model configured to perform incident linking using both graph data and text data. Beneficially, these systems and methods align the graph data and text data in order to more efficiently and accurately leverage information from the multi-modal data.

Abstract: A confidence estimation tool uses a calibrated confidence mapping model to estimate confidence for a model-generated candidate root cause. The tool uses a generative artificial intelligence (“AI”) model to determine, based on a description of a current event, a candidate root cause of the current event. The tool determines a description-based confidence score using the description of the current event and descriptions of a set of relevant historical events in a target domain. The tool also determines a cause-based confidence score using the candidate root cause of the current event and root causes of the set of relevant historical events. Finally, the tool determines a final confidence score using the description-based and cause-based confidence scores. Even if the generative AI model is configured for general-domain applications, by referencing relevant historical events, the tool can accurately estimate confidence for a model-generated candidate root cause within the target domain.

Abstract: Generating and utilizing personalized predictive models are provided. When an electronic input is received, a generic predictive model is used to predict a user response to the input. After a prescribed period of time, an analysis is performed to determine the user's actual response to the input, as well as, the user's actual responses to other inputs of the same type. Training is performed on the generic predictive model to generate a new and personalized predictive model based on the user's actual responses to the analyzed inputs. The personalized predictive model is then utilized for predicting user response to future inputs of the same type. At a prescribed frequency, the generated personalized predictive model is updated by analyzing actual user responses to predictions provided by the personalized predictive model.

Abstract: Generating and utilizing personalized predictive models are provided. When an electronic input is received, a generic predictive model is used to predict a user response to the input. After a prescribed period of time, an analysis is performed to determine the user's actual response to the input, as well as, the user's actual responses to other inputs of the same type. Training is performed on the generic predictive model to generate a new and personalized predictive model based on the user's actual responses to the analyzed inputs. The personalized predictive model is then utilized for predicting user response to future inputs of the same type. At a prescribed frequency, the generated personalized predictive model is updated by analyzing actual user responses to predictions provided by the personalized predictive model.