Abstract: A Video Highlight Summarization System (VHSS) is described for generating a personalized video highlight summary from a video source (e.g., a sport match) based on a user's query. In some embodiments, the VHSS may perform multimodal data analysis. The multimodal data may include information from video, audio, and text from images associated with the video and from user's query. A user may provide a query specifying the user's preferences (e.g., events of interest) and criteria (e.g., summary duration). In some embodiments, encoded embeddings based on the video, audio, text, and the user query may be aligned to enhance similarity search result. A subset (e.g., highlights) of the video clips is selected from the video source by maximizing the summation of scores of highlight clips to best fit the user's preferences while meeting the user's criteria with diverse clips.

Abstract: A Video Highlight Summarization System (VHSS) is described for generating a personalized video highlight summary from a video source (e.g., a sport match) based on a user's query. In some embodiments, the VHSS may perform multimodal data analysis. The multimodal data may include information from video, audio, and text from images associated with the video and from user's query. A user may provide a query specifying the user's preferences (e.g., events of interest) and criteria (e.g., summary duration). In some embodiments, encoded embeddings based on the video, audio, text, and the user query may be aligned to enhance similarity search result. A subset (e.g., highlights) of the video clips is selected from the video source by maximizing the summation of scores of highlight clips to best fit the user's preferences while meeting the user's criteria with diverse clips.

Abstract: A system and method for extracting structured key information from diverse document types using large multimodal models (LMMs) is disclosed. The invention employs a zero-shot analysis to identify candidate keys within an input document, then selects a document schema from a document schema database based on the identified keys. The LMM is prompted with the selected document schema to generate structured key-value pairs, with field constraints enforced by the document schema. Relationships among extracted keys are mapped to a graph representation, enabling robust handling of complex document layouts. The system supports nested structures, tabular data, and alias definitions for fields, and can update document schemas based on ground truth feedback. The resulting structured output is provided in a machine-readable format, enabling reliable and scalable document understanding across varied domains such as invoices, health cards, and driving licenses.

Abstract: A memorability prediction system (MPS) is described for predicting image memorability of an input image while considering the contribution of sub-images and pixels of the input image. In some embodiments, the input image may be partitioned (also referred to as diced) randomly into multiple sub-images. Various techniques for dicing the input images are described. In certain embodiments, the input image may be partitioned randomly into one or more segments in both the x-dimension (e.g., width) and the y-dimension (e.g., height). The segments in both dimensions may be combined to generate sub-images in different sizes. In some embodiments, objects in the input image may be identified, and the sub-images may be re-arranged or re-oriented according to an arrangement configuration to achieve higher probability of partitioning or preserving one or more objects. In some embodiments, a pre-defined partition may be used for one or more regions of the input image.

Abstract: A memorability prediction system (MPS) is described for predicting the image memorability of an input image while considering the contribution of sub-images and pixels of the input image. In some embodiments, a visual transformer-based memorability prediction network in the MPS may include three machine learning (ML) models responsible for processing different parts of the input image, namely, the whole input image (referred to as the main image), partitioned images (referred to as diced images or sub-images) of the input image, and pixels of the input image. In further embodiments, a relationship between the main image and one or more sub-images may be identified and passed between two ML models. In some embodiments, the three ML models may generate intermediate information to be combined to result in a final memorability score of the input image.

Abstract: Job execution environment control techniques are described to manage policy selection and implementation to control use of job executors by a computing device, automatically and without user intervention. These techniques are usable to select a policy from a plurality of policies that is then used to control lifecycles of job executors of a job execution environment of a computing device. Further, these techniques are usable to respond dynamically to change the selected policy during runtime of the application in response to changes in the job execution environment.

Abstract: A job scheduling system determines a rate at which a user is providing user inputs to a user interface of a computing device. A set of jobs that is to be performed to display or otherwise present a current view of the user interface is identified in response to a user input. This set of jobs is modified by excluding from the set of jobs at least one job that is not estimated to run prior to the next user input. The user interface is displayed or otherwise presented as the modified set of jobs is performed.

Abstract: Techniques are disclosed for improving transfer speed for a plurality of files (e.g., image files) by using a Markov decision process to determine an optimal number of parallel instances of transfer stages and optimal file batch sizes for each instance. The transfer (e.g., import or export) operation involves different stages that are each optimized using the algorithm. The stages include a file fetch operation, a file processing operation, and a database update operation. Each of the stages may have multiple parallel instances to process many files at the same time. The Markov decision process uses a reward structure to determine the optimal number of parallel instances for each stage and the number of files operated on at each instance at any given moment in time. The process is dynamic and adaptable to any system environment since it does not rely on any particular hardware or operating system configuration.

Abstract: A context-based recommendation system for feature search automatically identifies features of a feature-rich system (e.g., an application) based on the program code of the feature-rich system and additional data corresponding to the feature-rich system. A code workflow graph describing workflows in the program code is generated. Various data corresponding to the feature-rich system, such as help data, analytics data, social media data, and so forth is obtained. The code workflow graph and the data are analyzed to identify sentences in the workflow. These sentences are used to a train machine learning system to generate one or more recommendations. In response to a user query, the machine learning system generates and outputs as recommendations workflows identified based on the user query.

Abstract: A job scheduling system determines a rate at which a user is providing user inputs to a user interface of a computing device. A set of jobs that is to be performed to display or otherwise present a current view of the user interface is identified in response to a user input. This set of jobs is modified by excluding from the set of jobs at least one job that is not estimated to run prior to the next user input. The user interface is displayed or otherwise presented as the modified set of jobs is performed.

Abstract: A job scheduling system determines a rate at which a user is providing user inputs to a user interface of a computing device. A set of jobs that is to be performed to display or otherwise present a current view of the user interface is identified in response to a user input. This set of jobs is modified by excluding from the set of jobs at least one job that is not estimated to run prior to the next user input. The user interface is displayed or otherwise presented as the modified set of jobs is performed.

Abstract: A resource management system of an application takes various actions to improve or maintain the health of the application (e.g., keep the application from becoming sluggish). The resource management system maintains a reinforcement learning model indicating which actions the resource management system is to take for various different states of the application. The resource management system performs multiple iterations of a process of identifying a current state of the application, determining an action to take to manage resources for the application, and taking the determined action. In each iteration, the resource management system determines the result of the action taken in the previous iteration and updates the reinforcement learning model so that the reinforcement learning model learns which actions improve the health of the application and which actions do not improve the health of the application.

Abstract: Job execution environment control techniques are described to manage policy selection and implementation to control use of job executors by a computing device, automatically and without user intervention. These techniques are usable to select a policy from a plurality of policies that is then used to control lifecycles of job executors of a job execution environment of a computing device. Further, these techniques are usable to respond dynamically to change the selected policy during runtime of the application in response to changes in the job execution environment.

Abstract: In some embodiments, a key smoothener and predictor module of a software application executing on a computing device receives a sequence of key events from an input device of the computing device and through a user interface of the software application. The key smoothener and predictor module stores the sequence of key events in a key event queue and predicts the total number of key events for processing in a current processing cycle of the application based on the sequence of key events. A processing component of the software application processes an aggregated key event that indicates multiple keypresses. The number of the multiple keypresses is the same as the predicted total number of key events for the current processing cycle. The software application further causes the user interface of the software application to be updated based on processing the aggregated key event.

Abstract: In some embodiments, a key smoothener and predictor module of a software application executing on a computing device receives a sequence of key events from an input device of the computing device and through a user interface of the software application. The key smoothener and predictor module stores the sequence of key events in a key event queue and predicts the total number of key events for processing in a current processing cycle of the application based on the sequence of key events. A processing component of the software application processes an aggregated key event that indicates multiple keypresses. The number of the multiple keypresses is the same as the predicted total number of key events for the current processing cycle. The software application further causes the user interface of the software application to be updated based on processing the aggregated key event.

Abstract: A job scheduling system determines a rate at which a user is providing user inputs to a user interface of a computing device. A set of jobs that is to be performed to display or otherwise present a current view of the user interface is identified in response to a user input. This set of jobs is modified by excluding from the set of jobs at least one job that is not estimated to run prior to the next user input. The user interface is displayed or otherwise presented as the modified set of jobs is performed.

Abstract: Techniques are disclosed for improving transfer speed for a plurality of files (e.g., image files) by using a Markov decision process to determine an optimal number of parallel instances of transfer stages and optimal file batch sizes for each instance. The transfer (e.g., import or export) operation involves different stages that are each optimized using the algorithm. The stages include a file fetch operation, a file processing operation, and a database update operation. Each of the stages may have multiple parallel instances to process many files at the same time. The Markov decision process uses a reward structure to determine the optimal number of parallel instances for each stage and the number of files operated on at each instance at any given moment in time. The process is dynamic and adaptable to any system environment since it does not rely on any particular hardware or operating system configuration.

Abstract: A context-based recommendation system for feature search automatically identifies features of a feature-rich system (e.g., an application) based on the program code of the feature-rich system and additional data corresponding to the feature-rich system. A code workflow graph describing workflows in the program code is generated. Various data corresponding to the feature-rich system, such as help data, analytics data, social media data, and so forth is obtained. The code workflow graph and the data are analyzed to identify sentences in the workflow. These sentences are used to a train machine learning system to generate one or more recommendations. In response to a user query, the machine learning system generates and outputs as recommendations workflows identified based on the user query.

Abstract: A resource management system of an application takes various actions to improve or maintain the health of the application (e.g., keep the application from becoming sluggish). The resource management system maintains a reinforcement learning model indicating which actions the resource management system is to take for various different states of the application. The resource management system performs multiple iterations of a process of identifying a current state of the application, determining an action to take to manage resources for the application, and taking the determined action. In each iteration, the resource management system determines the result of the action taken in the previous iteration and updates the reinforcement learning model so that the reinforcement learning model learns which actions improve the health of the application and which actions do not improve the health of the application.

Abstract: Photo-editing application recommendations are described. A language modeling system generates a photo-editing language model based on application usage data collected from existing users of a photo-editing application. The language modeling system generates the model by applying natural language processing to words that are selected to represent photo-editing actions described by the application usage data. The natural language processing involves partitioning contiguous sequences of the words into sentences of the modeled photo-editing language and partitioning contiguous sequences of the sentences into paragraphs of the modeled photo-editing language. The language modeling system deploys the photo-editing language model for incorporation with the photo-editing application. The photo-editing application uses the model to determine a current workflow in real-time as input is received to edit digital photographs, and recommends tools for carrying out the current workflow.