Abstract: The techniques disclosed herein provide a system for automatically organizing content captures of user activity into collections based on a shared topic. Due to the significant portion of daily life that occurs via personal computing devices, there is increasing need for helpful user experiences to enhance productivity and engagement. Namely, those directed to assisting a user in managing and recalling their past activity. As such, the presented system retrieves content captures depicting a moment of interest. Accordingly, the system performs a visual analysis of each content capture to determine a semantic context and assign one or more topics. This is accomplished by utilizing a pair of machine learning models to identify and extract relevant text information. The extracted text information is then analyzed by a topic classification model to assign the one or more topics. The system further includes graphical user interfaces for viewing and managing collections of content captures.

Abstract: The techniques disclosed herein provide a system for automatically organizing content captures of user activity into collections based on a shared topic. Due to the significant portion of daily life that occurs via personal computing devices, there is increasing need for helpful user experiences to enhance productivity and engagement. Namely, those directed to assisting a user in managing and recalling their past activity. As such, the presented system retrieves content captures depicting a moment of interest. Accordingly, the system performs a visual analysis of each content capture to determine a semantic context and assign one or more topics. This is accomplished by utilizing a pair of machine learning models to identify and extract relevant text information. The extracted text information is then analyzed by a topic classification model to assign the one or more topics. The system further includes graphical user interfaces for viewing and managing collections of content captures.

Abstract: The techniques described herein provide systems for generating a record of user activity that enable downstream user experiences. Due to the significant portion of daily life that occurs via personal computing devices (e.g., laptops, personal computers, smartphones, tablets), service providers (e.g., operating system providers) may wish to enhance productivity and/or engagement through helpful user experiences. Moreover, such user experiences can be customized to a user's current context, preferences, and tendencies. Accordingly, the present system can, with the consent of the user, collect graphical captures recording a current state of a desktop environment at certain moments of interest. This is accomplished through a trigger mechanism utilizing operating system signals to intelligently collect graphical captures.

Abstract: The techniques disclosed herein provide a personalized interactive timeline of user activity graphical user interface (GUI) that enables a personalized search and retrieval user experience. This is accomplished by gathering a record of user activity such as graphical captures (e.g., screenshots) of a desktop environment. A graphical capture can define a software application that is currently in focus within the desktop environment. The graphical captures are then organized into subsets based on the software application that is in focus in each of the graphical captures. The graphical captures are then further organized into sessions which are delineated by substantially continuous user interaction with the corresponding software application. A graphical segment is then generated for each session and subsequently compiled into an interactive timeline. A user can scrub through the timeline to recall previous activity.