Abstract: A framework is provided for obtaining window information. The window information can be applied to different assignment models to assign windows to different groups. A group may correspond to a task being performed by a user. The window information can be semantic or temporal information captured as window events and properties of windows whose events are captured. Temporal information can be information about switches between windows. Semantic information can be window titles. Temporal information, semantic information, or both, can be used to assign windows to groups.

Abstract: Architecture for implementing a perceptual user interface. The architecture comprises alternative modalities for controlling computer application programs and manipulating on-screen objects through hand gestures or a combination of hand gestures and verbal commands. The perceptual user interface system includes a tracking component that detects object characteristics of at least one of a plurality of objects within a scene, and tracks the respective object. Detection of object characteristics is based at least in part upon image comparison of a plurality of images relative to a course mapping of the images. A seeding component iteratively seeds the tracking component with object hypotheses based upon the presence of the object characteristics and the image comparison. A filtering component selectively removes the tracked object from the object hypotheses and/or at least one object hypothesis from the set of object hypotheses based upon predetermined removal criteria.

Abstract: A system that can integrate activities across machines and/or devices is disclosed. The innovation can be used in an “asynchronous” manner that enables a user to move or direct a set of activities and/or tasks within an activity from one device to another. Further, the system can facilitate adapting user interface factors with respect to a particular device such that a user can “synchronously” utilize all devices for the given activities and/or tasks. In other words, the “synchronous” scenario enables a user to share activity information between devices when simultaneously utilizing multiple devices in accordance with a particular activity.

Abstract: Architecture for implementing a perceptual user interface. The architecture comprises alternative modalities for controlling computer application programs and manipulating on-screen objects through hand gestures or a combination of hand gestures and verbal commands. The perceptual user interface system includes a tracking component that detects object characteristics of at least one of a plurality of objects within a scene, and tracks the respective object. Detection of object characteristics is based at least in part upon image comparison of a plurality of images relative to a course mapping of the images. A seeding component iteratively seeds the tracking component with object hypotheses based upon the presence of the object characteristics and the image comparison. A filtering component selectively removes the tracked object from the object hypotheses and/or at least one object hypothesis from the set of object hypotheses based upon predetermined removal criteria.

Abstract: The claimed subject matter provides a system and/or a method that facilitates dynamically providing a question to ask a medical professional during an appointment. An interface can receive a portion of medical data. A counselor component can generate a question based on the portion of medical data, wherein the question is generated to elicit an answer from a medical professional during an appointment. Moreover, the counselor component can dynamically generate a second question directed toward the medical professional based upon at least one of the answer or a value of information (VOI) computation.

Abstract: The present invention relates to a system and methodology to facilitate machine learning and predictive capabilities in a processing environment. In one aspect of the present invention, a Mutual Information Model is provided to facilitate predictive state determinations in accordance with signal or data analysis, and to mitigate classification error. The model parameters are computed by maximizing a convex combination of the mutual information between hidden states and the observations and the joint likelihood of states and observations in training data. Once the model parameters have been learned, new data can be accurately classified.

Abstract: Space-time trails are annotated automatically and/or manually with meaningful information. Mechanisms are provided for receipt of implicit and/or passive data (e.g., provided by sensors) as well as explicit and/or active data (e.g., provided by users). Extraction techniques are provided to either or both types of data to produce useful information to annotate or otherwise interpret data associated with a space-time trail. Persisted annotated trails can subsequently be employed to glean valuable information. Furthermore, goal-directed recommendations can be generated with respect to annotated trails, among other things.

Abstract: A framework is provided for obtaining window information. The window information can be applied to different assignment models to assign windows to different groups. A group may correspond to a task being performed by a user. The window information can be semantic or temporal information captured as window events and properties of windows whose events are captured. Temporal information can be information about switches between windows. Semantic information can be window titles. Temporal information, semantic information, or both, can be used to assign windows to groups.

Abstract: A system that can automatically narrow the search space or recognition scope within an activity-centric environment based upon a current activity or set of activities is provided. In addition, the activity and context data can also be used to rank the results of the recognition or search activity. In accordance with the domain scoping, natural language processing (NLP) as well as other types of conversion and recognition systems can dynamically adjust to the scope of the activity or group of activities thereby increasing the recognition systems accuracy and usefulness. In operation, a user context, activity context, environment context and/or device profile can be employed to effectuate the scoping. As well, the system can combine context with extrinsic data, including but not limited to, calendar, profile, historical activity data, etc. in order to define the parameters for an appropriate scoping.

Abstract: User introspection is provided to help users achieve goals (e.g., work goals, personal goals) and become more effective at performing tasks. Activities can be monitored, logged, and compared to a set of goals or tasks. Feedback can be provided if a user has deviated from a specified goal or task, in the form of a recommendation as to a next action, or based upon user-defined criteria. Feedback can also be provided based on a multitude of parameters that can include prioritized goal definition and refinement, task avoidance behavior detection, baseline activity analysis (e.g. historical data), base-line capacity (personal and/or comparison to others), task reacquisition, etc.

Abstract: A unique monitoring system and method is provided that involves monitoring user activity in order to facilitate managing and optimizing the utilization of various system resources. In particular, the system can monitor user activity, detect when users need assistance with their specific activities, and identify at least one other user that can assist them. Assistance can be in the form of answering questions, providing guidance to the user as the user completes the activity, or completing the activity such as in the case of taking on an assigned activity. In addition, the system can aggregate activity data across users and/or devices. As a result, problems with activity templates or activities themselves can be more readily identified, user performance can be readily compared, and users can communicate and exchange information regarding similar activity experiences. Furthermore, synchronicity and time-sensitive scheduling of activities between users can be facilitated and improved overall.

Abstract: A unique system and method is provided that facilitates managing an activity centric environment via a master profile (which includes user, group, and device profiles). The master profile follows or stays with the user and can be applied universally across devices and activities (activity templates). When profile data does not currently exist (e.g., a new activity or a new device), portions of the existing profile data can be applied to such new activities or device as appropriate. Thus, current profile data for existing or known user interactions and devices can be inferentially extended to new user interactions and devices. When conflicts arise between applicable profile data, they can be solved by applying the profile data in accordance with their priority. User intervention can be requested whereby the system can adapt previous user-based resolutions to future conflicts. Profile data can also be scaled according to the context of the user-device-activity interaction.

Abstract: A system that enables generation and/or modification of an activity template is disclosed. The activity template describes overall actions and resources associated with an activity. The system can automatically convert activity logs into recipes thereby reducing effort in activity reproduction. Self-tuning via machine learning-based algorithms can be employed thereby dynamically altering the template in accordance with on-going or historical user action.

Abstract: A system that can log user actions associated with an activity is disclosed. For example, the system can maintain a log of user keystrokes, files accessed, files opened, files created, websites visited, communication events (e.g., phone calls, instant messaging communications), etc. Additionally, the system can log extrinsic data (e.g., context data) associated with the user actions. As well, these logged actions can be aggregated, synchronized and/or shared between multiple users and/or devices.

Abstract: A system that can enable the atomization of application functionality in connection with an activity-centric system is provided. The system can be utilized as a programmatic tool that decomposes an application's constituent functionality into atoms thereafter monitoring and aggregating atoms with respect to a particular activity. In doing so, the functionality of the system can be scaled based upon complexity and needs of the activity. Additionally, the system can be employed to monetize the atoms or activity capabilities based upon respective use.

Abstract: A system that can identify, create, update and/or process a workflow based upon a current, past or future activity is disclosed. A ‘workflow’ can be defined as an activity flow that includes interaction with, or assignment of work to, people, devices, or services by a single individual or a group of individuals. Once a workflow is determined in accordance with the innovation, the system can inform other users or groups that are performing, or intend to perform, a similar or like activity. In establishing the workflow, the innovation can operate in an ad hoc or authored manner. As well, the system can employ a combination of either ad hoc or authored mechanisms in establishment of the workflow.

Abstract: The innovation enables “total system” experiences for activities and activity-specialized experiences for applications and gadgets that allow them to align more closely with the user, his work, and his goals. In particular, the system provides for dynamically changing the user interface (UI) of the system level shell (“desktop”), of applications, and of standalone UI parts (“gadgets”), based upon a current (or future) activity of the user and other context data. The system can consider context data that includes extended activity data, information about the user's state, and information about the current environment. Preprogrammed and/or inferred rules can be used to decide how to adapt the UI based upon the activity. These rules can include, user rules, group rules, and device rules. Additionally, activities and applications can also participate in the decision of how to adapt the UI.

Abstract: A system that can integrate activities across machines and/or devices is disclosed. The innovation can be used in an “asynchronous” manner that enables a user to move or direct a set of activities and/or tasks within an activity from one device to another. Further, the system can facilitate adapting user interface factors with respect to a particular device such that a user can “synchronously” utilize all devices for the given activities and/or tasks. In other words, the “synchronous” scenario enables a user to share activity information between devices when simultaneously utilizing multiple devices in accordance with a particular activity.

Abstract: The present invention relates to a system and methodology providing layered probabilistic representations for sensing, learning, and inference from multiple sensory streams at multiple levels of temporal granularity and abstraction. The methods facilitate robustness to subtle changes in environment and enable model adaptation with minimal retraining. An architecture of Layered Hidden Markov Models (LHMMs) can be employed having parameters learned from stream data and at different periods of time, wherein inferences can be determined relating to context and activity from perceptual signals.

Abstract: The present invention relates to a system and methodology to facilitate machine learning and predictive capabilities in a processing environment. In one aspect of the present invention, a Mutual Information Model is provided to facilitate predictive state determinations in accordance with signal or data analysis, and to mitigate classification error. The model parameters are computed by maximizing a convex combination of the mutual information between hidden states and the observations and the joint likelihood of states and observations in training data. Once the model parameters have been learned, new data can be accurately classified.