Abstract: Methods and apparatus for an interaction system specifying cross-device interaction are provided. The interaction system provides application program interfaces (APIs) and a scripting environment that allow development of scripts that combine user input and sensing events and distribute output and across devices to create a range of rich cross-device behaviors with minimal development. The interaction system includes an integrated environment for developers to author and test cross-device behaviors. When a script is ready, the script can be deployed on a network of mobile and wearable computing devices using the interaction system's runtime environment. An evaluation of the interaction system with twelve participants revealed that the interaction system significantly reduced developer effort for creating and iterating on cross-device behaviors. The interaction system allowed developers to focus on target interaction behaviors and high-level device capabilities, rather than low-level specifications.

Abstract: Methods and apparatus for an interaction system specifying cross-device interaction are provided. The interaction system provides application program interfaces (APIs) and a scripting environment that allow development of scripts that combine user input and sensing events and distribute output and across devices to create a range of rich cross-device behaviors with minimal development. The interaction system includes an integrated environment for developers to author and test cross-device behaviors. When a script is ready, the script can be deployed on a network of mobile and wearable computing devices using the interaction system's runtime environment. An interaction system script can. An evaluation of the interaction system with twelve participants revealed that the interaction system significantly reduced developer effort for creating and iterating on cross-device behaviors.

Abstract: A sequence of low-level user input events that takes place during a demonstration is converted into a sequence of high-level events which is used to identify portions of a screencast video of the demonstration as event portions. A presenter rehearses, edits, and re-performs the demonstration during a live presentation by playing back an augmented version of the video that is generated on-the-fly and during at least one point in time includes visualizations of the current and next high-level events that are automatically overlaid on top of the video at the screen locations where these events takes place. A graphical user interface that includes a video sector and a timeline sector is displayed to the presenter. The augmented version of the video is played back within the video sector. An event timeline that includes an overall timeline showing each of the portions of the video is displayed within the timeline sector.