Abstract: Embodiments of the invention present a system and method for identifying relationships between different types of data captured by a pen-based computing system, such as a smart pen. The pen-based computing system generates one or more sessions including different types of data that are associated with each other. In one embodiment, the pen-based computing system generates an index file including captured audio data and written data, where the written data is associated with a temporal location of the audio data corresponding to the time the written data was captured. For example, the pen-based computing system applies one or more heuristic processes to the received data to identify relationships between various types of the received data, used to associated different types of data with each other.

Abstract: A system and method for identifying temporal positions in audio data and accessing the identified temporal positions are disclosed. “Audio bookmarks” are created by using various types of input, such as accessing a printed control with a smart pen, providing a voice command to the smart pen or providing a written command to the smart pen to identify temporal positions within the audio data. Alternatively, one or more rules are applied to audio data by a pen-based computing system to identify temporal positions in the audio data. The audio bookmarks are associated with one or more visual, auditory or tactile indicators showing the location of the audio bookmarks in the audio data. When an indicator is accessed, a portion of the audio data is played beginning from the temporal position associated with the accessed indicator. Additional data, such as written data, may also be associated with an indicator.

Abstract: Control inputs are provided to an application executing on a mobile computing device by moving the mobile computing device in certain recognizable patterns. The control inputs may execute various functions in the application such as starting or stopping audio playback or navigating through a menu. A writing gesture made by a user on a writing surface using a smart pen device is digitally captured. This gesture may be, for example, a tap or a stroke of the smart pen device on the writing surface. A control on the writing surface is identified, where the control at least partially corresponds to a location of the writing gesture on the writing surface. A control input is determined based on the identified control and the writing gesture. Responsive to the control input, a command is executed in an application running on the smart pen device or an attached computing system.