Abstract: The present invention facilitates transparent and cost effective capture, sharing, and re-use of knowledge in informal media such as audio, video, and individual sketching actions. Sketches, handwritings, verbal comments and the likes captured during a session are timestamped and, once completed, automatically indexed and published on a web server that allows for real time, streamed and synchronized replay thereof. All timestamps are converted to a common time base. To control the synchronous replay, multiple treads collaborate based on common base timestamps corresponding to the sketching actions and correlated audio/video stream(s). An enhanced interactive graphical user interface enables a network user to create and capture a session, e.g., annotate an imported image with sketches and verbal comments. The user or others can access the published session in real time and select a keyword or an individual sketch entity from which to begin replay. Transcribed audio stream can also be synchronously replayed.

Abstract: The present invention tightly integrates and significantly leverages media capture systems to realize a ubiquitous collaborative multi-user environment, referred to as TalkingPaper, taking full advantages of what each unique medium can offer and further enhancing their respective functions and utilities. The invention comprises a paper-pen based knowledge capture subsystem A and a knowledge processing subsystem B. A user uses a subsystem A compliant pen to sketch on a piece of subsystem A enabled paper. The sketching activity, speech and gesture are recorded by the pen, which sends the captured data to a multi-threaded application server of subsystem B for further processing. The server converts and indexes the data to associate and synchronize each line stroke with corresponding audio time frames and enable the content interaction. Thus, users can easily find, select, and replay a session with synchronized speech, text, and video to understand the rationale behind certain ideas or decisions.

Abstract: The invention provides a cross-media software environment that enables seamless transformation of analog activities, such as gesture language, verbal discourse, and sketching, into integrated digital video-audio-sketching (DiVAS) for real-time knowledge capture, and that supports knowledge reuse through contextual content understanding.

Abstract: The present invention facilitates transparent and cost effective capture, sharing, and re-use of knowledge in informal media such as audio, video, and individual sketching actions. Sketches, handwritings, verbal comments and the likes captured during a session are timestamped and, once completed, automatically indexed and published on a web server that allows for real time, streamed and synchronized replay thereof. All timestamps are converted to a common time base. To control the synchronous replay, multiple treads collaborate based on common base timestamps corresponding to the sketching actions and correlated audio/video stream(s). An enhanced interactive graphical user interface enables a network user to create and capture a session, e.g., annotate an imported image with sketches and verbal comments. The user or others can access the published session in real time and select a keyword or an individual sketch entity from which to begin replay. Transcribed audio stream can also be synchronously replayed.

Abstract: For a number of users is a system provided to create, edit, replay and view documents of free hand drawn sketches. The system captures the creation process together with verbal and/or visual information provided by each user and automatically correlates them for a later synchronized replay. The system provides a number of tools and features, mainly to: combine the sketching activity with existing images, to selectively retrieve media information correlated to individual sketch entities and to quasi simultaneously collaborate at a common document. The system architecture can by adjusted to various parameters in the communication infrastructure. The system may be implemented in any software program, a web based service, a web browser, an operating system for computers and/or communication devices.