Abstract: The present invention relates to a system, method and medium for receiving and acting upon user input. In one embodiment, the user may only have access to a limited input device, like a stylus. Using the present invention, a user is provided with intuitive responses from the system based on inputs from the limited input device.

Abstract: A digital ink environment is provided for non-native ink application programs to create independent ink services for sharing between different application programs and transmission. The digital ink environment is provided for interacting with embedded or linked ink words or other ink data in a compound ink document. The user is provided with the freedom to input digital ink so they can write anywhere in the ink environment, in any orientation, and using any desired stroke size provided by the ink environment. The ink environment is provided in an activated state for in-situ user interaction and manipulation in a computing environment, for example a pen-based environment.

Abstract: A universal file format for providing digital rich ink data documents. Digital rich ink data is hidden among the ink image data otherwise stored in a widely-accepted image file format. For example, depending on the type of file, data can be concealed in the least significant bit or bits of pixel information, or stored in-different color indexes of a palette that result in the same color output. In other file types, digital rich ink data may be appended after the end of file marker, or placed in a comment area. When such a file is accessed, computing devices capable of handling rich ink data recognize and extract the rich ink data from the file, while devices incapable of recognizing the format are still able to display the ordinary ink data. Two file formats for describing the arrangement of rich ink data are also provided.

Abstract: In a computing device that receives handwritten data, a method and system that facilitates the selection of alternates for a hand written word. A mechanism uses rules to select a word for the user based on the operating mode (state) of the system. In a cursor operating mode, a word is selected and the alternates are provided for the word based on the position of on screen cursor relative to the word (within or adjacent the word). If not on a word having alternates, the selection can move to another word that has alternates. If multiple words are selected, (selection mode), the system selects the first word of those selected that has alternates. Menu options to go to the next or previous recognized word may also be placed on the menu for easy navigation among words, and a user can automatically walk through multiple word sets to receive alternates for each word, one-word at a time.

Abstract: A method and system for automatically determining when an application should switch from a writing mode to a text entry mode, and vice-versa. When a soft input panel (SIP) or input method (IM) is selected, the application is notified and enters a text entry mode. Inputting via the IM while in this mode causes a character to be sent to the application, while pen events in the application window enable text editing and the selection of text and ink data in the application widow. When the SIP is deselected, the application is notified and enters a pen (writing) mode, whereby the pen events are interpreted as digital ink strokes instead of characters. Sub-modes of the pen mode are also available, via which the pen events are used to manipulate existing ink in the application window. Visual feedback may be provided to remind the user of the current mode and/or sub-mode.

Abstract: A logical separation between pages, such as an implicit page break, is introduced to separate text entered during one handwriting session from text entered during another handwriting session. If the user leaves more than a threshold amount of blank space at the bottom of the page immediately preceding the new page, then an implicit page break may be inserted at the beginning of the new page. The amount of blank space left at the end of the preceding page may be combined with other criteria to determine whether to insert an implicit page break. The amount of time elapsed since ink has been captured on the previous page is another factor that may be used by itself or combined with other factors to determine whether to insert an implicit page break into the new page. A change in context, such as a different date or different recognized subject matter labels, is also a factor that may be considered in determining whether to insert an implicit page break.

Abstract: A logical separation between pages, such as an implicit page break, is introduced to separate text entered during one handwriting session from text entered during another handwriting session. If the user leaves more than a threshold amount of blank space at the bottom of the page immediately preceding the new page, then an implicit page break may be inserted at the beginning of the new page. The amount of blank space left at the end of the preceding page may be combined with other criteria to determine whether to insert an implicit page break. The amount of time elapsed since ink has been captured on the previous page is another factor that may be used by itself or combined with other factors to determine whether to insert an implicit page break into the new page. A change in context, such as a different date or different recognized subject matter labels, is also a factor that may be considered in determining whether to insert an implicit page break.

Abstract: A computer system and method relating to editing an electronic document having both text and drawings is disclosed. Drawings may be rescaled and/or repositioned responsive to a change in line height or font size of text. The resealing may be done in proportion to the change in line height or font size, and the repositioning may be performed to avoid overlapping of the rescaled drawings. A document may be divided into a plurality of adjacent regions, and drawings within the document may each have an offset value representing a distance between the drawing and a reference axis of one of the regions.

Abstract: In a computing device that receives handwritten data, a method and system that maintains an association between alternates for a given ink word, regardless of the handwritten or text state of the word, and regardless of the position of the word as it may be edited in a document. Handwritten data is maintained in an ink word data structure, and once the word is recognized and an alternate is selected for it, the first character of the word remains as an ink word (in a text buffer) pointing to the data structure, with a flag set in the data structure indicating that the word is now recognized as text. In this state, the first character is displayed to the user as a recognized text letter instead of as the handwritten word. The other characters that make up the recognized word are inserted as text into the text buffer. Any alternates returned by the recognizer are thus stored with the ink word data structure displayed as this first character of a recognized word, which also maintains the ink data, e.g.

Abstract: A method of embedding voice data in a computing system includes detecting a record event and detecting if a software application currently running on the computing system is voice-aware. The method also includes embedding the voice data within associated data in the software application, if the software application is voice-aware. If the software application is not voice-aware, the method also includes triggering a voice note application to record and store the voice data. A method in a computing system for modifying a user interface displayed on a display device includes receiving an indication from the computing device to modify the user interface. The method further includes displaying an identification block, a record toolbar, a note pad, and a note tab.

Abstract: A method of embedding voice data in a computing system includes detecting a record event and detecting if a software application currently running on the computing system is voice-aware. The method also includes embedding the voice data within associated data in the software application, if the software application is voice-aware. If the software application is not voice-aware, the method also includes triggering a voice note application to record and store the voice data. A method in a computing system for modifying a user interface displayed on a display device includes receiving an indication from the computing device to modify the user interface. The method further includes displaying an identification block, a record toolbar, a note pad, and a note tab.

Abstract: A user interface allows a user to input handwritten, key-press, and spoken text in a seamless, synchronized manner. A text input panel accepts soft keyboard presses and handwritten words, characters, and gestures. A text recognizer/synchronizer integrates textual input from various sources while recognizing and preserving the order in which a user entered text via the soft keyboard, via handwriting, and/or by speaking. Synchronized text may be displayed in a stage area of the text input panel before being passed to an operating system message router and/or an application program. While in handwriting recognition mode, various permutations and combinations of a word recognition area, a character recognition area, and a keybar/keypad may optionally be displayed.

Abstract: In a computing device that receives handwritten data, a method and system for finding matches for recognized handwritten words, by comparing a given search word (and possibly its alternates) with the words in a document, including recognized ink words and any possible alternates for those recognized words as returned by a recognizer. One described test looks for an exact match between an entered search word (and possibly its alternates) and the recognized words and their alternates stored in a handwritten document. Other tests are possible because of the use of alternates, which also may be returned with a probability ranking. For example, one scheme looks for a percentage of matching characters, with a user-determined threshold percentage. Other variations include giving different weight to certain characters, and/or factoring in the relative number of syllables and/or the relative lengths of the words.

Abstract: Digitized handwriting is captured and provided in real-time to a display in the form of polylines. The polylines are then converted to a parametric representation, thereby filtering out noise and distortion effects attributable to the digitization process. To further refine quality, the smoothed, digitized handwriting is further subjected to edge-smoothing processing to mitigate the effects of relatively low-resolution displays. In this manner, the present invention improves the appearance of digitized handwriting in comparison to prior art techniques.

Abstract: In a computing device that receives handwritten data, a method and data structure that enables extended data to be added to an existing ink word data structure without compromising backwards-compatibility. A flag in the header data structure is indicates to new ink processing programs the presence or absence of the extended data, and the size information maintained in the header is adjusted ensure that earlier versions of ink programs do not lose the extended data. The extended data is then added by including it in a copy of the existing ink word data structure, along with a tail structure that includes information describing the extended data and the tail structure to the new ink code, e.g., version and offset information. The tail structure can be used to locate a list of alternate word choices for an ink word that are maintained within the extended data.

Abstract: Voice recording is initiated immediately upon pressing a record button on a digital computing device such as a palm computer. To enable an immediate response to the button being pressed without a delay that would be normally be incurred to load a user interface and a recording application, a stub program is continuously executed on the device, even when a main user interface program is not running. When the record button is pressed, the stub program immediately begins recording a digital signal corresponding to the sound waves incident on a microphone of the device in a compressed format. If the record button is depressed for more than a second, the recorded digital signal is stored in a file so that it can subsequently be played back, and if the record button was pressed for less than one second, the file is not retained. Sound recording can be initiated in an “eyes free” mode, e.g., while using the device under low ambient light conditions, or while engaged in another activity.

Abstract: In a computing device that receives handwritten data, a method and system that corrects for parser segmentation errors by sending an entire line of ink to a recognizer, and then comparing, on a word-by-word basis, the initial segmentation guesses of the parser with the more-thoroughly recognized segmentation results of the handwriting recognition engine. In the correction process, the ink words are efficiently adjusted with relatively little data manipulation. As the recognizer is fed a series of strokes on a line, the recognizer returns segmentation information. For ink word breaks that are the same for any given set of data, the existing ink word is unchanged. For ink words that are recognized differently relative to their initial segmentation, one or more new ink words are created and the handwriting (including stroke) data of the parser's ink word is manipulated to create a new ink processor word (or words) to match the recognizer output.

Abstract: Methods for rendering electronic ink with highlighting attributes include: (a) receiving data representing at least one electronic ink stroke including a highlighting attribute; (b) rendering a first highlighting stroke in a first dimension and in a first highlighting color and/or pattern; and (c) rendering an ink color stroke in an ink color and/or pattern and in a second dimension that is smaller than the first dimension to produce a composite stroke that has a highlighted appearance. When the highlighted stroke is selected, this stroke may be rendered with a third, inner layer that is narrower than the ink colored layer, in a highlight color and/or pattern. This invention also relates to systems and computer-readable media for performing these methods, and to data structures that include a highlight attribute with electronic ink data.

Abstract: An adaptive pen mode selection is included within a pen input based computing system. The adaptive pen mode selection automatically switches the operating mode from an input data mode to the data item selection mode when a pen stroke is made over an existing data item to permit it to be selected for editing, deletion and replacement. The adaptive pen mode selection provides an improved user interface returns the computer to the user specified operating mode when the selection operation has been completed to allow the user to continue entering data in the user specified operating mode. With adaptive pen mode selection, a user of a pen input based computing system does not need to change the operating mode of the computer to an data item selection mode using the control inputs in order to select one or more data items for editing.

Abstract: A method and system are provided for extending the functionality of application buttons on a limited resource computing device. Alternative application functions are launched based on the length of time an application button is pressed. A default function for an application is launched if the button is pressed for a short, i.e., normal, period of time. An alternative function of the application is launched if the button is pressed for a long, (e.g., at least one second), period of time. Still another function can be launched if the application button is pressed multiple times within a short period of time, e.g., double click.