Abstract: A method for enhancing a textual image for undergoing optical character recognition begins by receiving an image that includes native lines of text. A background line profile is determined which represents an average background intensity along the native lines in the image. Likewise, a foreground line profile is determined which represents an average foreground background intensity along the native lines in the image. The pixels in the image are assigned to either a background or foreground portion of the image based at least in part on the background line profile and the foreground line profile. The intensity of the pixels designated to the background portion of the image is adjusted to a maximum brightness so as to represent a portion of the image that does not include text.

Abstract: An optical character recognition process characterizes text lines in a textual image by their base-line, mean-line and x-height. The base-line for at least one text line in the image is determined by finding a parametric curve that maximizes a first fitness function that depends on the values of pixels through which the parametric curve passes and pixels below the parametric curve. The base-line corresponds to the parametric curve for which the first fitness function is maximized. The first fitness function is designed so that it increases with increasing lightless or brightness of pixels immediately below the parametric curve while also increasing with decreasing lightness of pixels through which the parametric curve passes. The mean-line is determined by incrementally shifting the base-line upward by predetermined amounts (e.g., a single pixel) until a second fitness function for the shifted base-line is maximized. The second fitness function is essentially the inverse of the first fitness function.

Abstract: In embodiments consistent with the subject matter of this disclosure, a user may input strokes as digital ink to a processing device. The processing device may partition the input strokes into multiple regions of strokes. A first recognizer and a second recognizer may score grammar objects included in regions and represented by chart entries. The scores may be converted to a converted score, which may have at least a near standard normal distribution. The processing device may present a recognition result based on highest converted scores according to a recurrence formula. The processing device may receive a correction hint with respect to misrecognized strokes and may add a penalty score with respect to chart entries representing grammar objects breaking the correction hint. Incremental recognition may be performed when a pause is detected during inputting of strokes.

Abstract: Embodiments of camera-based scanning are described. In various embodiments, scanned documents can be created using images captured by a camera associated with a device. An image captured by the camera is processed to identify portions within the image that correspond to rectangular objects such as paper, business cards, whiteboards, screens, and so forth. One or more of these portions can be selected for scanning automatically based on a scoring scheme and/or semi-automatically with the aid of input from a user. One or more scanned documents are created from the selected portions by un-warping the selected portions to remove effects of perspective (e.g., rectify the portions to rectangles) and applying various image enhancements to improve appearance.

Abstract: A lasso select tool that allows a user to draw a flexible line of selection ink in order to select one or more graphical objects for manipulation. As the user draws this selection line, the lasso select tool continuously redraws and displays a connection line connecting one end of the selection line with the opposite end of the selection line. This connection line thus graphically demonstrates to the user the area that is presently encompassed by the resulting lasso enclosure. Further, the lasso select tool will change the appearance of any object selected with the lasso enclosure. The lasso select tool also selects objects that are only substantially encompassed by the lasso enclosure. For example, the lasso select tool may select ink objects that are only 60% or more encompassed by the lasso enclosure.

Abstract: An improved approach to highlighting a graphical object in a graphical user interface is disclosed. A body of a highlighted graphical object may, upon selection or highlighting, be changed to a transparent color, and a graphical outline, or “halo,” is added surrounding the original graphical object's body. The halo may be in the same color as the graphical object originally was, preserving any meaning attributed to the color of the graphical object and simplifying the user's ability to identify the selected object(s), as well as the ability to select the highlighted object for further processing. The highlighted graphical object may also be displayed with a bounding box to further identify the selected object(s). A moving version of the highlighted graphical object may be drawn during a moving process, in which the moving version has an appearance distinct from the original object.

Abstract: The present invention relates to interfacing with electronic ink. Ink is stored in a data structure that permits later retrieval by applications. The ink includes stroke information and may include property information. Through various programming interfaces, one may interact with the ink through methods and setting or retrieving properties. Other objects and collections may be used as well in conjunction with the ink objects.

Abstract: A natural input system is described for creating and editing complex structures in a typeset application. The natural input system receives a typeset representation of an object and converts the typeset format to generate a standard digital ink representation. The natural input system provides the generated ink representation to a natural input application where can be manipulated by the user with a rich set of correction and editing features provided by the natural input application. Once the end user is satisfied with the recognition result in the natural input application, the natural input system receives the recognition result based on the modified digital ink representation. The natural input system may convert the received recognition result to the typeset application format and provides the modified typeset representation to the typeset application for merging into the document the user is editing.

Abstract: A number of regions and partitions may be created based on input handwritten atoms and a grammar parsing framework. Productions for tabular structures may be added to the grammar parsing framework to produce an extended grammar parsing framework. Each of the regions may be searched for a tabular structure. Upon finding a tabular structure, a type of tabular structure may be determined. Configuration partitions may be created, based on the added productions, and added to the created partitions. A set of configuration regions may be created based on the configuration partitions and added to the created regions. The productions for tabular structures and productions of the grammar parsing framework may be applied, as rewriting rules, to the atoms to produce possible recognition results. A best recognition result may be determined and displayed. A mechanism for correcting misrecognition errors, which may occur while recognizing tabular structures, may be provided.

Abstract: A processing device may recognize a number of input handwritten strokes, which may represent a mathematical expression, a chemical formula, or other two-dimensional structure. Rewriting rules of a grammar may be applied to the strokes to produce a number of possible recognition results. Each of the possible recognition results has a respective score based on a sum of rewriting rules applied to the strokes to produce respective ones of the possible recognition results. Input may be provided to identify misrecognized strokes and a correct terminal production, or symbol corresponding to the misrecognized strokes. Strokes may be misrecognized for many reasons, including parsing errors, over-grouping or under-grouping of matrices, and improper placement of a recognized terminal production, or symbol, with respect to a root structure. Correction hints may be leveraged for correcting types of errors mentioned above.

Abstract: Computer-readable media, systems, and methods for document layout extraction are described. In embodiments, textual data in an electronic format is received and the textual data is converted from the electronic format to an independent interface format, the independent interface format including coordinates to one or more structural elements of the textual data. Further, in embodiments, a structure and layout analysis of the textual data is performed to generate a set of structure and layout information. Still further, in embodiments, the textual data and the set of structure and layout information is stored in an enriched interface format, the enriched interface format providing for search and navigation of the textual data.

Abstract: Embodiments of the present invention relate to classifying pages of an electronic document, such as a scanned book page. OCR software is applied to the contents of the electronic document, revealing semantic information about the content of the electronic document. Software-based features are applied to the semantic information to determine the type of page the electronic document is. Page types may include table of contents (TOC), table of figures (TOF), bibliography, index, or other types of pages commonly found in a book, magazine, or other publication. Once determined, the determined page type is stored and used by other software engines.

Abstract: Computer-storage media, computerized methods and systems for classifying character strings within electronic documents are provided. Initially, textual data, which includes one or more character strings, is extracted from an electronic version of a document, typically scanned from a physical document utilizing optical character recognition. The textual data is received at a table-of-contents (TOC) engine that extracts semantic information from the textual data. Sub-engines within the TOC engine analyze the semantic information to determine at least one appropriate classification for character strings within the textual data. Labels selected from a predetermined set of TOC-architecture labels are appended to the character strings according to the appropriate classification. The character strings, and labels appended thereto, are stored in association with each other generating an electronic document file that includes enriched textual data.

Abstract: Systems, methods, and computer-readable media for making rich, flexible, and more natural electronic ink annotations in an electronic document include creating a first context node associated with a first portion of a base portion of an electronic document; creating a second context node associated with an annotation to the base portion; and linking the second context node with the first context node.

Abstract: An application programming interface instantiates an ink analyzer object that receives document data for a document containing electronic ink content from a software application hosting the document and running on a first processing thread. The ink analyzer object then employs the first thread to make a copy of the document data, provides the copy of the document data to an electronic ink analysis process, and returns control of the first processing thread to the analysis process. After the analysis process has analyzed the electronic ink, the ink analyzer object reconciles the results of the analysis process with current document data for the document.

Abstract: In embodiments consistent with the subject matter of this disclosure, a user may input strokes as digital ink to a processing device. The processing device may partition the input strokes into multiple regions of strokes. A first recognizer and a second recognizer may score grammar objects included in regions and represented by chart entries. The scores may be converted to a converted score, which may have at least a near standard normal distribution. The processing device may present a recognition result based on highest converted scores according to a recurrence formula. The processing device may receive a correction hint with respect to misrecognized strokes and may add a penalty score with respect to chart entries representing grammar objects breaking the correction hint. Incremental recognition may be performed when a pause is detected during inputting of strokes.

Abstract: In embodiments consistent with the subject matter of this disclosure, a user may input one or more strokes as digital ink to a processing device. The processing device may produce and present a recognition result, which may include a misrecognized portion. A user may indicate a desire to correct the misrecognized portion and may further select one or more strokes of the misrecognized portion. The processing device may then present the one or more recognition alternates corresponding to the selected one or more strokes of the misrecognized portion. In some embodiments, the processing device may permit a user to rewrite the selected one or more strokes of the misrecognized portion with newly entered digital ink. Features, such as, rewriting and correction of the input digital ink may be discoverable in some embodiments.

Abstract: A processing device may parse a group of strokes representing a mathematical expression. The group of strokes may be examined to determine whether the group of strokes satisfies any of a finite set of rules. When the group of strokes, included in a region, satisfies any of the finite set of rules, the region may be partitioned according to a satisfied one of the finite set of rules. The group of strokes included in the region may be further examined to determine whether the group of strokes may be further partitioned according to any of the finite set of rules. After all regions have been examined and no further partitioning of regions may be performed, all mathematical symbols of the mathematical expression may be isolated in at least some of the regions and may be recognized.

Abstract: The present invention relates to storing electronic ink. Ink is stored in a data structure that permits later retrieval by applications. The ink includes stroke information and property information. Properties may be associated globally with the ink strokes and/or with each stroke individually. The disclosed embodiments include a data structure that supports ink for multiple applications. Using the disclosed storage system, method, and data structure, various applications may expand on ink to include additional attributes while permitting the ink to be used in applications not supporting the additional attributes. When encountering an unknown tag, the applications will skip past the end of the tag and its associated data to start reading the next tag.

Abstract: Apparatus and methods for dynamically rendering transparent ink strokes, in some situations such that the rendered ink stroke has transparency similar to physical ink while it is being drawn. For example, the ink stroke may be dynamically rendered as a stroke having uniform transparency while it is being drawn. Only the new ink segment that has most recently been added to the stroke may be drawn, and areas of the new ink segment that overlap older segments of the ink stroke may be frozen, or excluded from being re-painted.